Mental Thrashing
Dear [Mentat],
Yes, people often waste thier brain power accomplishing too little, by spending too much time engaged in mental thrashing. They work one task for a short time, then switch to another, and then another, and so on. Of course, the switching itself takes time for the person to “ramp down” from the first task, and to “ramp up” on the next one. Consider that if we switch tasks too frequently, then we end up accomplishing very little work in any of them, and spend more of our mental energy doing this mental thrashing than actually completing the jobs at hand.
In fact, mental thrashing is similar to a term found term in computer technology literature, known basically as thrashing. To understand how computer thrashing very closely resembles mental thrashing in humans, consider that the switching of context from one program to the next in the computer requires a certain amount of overhead. It requires memory to remember where it last executed in the previous program, and where to resume execution in the next program once the context switch is completed. Put simply, the code to be executed next must be loaded into main memory from the paging file. The stack and CPU registers must be properly initialized or restored to what they were the last time this program ran. Then the program counter must be set to point to the next instruction to execute in the newly loaded program pages. Once all this is done, the next program finally runs.
Now without going into detail about all the reasons that program execution stops so the CPU can execute other programs, suffice it to say that each program executes for a rather small period of time (called a time slice). When it’s time to jump to the next program in the run queue, the CPU has to switch context again, and again incur the same overhead just described, to get the next program running. The shorter the amount of time that the CPU spends actually running programs before it must switch, the more of its time it spends switching. Hopefully it’s clear that when programs run for very small slices before relinquishing control, that the amount of real work done by the CPU on each program’s behalf can approach zero. When the CPU spends too much time switching context and very little actually executing programs, we refer to this condition as thrashing. Needless thrashing is undesirable in computers because it can excessively burden the CPU and drastically impede the computer’s real-work potential.
Mental thrashing likewise, has the same wasteful effects in the brain. Too much switching from one thought to the next as you point out, impedes one’s ability to effectively solve problems. If you have to constantly remember where you were the last time you worked on a particular problem, you have to repeatedly refresh your memory, just like the CPU does when it restores context of a program it’s about to resume executing. This excessive refreshing (or mental thrashing) can be frustrating and debilitating. So yes, the longer we can concentrate on a single issue before being interrupted by a more urgent other one, the more effective the work we can accomplish, within limits of course. To maximize our effectiveness as workers we must ensure that we do not allow our mental thrashing to get out of control and consume too much of our daily mental bandwidth.
But in the quest to eliminate mental thrashing, becoming too single-minded can impede our mental productivity as well. Again, allow me to draw on the computer example. While it’s good in the computer to assure that the CPU spends sufficient time executing each program, too much time spent in one has a detrimental effect on the rest of the programs in the system. We used to see the bad effects of this in older versions of Microsoft Windows, where insufficient safeguards allowed one program to completely take over the CPU so that the rest of them were starved and did not execute at all. This swamping would appear as system hang-ups, non responsiveness, slow responsiveness, or strange responsiveness. It would typically require a reboot to clear. But nowadays, with the obsolescence of the 16-bit real mode programs, and with the emergence of the 32-bit protected mode programs, it’s much more difficult for errant software to overly dominate the system.
Indeed, there would be negative impacts of concentrating too long on a single problem in the brain as well. But as you say, very few people likely ever encounter these [this extreme single mindedness]. 
They are more likely to suffer the effects of mental thrashing instead.
Your analogy is a good one, yes, particularly since in it, you, though perhaps you didn’t realize it, described the basic principles of operation behind the time-shared computer systems such as found in Windows, UNIX, and most any other multi-tasking operating system. In fact, if the brain indeed resembles a computer CPU, then to maximize the real work it does, we should avoid mental thrashing where possible but keep in mind that periodic task switching is also a good thing, so long as we do not overdo it, and allow it to degenerate into the wasteful condition of mental thrashing.