Take the 2-minute tour ×
Cognitive Sciences Stack Exchange is a question and answer site for practitioners, researchers, and students in cognitive science, psychology, neuroscience, and psychiatry. It's 100% free, no registration required.

I am asking a question regarding parallel processing as done by billions of Neurons inside our brain and parallel processing done by our computers in a cluster for example or even on a Graphics processor having 500 processing cores.

Recently we have seen great success in applying computers to solve complex problems applying parallel processing, such as in the field of scientific simulations, modeling, video/image processing etc. Also due to limitations on semiconductor devices, chip makers are now forced to develop multi-core CPUs. This encourages and motivates software developers to apply parallel programming to gain the required performance, exploiting all the processing cores inside the CPU.

While people are encouraging to solve problems using parallel programming on the computers, at the same time generally discouraging to do parallel processing inside the brain. Expert say that we must concentrate in one dimension to get best results. Why can't human brains be used to do massive parallel processing in the same way computers are doing today? The brain is having billions of computational units called Neurons, and thus it provides an opportunity to do processing at both coarser level and at finer level. Why can't we use our brain to do certain jobs in parallel in the same way as computers, for example, while I am writing this question using my right hand, why cant I solve a problem in geometry at the same time, using my left hand? Using much less powerful computers having just two processing cores (vs billions in brain) I can do these two jobs of writing and solving geometry problem, in parallel by launching two separate threads. Why it is so?

share|improve this question
    
One thing I can answer: comparing a neuron to a computational unit seems wrong to me. Rather a set of neurons could make up a computational unit to some degree. –  Steven Jeuris Nov 30 '12 at 10:07
    
It might also be interesting to look into some papers discussing multitasking, I'm quite certain plenty of research has been done related to that. –  Steven Jeuris Nov 30 '12 at 10:14
    
A fantastic paper about the nature of the universe and how we think of it as a "computer" but probably shouldn't - arxiv.org/abs/1211.7081 As a graduate student also working in neuroscience, I found its overall message equally applicable to the brain –  Diego Dec 11 '12 at 19:07
    
I would like to add the idea that even computers can not break down some tasks to multiple threads. Tasks which are made up of many individual calculations can be broken down to be operated on by many processors. Rendering, filtering, modeling are all amenable to multi core processing. Individual calculations can't be broken down, so a process that consists of many calculations serially can't be broken down. One of the things that can only occupy one thread is communicating with the user. –  timquinn Dec 14 '12 at 11:34
    
This would suggest the bottleneck is the fact that you have only one consciousness. The brain does use multi-threading, I assume, when image processing or modeling the future, but we only get the results posted to the conscious mind as images or emotions. The conscious mind can only do one thing at a time and our sense of ourselves is the sum of those things we are conscious of being in control of. –  timquinn Dec 14 '12 at 11:34

4 Answers 4

I think you are beginning to push at the limits of human brain versus computer metaphor.(mildly related link here) I'll list the objections as:

  1. While neurons firing can be translated/compared to 0 or 1 states, am not convinced it is a valid equivalence. (i.e: simulating 100 billion neurons on a computer would still not be able to match a human brain). My suspicions are while we have gone through a very accelerated evolution in building and controlling the performance of transistors, evolution hasn't progressed to that stage in terms of brain development.

  2. Our neuronal cells do exhibit variations(range of electrical activity, not just spikes and flats) and in some sense you can call procedural learnt tasks to be parallel processing. i.e: you can speak and drive to some extent etc..Not to mention the Autonomic nervous system and it's functionalities like breathing, heart beat etc..Here's the paper i was referring to. My point being this can be considered as parallel processing.

  3. Another ignored factor is the resolution at which we can control/program processors to compute and co-ordinate with each other on a single problem. Parallel processing is hardly a solved problem in the programming world.

  4. Yet another factor that is ignored is that if you do zoom down on the computers to the level of capacitors, they are not all binary and have some analog activities too.This article mentions it, though it goes on to make a different point.

  5. Another relevant point is that there are algorithms that cannot be sped up by using parallel computing. For ex: iterative algorithms like Newton's method, in these cases the next computation depends on the immediately preceding computation and so has to wait till the previous computation is done.

So to summarize, our brains do some forms of parallel processing already.It may just be that the jobs of writing and solving geometry problem have a set of processes too much in common that they cannot be done parallel.

share|improve this answer
    
Nice of you to point out the brain actually does multitask, of which breathing and controlling muscles are very good examples. You make some interesting comments, but I can't up vote this as an answer. –  Steven Jeuris Nov 30 '12 at 10:17
1  
I know, i initially wrote the whole thing up as a comment, but exceeded word limit. will perhaps come back when i have time to search and add proper citations. –  Anand Jeyahar Nov 30 '12 at 10:37
    
Our neuronal cells do exhibit variations(range of electrical activity, not just spikes and flats) this is an interesting point, as graded potentials are important in smaller organisms like insects, I don't think it's known what their role in humans is, yet, though. –  Chuck Sherrington Nov 30 '12 at 11:30
    
I agree 100% with your underlying thesis (that the brain/computer analogy becomes ever more strained as we learn more). I'm with Steven, though, in that you just need to flesh out the points a little bit. –  Chuck Sherrington Nov 30 '12 at 11:32
1  
Chuck, graded potentials would be important in gap-junction coupling, probably partiuclarly axo-axonic gap junction coupling (which occurs in the dentate gyrus of the hippocampus). They're mostly known for their role in network synchronization. But also, underlying membrane potential is always fluxuation due to passive currents both in and outside the cell; thresholds are kind of a fuzzy value that can be altered by global activity in the body. –  Keegan Keplinger Dec 4 '12 at 6:15

This probably won't be a very good answer, but I studied this a bit in a Human Factors class I took this semester. The metaphor that the professor used is a bucket of mental resources. We have several buckets for certain things. When we are using resources from the bucket the bucket has less resources for other things.

We can walk and talk at the same time because they are using different buckets( motor-physical and language processing ). However we are very bad at reading and talking to someone at the same time because both task are taking up mental resources from the same bucket( language processing ).

So, yes we are able to do various things well that aren't requiring resources from the same bucket, but these buckets are only so deep.

Maybe that makes since. Oh yeah there's a more technical term, maybe Multi-Channel Human Information Processing. I know if you researched Human Information Processing( HIP ) you would stumble upon what you may be looking for.

share|improve this answer

There is a passage in On intelligence about the differences between parallel processing in human versus computers :

From the dawn of the industrial revolution, people have viewed the brain as some sort of machine. They knew there weren't gears and cogs in the head, but it was the best metaphor they had. Somehow information entered the brain and the brain-machine determined how the body should react. During the computer age, the brain has been viewed as a particular type of machine, the programmable computer. And as we saw in chapter 1, AI researchers have stuck with this view, arguing that their lack of progress is only due to how small and slow computers remain compared to the human brain. Today's computers may be equivalent only to a cockroach brain, they say, but when we make bigger and faster computers they will be as intelligent as humans.

There is a largely ignored problem with this brain-as-computer analogy. Neurons are quite slow compared to the transistors in a computer. A neuron collects inputs from its synapses, and combines these inputs together to decide when to output a spike to other neurons. A typical neuron can do this and reset itself in about five milliseconds (5 ms), or around two hundred times per second. This may seem fast, but a modern silicon-based computer can do one billion operations in a second. This means a basic computer operation is five million times faster than the basic operation in your brain! That is a very, very big difference. So how is it possible that a brain could be faster and more powerful than our fastest digital computers? "No problem," say the brain-as-computer people. "The brain is a parallel computer. It has billions of cells all computing at the same time. This parallelism vastly multiplies the processing power of the biological brain."

I always felt this argument was a fallacy, and a simple thought experiment shows why. It is called the "one hundred–step rule." A human can perform significant tasks in much less time than a second. For example, I could show you a photograph and ask you to determine if there is cat in the image. Your job would be to push a button if there is a cat, but not if you see a bear or a warthog or a turnip. This task is difficult or impossible for a computer to perform today, yet a human can do it reliably in half a second or less. But neurons are slow, so in that half a second, the information entering your brain can only traverse a chain one hundred neurons long. That is, the brain "computes" solutions to problems like this in one hundred steps or fewer, regardless of how many total neurons might be involved. From the time light enters your eye to the time you press the button, a chain no longer than one hundred neurons could be involved. A digital computer attempting to solve the same problem would take billions of steps. One hundred computer instructions are barely enough to move a single character on the computer's display, let alone do something interesting.

But if I have many millions of neurons working together, isn't that like a parallel computer? Not really. Brains operate in parallel and parallel computers operate in parallel, but that's the only thing they have in common. Parallel computers combine many fast computers to work on large problems such as computing tomorrow's weather. To predict the weather you have to compute the physical conditions at many points on the planet. Each computer can work on a different location at the same time. But even though there may be hundreds or even thousands of computers working in parallel, the individual computers still need to perform billions or trillions of steps to accomplish their task. The largest conceivable parallel computer can't do anything useful in one hundred steps, no matter how large or how fast.

Here is an analogy. Suppose I ask you to carry one hundred stone blocks across a desert. You can carry one stone at a time and it takes a million steps to cross the desert. You figure this will take a long time to complete by yourself, so you recruit a hundred workers to do it in parallel. The task now goes a hundred times faster, but it still requires a minimum of a million steps to cross the desert. Hiring more workers— even a thousand workers— wouldn't provide any additional gain. No matter how many workers you hire, the problem cannot be solved in less time than it takes to walk a million steps. The same is true for parallel computers. After a point, adding more processors doesn't make a difference. A computer, no matter how many processors it might have and no matter how fast it runs, cannot "compute" the answer to difficult problems in one hundred steps.

So how can a brain perform difficult tasks in one hundred steps that the largest parallel computer imaginable can't solve in a million or a billion steps? The answer is the brain doesn't "compute" the answers to problems; it retrieves the answers from memory. In essence, the answers were stored in memory a long time ago. It only takes a few steps to retrieve something from memory. Slow neurons are not only fast enough to do this, but they constitute the memory themselves. The entire cortex is a memory system. It isn't a computer at all.

The point made here is that the computing paradigm (that is, the way the whole thing works) of the brain and the computer are completely different. The computer is a Turing machine, and the brain is something else, possibly a memory system if you think that Jeff Hawking is right. Whatever it is, the brain is not a Turing machine.

To go back to your question:

Why can't human brains be used to do massive parallel processing in the same way computers are doing today?

It has to do with the way the human brain works. If you assume that the brain will do any task in a parallel fashion, and the more neurons involved, the better the performance; then in order to maximize your performance you should use your whole brain. 1 task: 100% performance, 2 tasks: 50% performance, 3 tasks: 33% performance, and so on.

But if you add an "attention switching cost" to go from one task to another, then you are better off just focusing on one task where the switching cost is zero.

So you can multitask, but it won't be efficient.

share|improve this answer

Maybe I just don't get it, but I see your question as confusing because:

1/ Your brain is capable of running multiple parallel processes. Actualy each one of tasks you've mentioned consists of number of processes that are done at the same time. Lots of your neurons and neuronal networks are being used at the very same moment.

2/ If you can do something automatically, without thinking about it, but not without using our brain to guide the action (there are very few things you can do without actually using your brain and its neurons), you will find that you actually are able to do even more complex tasks in parallel. Even the two you've mentioned.

3/ So the problem of you not being able to do some tasks together is a problem of you not being able to concentrate at both of them as good, as you do with one task. Your question asks for the difference using a metaphor and comparison of the 'hardware' of your brain, but the problem you are asking about goes on higher cognition. You not being able to effectively 'double' your attention, you having limited working memory capacity.

share|improve this answer
    
Thanks for observations you made. "...you having limited working memory capacity. " any reference on this, and that this is the reason "...I am not able to effectively 'double' my attention"? –  gpuguy Dec 28 '12 at 7:01

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.