Take the 2-minute tour ×
Cognitive Sciences Stack Exchange is a question and answer site for practitioners, researchers, and students in cognitive science, psychology, neuroscience, and psychiatry. It's 100% free, no registration required.

This is an AI question regarding "3rd generation neural networks" - spiking neural networks (SNN).

I hve been studying this concept online from various papers, mainly Maass (1997). I and am not entirely sure I understand why SNN's are considered pulse-code in contrast to earlier ANN's which are rate-code.

I have background in neuroscience so I understand the terms and ratio, I am asking regarding the actual implementation.

Is the practical difference in the fact that when each neuron updates its current state in an SNN it deals with the entire history of every pre-synaptic neuron and not only the last step? Is that what gives it temporal characteristics which previous generation ANN's lack? What is the key computational difference between SNN and earlier approaches?

Notes

This question was migrated from SO, and a duplicate was asked on cstheory that was migrated to CS.SE.

share|improve this question

migrated from stackoverflow.com Aug 15 '12 at 12:56

This question came from our site for professional and enthusiast programmers.

1  
4  
Generally at Stack Exchange this is heavily frowned upon; questions should exist on one site only. Amir, please make sure to keep both questions updated, and when you accept an answer on one site, be sure to answer the question yourself on the other site, and link the two together. And please avoid cross-posting in the future. Thank you! –  Josh Gitlin Aug 15 '12 at 13:19
    
Sorry, I wasn't aware of the policy. I just thought the question was in the gray area between the two fields. I will maintain both and link a final answer if I reach it. –  Amir Aug 16 '12 at 8:08

1 Answer 1

Rate-based-network is a specific implementation of the more general spiking-networks. One may see a rate-based-network as a spiking-network in which the inputs from each neuron are accumulated over a short time-period (think of "one second") and are used to update the state of their target neuron only once in each time-period.

For example, if an input neuron has an output of '40' in a rate-based-network, you should think about it as if it fire 40 times in the time-period, and only once in each time-period it's target neuron 'read' this input.

On the other hand, with spiking-neurons the output is limited to binary and hence the time-period used must be shorter than the maximal rate. In each time-period the target neuron updates its state based on it's (binary) input.

So to summarize:
* Spiking networks describe a more general phenomena.
* Spiking networks can react to inputs with fine temporal structure and act in time-periods which are order of magnitude smaller than rate-based-models.

Also one may add that spiking networks are likely to be more accurate biological description of the brain (but still a very rough account).

share|improve this answer
    
Let's take a simple perceptron also with a binary output. From your description it sounds like he too can be interpreted as pulse-code, it's all a question of which time-scale you attribute each iteration of the net (each time-period). However, IAF neurons are considered to have different abilities than perceptrons, namely the ability to find temporal patterns in data. So regardless of interpretation, there should be some essential difference in implementation to account for that. Am I missing something? Thanks a lot Uri –  Amir Mar 14 '12 at 15:52
    
Let consider a network which receive inputs at M time-periods from N inputs. Then for each of those time-periods you may think of the network as being a perceptron; but the input-output relation on all M periods, where there are 2^M different outputs, is inherently different than having M independent perceptrons, because of the non-trivial interactions between them. –  Uri Cohen Mar 15 '12 at 8:27
    
OK, I think I understand that. But lets take that network, with the exact same structure, and swap the neuron implementation from IAF to perceptron. Have 2 copies of the same topology and running time (M time, N inputs) with a different implementation for the neurons. Why would one be pulse code and the other rate code? –  Amir Mar 19 '12 at 19:39
    
Those are just names, which indicate each time-period is processed independently (in perceptron) of interact with each other through change of the network state (the 'voltage', in IAF). –  Uri Cohen Mar 19 '12 at 21:16

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.