5
$\begingroup$

This is an AI question regarding "3rd generation neural networks" - spiking neural networks (SNN).

I hve been studying this concept online from various papers, mainly Maass (1997). I and am not entirely sure I understand why SNN's are considered pulse-code in contrast to earlier ANN's which are rate-code.

I have background in neuroscience so I understand the terms and ratio, I am asking regarding the actual implementation.

Is the practical difference in the fact that when each neuron updates its current state in an SNN it deals with the entire history of every pre-synaptic neuron and not only the last step? Is that what gives it temporal characteristics which previous generation ANN's lack? What is the key computational difference between SNN and earlier approaches?

Notes

This question was migrated from SO, and a duplicate was asked on cstheory that was migrated to CS.SE.

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Apparently this question has been cross-posted on our Computer Science site. $\endgroup$
    – Josh
    Aug 15, 2012 at 13:19
  • 4
    $\begingroup$ Generally at Stack Exchange this is heavily frowned upon; questions should exist on one site only. Amir, please make sure to keep both questions updated, and when you accept an answer on one site, be sure to answer the question yourself on the other site, and link the two together. And please avoid cross-posting in the future. Thank you! $\endgroup$
    – Josh
    Aug 15, 2012 at 13:19
  • $\begingroup$ Sorry, I wasn't aware of the policy. I just thought the question was in the gray area between the two fields. I will maintain both and link a final answer if I reach it. $\endgroup$
    – Amir
    Aug 16, 2012 at 8:08

1 Answer 1

Reset to default
2
$\begingroup$

Rate-based-network is a specific implementation of the more general spiking-networks. One may see a rate-based-network as a spiking-network in which the inputs from each neuron are accumulated over a short time-period (think of "one second") and are used to update the state of their target neuron only once in each time-period.

For example, if an input neuron has an output of '40' in a rate-based-network, you should think about it as if it fire 40 times in the time-period, and only once in each time-period it's target neuron 'read' this input.

On the other hand, with spiking-neurons the output is limited to binary and hence the time-period used must be shorter than the maximal rate. In each time-period the target neuron updates its state based on it's (binary) input.

So to summarize:
* Spiking networks describe a more general phenomena.
* Spiking networks can react to inputs with fine temporal structure and act in time-periods which are order of magnitude smaller than rate-based-models.

Also one may add that spiking networks are likely to be more accurate biological description of the brain (but still a very rough account).

Improve this answer
$\endgroup$
4
  • $\begingroup$ Let's take a simple perceptron also with a binary output. From your description it sounds like he too can be interpreted as pulse-code, it's all a question of which time-scale you attribute each iteration of the net (each time-period). However, IAF neurons are considered to have different abilities than perceptrons, namely the ability to find temporal patterns in data. So regardless of interpretation, there should be some essential difference in implementation to account for that. Am I missing something? Thanks a lot Uri $\endgroup$
    – Amir
    Mar 14, 2012 at 15:52
  • $\begingroup$ Let consider a network which receive inputs at M time-periods from N inputs. Then for each of those time-periods you may think of the network as being a perceptron; but the input-output relation on all M periods, where there are 2^M different outputs, is inherently different than having M independent perceptrons, because of the non-trivial interactions between them. $\endgroup$
    – Uri Cohen
    Mar 15, 2012 at 8:27
  • $\begingroup$ OK, I think I understand that. But lets take that network, with the exact same structure, and swap the neuron implementation from IAF to perceptron. Have 2 copies of the same topology and running time (M time, N inputs) with a different implementation for the neurons. Why would one be pulse code and the other rate code? $\endgroup$
    – Amir
    Mar 19, 2012 at 19:39
  • $\begingroup$ Those are just names, which indicate each time-period is processed independently (in perceptron) of interact with each other through change of the network state (the 'voltage', in IAF). $\endgroup$
    – Uri Cohen
    Mar 19, 2012 at 21:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.