Neuromorphic VLSI Event-Based devices and systems

Transcription

1 Neuromorphic VLSI Event-Based devices and systems Giacomo Indiveri Institute of Neuroinformatics University of Zurich and ETH Zurich LTU, Lulea May 28, 2012 G.Indiveri ( Neuromorphic spiking chips 1 / 38

2 Outline 1 Neuromorphic Engineering 2 Spike-based sensory systems 3 Spiking Neural Networks Silicon Neurons Silicon synapses Winner-Take-All networks Multi-chip networks 4 Learning 5 Neuromorphic Cognitive Systems G.Indiveri ( Neuromorphic spiking chips 2 / 38

3 Natural Computation The Honeybee The brain of the worker honeybee occupies a volume of around 1mm 3 and weighs about 1mg. The total number of neurons in the brain is estimated to be 950,000 Flies acrobatically Recognizes patterns Navigates Forages Communicates G.Indiveri ( Neuromorphic spiking chips 3 / 38

4 Natural Computation The Honeybee The brain of the worker honeybee occupies a volume of around 1mm 3 and weighs about 1mg. The total number of neurons in the brain is estimated to be 950,000 Energy consumption: J/op, at least 10 6 more efficient than digital silicon (20watts vs. 1Mil.watts) Flies acrobatically Recognizes patterns Navigates Forages Communicates G.Indiveri ( Neuromorphic spiking chips 3 / 38

5 Neocortex Neural computation Silicon Behavior Synaptic Inputs Constant current Synapse V mem (V) Soma G.Indiveri ( Neuromorphic spiking chips Time (s) / 38

6 Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics of silicon to reproduce the bio-physics of neural systems, using subthreshold analog VLSI circuits. G.Indiveri ( Neuromorphic spiking chips 5 / 38

7 Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics of silicon to reproduce the bio-physics of neural systems, using subthreshold analog VLSI circuits. To develop multi-chip spike-based computing systems, using the Address-Event Representation (AER) and asynchronous digital VLSI technology. G.Indiveri ( Neuromorphic spiking chips 5 / 38

8 Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics of silicon to reproduce the bio-physics of neural systems, using subthreshold analog VLSI circuits. To develop multi-chip spike-based computing systems, using the Address-Event Representation (AER) and asynchronous digital VLSI technology. To automatically configure and program neuromorphic processing systems distributed across multiple chips, to carry out real time behavioral tasks. G.Indiveri ( Neuromorphic spiking chips 5 / 38

9 Outline 1 Neuromorphic Engineering 2 Spike-based sensory systems 3 Spiking Neural Networks Silicon Neurons Silicon synapses Winner-Take-All networks Multi-chip networks 4 Learning 5 Neuromorphic Cognitive Systems G.Indiveri ( Neuromorphic spiking chips 6 / 38

10 AER silicon retinas Tobi Delbruck G.Indiveri ( Neuromorphic spiking chips 7 / 38

11 Silicon retina properties (movie) G.Indiveri ( Neuromorphic spiking chips 8 / 38

12 An AER silicon cochlea Shih-Chii Liu G.Indiveri ( Neuromorphic spiking chips 9 / 38

13 Silicon cochlea properties G.Indiveri ( Neuromorphic spiking chips 10 / 38

14 Silicon cochlea properties G.Indiveri ( Neuromorphic spiking chips 10 / 38

15 Outline 1 Neuromorphic Engineering 2 Spike-based sensory systems 3 Spiking Neural Networks Silicon Neurons Silicon synapses Winner-Take-All networks Multi-chip networks 4 Learning 5 Neuromorphic Cognitive Systems G.Indiveri ( Neuromorphic spiking chips 11 / 38

16 Implement neural computation in silicon Classical neural networks w 2 w 1 Pre-synaptic Inputs w 4 w 3 Post-synaptic Output w n Neuromorphic multi-neuron networks Post-synaptic Output w 1 w 2 w 3 w 4 w n Pre-synaptic Inputs G.Indiveri ( Neuromorphic spiking chips 12 / 38

17 Spiking multi-neuron architectures Networks of silicon neurons with adaptation, refractory period, etc. Silicon synapses with realistic temporal dynamics Winner-Take-All architectures Spike-based plasticity mechanisms G.Indiveri ( Neuromorphic spiking chips 13 / 38

18 Outline 1 Neuromorphic Engineering 2 Spike-based sensory systems 3 Spiking Neural Networks Silicon Neurons Silicon synapses Winner-Take-All networks Multi-chip networks 4 Learning 5 Neuromorphic Cognitive Systems G.Indiveri ( Neuromorphic spiking chips 14 / 38

19 Silicon neurons The low-power adaptive exponential I&F neuron Adaptation Positive Feedback M 5 M 11 M 14 I in V ahp M 12 V rest DPI M 6 M 18 M 1 V thra I fb M 7 M 8 M 15 M 19 V thr V mem M 2 M 3 C mem I mem V spk M 16 M 20 M 22 I ahp V rf Leak C ahp DPI M 17 M 21 M 9 V tau M 4 V taua M 10 M 13 Refractory Period τ d dt I mem + I mem I gi in I τ + f (I mem ) [Indiveri et al., ISCAS 2010] G.Indiveri ( Neuromorphic spiking chips 15 / 38

20 The low-power I&F neuron Positive Feedback data fit I mem /I Time (ms) G.Indiveri ( Neuromorphic spiking chips 16 / 38

21 The low-power I&F neuron Spike frequency adaptation Instantaneous firing rate (Hz) Spike count G.Indiveri ( Neuromorphic spiking chips 17 / 38

22 The low-power I&F neuron Basic response properties Leaky I&F model F-F curve (note mismatch) 1 Membrane potential (V) Time (s) G.Indiveri ( Neuromorphic spiking chips 18 / 38

23 Outline 1 Neuromorphic Engineering 2 Spike-based sensory systems 3 Spiking Neural Networks Silicon Neurons Silicon synapses Winner-Take-All networks Multi-chip networks 4 Learning 5 Neuromorphic Cognitive Systems G.Indiveri ( Neuromorphic spiking chips 19 / 38

24 Synapses Real synapses Artificial synapses w 1 w 2 Pre-synaptic Inputs w 4 w 3 Post-synaptic Output w n Synapses are often modeled as instantaneous multipliers. Science and Engineering Visualization Challenge 2005 winner, Graham Johnson, Medical Media, Boulder, Colorado. G.Indiveri ( Neuromorphic spiking chips 20 / 38

25 The diff-pair integrator (DPI) circuit V thr M thr V τ M τ I τ I in M in V syn C syn M syn I syn I syn (t) = I 0 e κ (V U syn (t) V dd ) T I thr = I 0 e κ(v thr V dd ) U T C syn d dt V syn = (I in I τ ) V w M w I w M pre τ d dt I syn + I syn = I thr I w I τ [Bartolozzi and Indiveri, Neural Computation, 2007] G.Indiveri ( Neuromorphic spiking chips 21 / 38

26 The DPI synapse Temporal dynamics EPSC (na) V w =300mV V w =320mV V w =340mV Time (s) G.Indiveri ( Neuromorphic spiking chips 22 / 38

27 Outline 1 Neuromorphic Engineering 2 Spike-based sensory systems 3 Spiking Neural Networks Silicon Neurons Silicon synapses Winner-Take-All networks Multi-chip networks 4 Learning 5 Neuromorphic Cognitive Systems G.Indiveri ( Neuromorphic spiking chips 23 / 38

28 Recurrent cooperative-competitive architectures Hardwired local synapses Local excitatory connections Global inhibitory connections [Chicca et al., Nips, 2006] G.Indiveri ( Neuromorphic spiking chips 24 / 38

29 Local recurrent connectivity Winner-take-All architectures Input Stimulus AER INPUT Y AER INPUT X AER OUTPUT Neuron address Time (s) Mean f (Hz) Input signals are encoded with mean firing rates Computation and information transfer is data driven G.Indiveri ( Neuromorphic spiking chips 25 / 38

30 Local recurrent connectivity Winner-take-All architectures Feedforward Network Neuron address Time (s) Mean f (Hz) Without local connectivity activated output spike rates represent linearly input spike rates (modulo mismatch effects) G.Indiveri ( Neuromorphic spiking chips 25 / 38

31 Local recurrent connectivity Winner-take-All architectures Feedback Network Neuron address Time (s) Mean f (Hz) With local WTA connectivity the network exhibits: Selective amplification Signal normalization Signal restoration [Chicca et al., Nips, 2006] G.Indiveri ( Neuromorphic spiking chips 25 / 38

32 Outline 1 Neuromorphic Engineering 2 Spike-based sensory systems 3 Spiking Neural Networks Silicon Neurons Silicon synapses Winner-Take-All networks Multi-chip networks 4 Learning 5 Neuromorphic Cognitive Systems G.Indiveri ( Neuromorphic spiking chips 26 / 38

33 Spikes and Address-Event Systems V mem (V) Encode Decode Time (s) Address Event Bus Inputs Source Chip Address-Event representation of action potential Outputs Destination Chip Action Potential G.Indiveri ( Neuromorphic spiking chips 27 / 38

34 least some feedback (see Box 2). Thus, generalization in the brain can emerge from the linear combination of neurons tuned to an optimal stimulus effectively defined by multiple dimensions25,23,26. This is a powerful extension of the older computation-through-memory models of vision and motor control. The question now is whether the available evidence supports the existence of a similar architecture underlying generalization in domains other than vision. Hierarchical or multi-layer networks AER INPUT Y Categ. Ident. AER OUTPUT AIT AER INPUT X PFC IT V4/PIT V4 V1 V1 Figure 2 A model of visual learning. The model summarizes in quantitative terms other models and many data about visual recognition in the ventral stream pathway in cortex. The correspondence between the layers in the model and visual areas is an oversimplification. Circles represent neurons and arrows represent connections between them; the dots signify other neurons of the same type. Stages of neurons with bell-shaped tuning (with black arrow inputs), that provide example-based G.Indiveri ( Neuromorphic spiking chips 28 / 38

35 least some feedback (see Box 2). Thus, generalization in the brain can emerge from the linear combination of neurons tuned to an optimal stimulus effectively defined by multiple dimensions25,23,26. This is a powerful extension of the older computation-through-memory models of vision and motor control. The question now is whether the available evidence supports the existence of a similar architecture underlying generalization in domains other than vision. Hierarchical or multi-layer networks AER INPUT Y Categ. Ident. AER OUTPUT AIT AER INPUT X PFC IT V4/PIT V4 V1 V1 Figure 2 A model of visual learning. The model summarizes in quantitative terms other models and many data about visual recognition in the ventral stream pathway in cortex. The correspondence between the layers in the model and visual areas is an oversimplification. Circles represent neurons and arrows represent connections between them; the dots signify other neurons of the same type. Stages of neurons with bell-shaped tuning (with black arrow inputs), that provide example-based G.Indiveri ( Neuromorphic spiking chips 28 / 38

36 Outline 1 Neuromorphic Engineering 2 Spike-based sensory systems 3 Spiking Neural Networks Silicon Neurons Silicon synapses Winner-Take-All networks Multi-chip networks 4 Learning 5 Neuromorphic Cognitive Systems G.Indiveri ( Neuromorphic spiking chips 29 / 38

37 Spike-timing dependent plasticity (STDP) Abbot, Nelson, If an input (pre-synaptic) spike arrives shortly before an output (post-synaptic) spike is emitted, the synaptic efficacy is increased. 2 If it arrives soon after the output spike is emitted, the synaptic efficacy is decreased. G.Indiveri ( Neuromorphic spiking chips 30 / 38

38 STDP is not enough for learning complex spatio-temporal patterns Senn, Biological Cybernetics, 2002 [...] additional non linearities are required if STDP should be relevant for both encoding information represented in a spike correlation code and a mean rate code without spike correlations. G.Indiveri ( Neuromorphic spiking chips 31 / 38

39 STDP is not enough for learning complex spatio-temporal patterns Spike-based learning mechanisms ideal for VLSI implementations depend on the neuron s membrane potential; synaptic weights have two stable states (bi-stability); many synapses see the same pre- and post-synaptic mean activity (redundancy); LTP/LTD is induced only in a random subset of stimulated synapses (stochasticity). [Fusi et al. 2000]; [Gütig, Sompolinsky 2006]; [Brader et al. 2007] G.Indiveri ( Neuromorphic spiking chips 31 / 38

40 Spike-driven plasticity in silicon Post-synaptic Output w 1 w 2 w 3 w 4 w n Pre-synaptic Inputs Pre-synaptic component Post-synaptic component ~pre V whi V UP I&F Circuit V spk Diff-pair Integrator V Ca V UP AER input spike V wth V DN V Wi Diff-pair Intergator I syn V mem V mem Current Comparator V wlow V ilk V mth Comparator V cmp V DN pre V mem G.Indiveri ( Neuromorphic spiking chips 32 / 38

41 Spike-driven plasticity in silicon Up θ Dn pre ~weight post Time(s) [Mitra et al. 2009] G.Indiveri ( Neuromorphic spiking chips 32 / 38

42 Stochastic weight update LTP/LTD probabilities and stop-learning LTD consolidation No LTD consolidation V H θ V L w V mem V H θ V L w V mem pre pre Time(s) Time(s) G.Indiveri ( Neuromorphic spiking chips 33 / 38

43 Outline 1 Neuromorphic Engineering 2 Spike-based sensory systems 3 Spiking Neural Networks Silicon Neurons Silicon synapses Winner-Take-All networks Multi-chip networks 4 Learning 5 Neuromorphic Cognitive Systems G.Indiveri ( Neuromorphic spiking chips 34 / 38

44 Distributed event-driven systems Neuromorphic cognitive systems can be assembled by using: 1 Full custom hybrid analog/digital neural processing VLSI devices. 2 A spike based communication protocol (e.g., the Address-Event Representation). 3 Systematic methods for parameter tuning. 4 Methods for implementing state-dependent computation using spiking neural networks. G.Indiveri ( Neuromorphic spiking chips 35 / 38

45 State dependent computation swta diagram Inhibitory }neurons Excitatory neurons I E E E E E swta networks as building blocks Linear behaviors Global Inhibition Nearest-N Excitation Analog gain Locus invariance Gain control by common mode input Non linear behaviors Selective amplification Signal restoration Multi-stability Configure key parameters of the WTA network automatically. Implement state-holding elements. Learn network connectivity patterns. [Douglas and Martin, 2007] G.Indiveri ( Neuromorphic spiking chips 36 / 38

46 Conclusions Toward neuromorphic cognitive behaving systems G.Indiveri ( Neuromorphic spiking chips 37 / 38

47 Conclusions Toward neuromorphic cognitive behaving systems By using event based sensors and spike-based neural processing circuits it is possible to implement real-time sensory-motor systems. G.Indiveri ( Neuromorphic spiking chips 37 / 38

48 Conclusions Toward neuromorphic cognitive behaving systems By using event based sensors and spike-based neural processing circuits it is possible to implement real-time sensory-motor systems. By using soft WTA multi-chip networks it is possible to implement real-time state-dependent computation. G.Indiveri ( Neuromorphic spiking chips 37 / 38

49 Conclusions Toward neuromorphic cognitive behaving systems By using event based sensors and spike-based neural processing circuits it is possible to implement real-time sensory-motor systems. By using soft WTA multi-chip networks it is possible to implement real-time state-dependent computation. The AER communication infrastructure and automated parameter tuning techniques, allow us to synthesize spike-based neural finite state machines. G.Indiveri ( Neuromorphic spiking chips 37 / 38

50 Acknowledgments The Institute of Neuroinformatics Elisabetta Chicca Stefano Fusi Chiara Bartolozzi The NCS group ( Funding sources neurop (257219) ERC SCANDLE (ICT ) emorph (ICT ) Rodney Douglas Kevan Martin Richard Hahnloser nattention (121713) SNF SoundRec (119973) SNF G.Indiveri ( Neuromorphic spiking chips 38 / 38