Optical hybrid analog-digital signal processing based on spike processing in neurons

Transcription

1 Invited Paper Optical hybrid analog-digital signal processing based on spike processing in neurons Mable P. Fok 1, Yue Tian 1, David Rosenbluth 2, Yanhua Deng 1, and Paul R. Prucnal 1 1 Princeton University, Princeton, New Jersey 08544, USA 2 Lockheed Martin Advanced Technologies Laboratory, Cherry Hill, New Jersey 08002, USA ABSTRACT Spike processing is one kind of hybrid analog-digital signal processing, which has the efficiency of analog processing and the robustness to noise of digital processing. When instantiated with optics, a hybrid analog-digital processing primitive has the potential to be scalable, computationally powerful, and have high operation bandwidth. These devices open up a range of processing applications for which electronic processing is too slow. Our approach is based on a hybrid analog/digital computational primitive that elegantly implements the functionality of an integrate-and-fire neuron using a Ge-doped non-linear optical fiber and off-the-shelf semiconductor devices. In this paper, we introduce our photonic neuron architecture and demonstrate the feasibility of implementing simple photonic neuromorphic circuits, including the auditory localization algorithm of the barn owl, which is useful for LIDAR localization, and the crayfish tail-flip escape response. Keywords: Spike processing, optical signal processing, nonlinear optics, photonics, neuromorphic processing 1. INTRODUCTION The success of neuromorphic engineering in emulating the biophysics of neurons has lead to important practical designs for low power adaptive analog computing and signal processing systems. With the use of analog VLSI technology, devices that closely replicate the capabilities of the retina and the cochlea have been implemented. Although these frontend sensor devices are small and have low-power consumption, their speed is far too slow for many real-time signal processing applications. Fiber optics has been known to have fast processing speed and has a large bandwidth. With the marriage of spike processing and the characteristics of optics, we have designed and implemented a hybrid analog/digital signal-processing device that elegantly implements the functionality of an integrate-and-fire neuron using a novel type of non-linear optical fiber and an off-the-shelf semiconductor device. Combining the advantages of both spike processing and optics, the resultant spike-processing device not only has the potential to be scalable and computationally powerful, but also has large bandwidth required for high-speed processing applicationa where the use of electronic processing is too slow. The spiking neuron comprises a small set of basic operations (delay, weighting, spatial summation, temporal integration, and thresholding), and is capable of performing a variety of computations, depending on how its parameters (e.g., delays, weights, integration time constant, threshold) are configured. In our photonic neuromorphic device [1], we use a novel type of Ge-doped highly nonlinear optical fiber for thresholding, an off-the shelf semiconductor device for temporal integration, and several simple optical components for delay, weighting, and spatial summation. Utilizing this opticsbased hybrid analog-digital processing primitive, complex and high bandwidth processing algorithms can be implemented. In this paper, we introduce the integrate-and-fire neuron architecture used in our photonics-based neuromorphic circuits, followed by the implementation of the spike processing device we used for different photonics neuromorphic circuits. 2. SPIKE PROCESSING IN A LEAKY-INTEGRATE-AND-FIRE (LIF) NEURON Our approach based on the standard leaky-integrate-and-fire (LIF) model of a neuron that operates as follows [2]: The neuron has N inputs that are a continuous time series, consisting either of spikes or continuous analog values representing voltages. After each input is independently weighted and delayed, they are spatially summed (summed Optics and Photonics for Information Processing V, edited by Khan M. Iftekharuddin, Abdul Ahad Sami Awwal, Proc. of SPIE Vol. 8134, SPIE CCC code: X/11/$18 doi: / Proc. of SPIE Vol

2 point-wise). The resulting single time-series is then temporally integrated using an exponentially decaying impulse response function. If the integrated signal exceeds a threshold, then the neuron outputs a spike. After the spike, there is a short refractory period during which no other spikes can be issued. The output of the neuron consists of a continuous time-series of spikes. The above spiking behavior is formalized below: Inputs to neuron: Neuron outputs: () t = 1.. () t n Σ ω I ( t δ ) I, where N I = =1, whereω is the weight and δ is the delay () t O If ν () t ν threshold then else [spike equations: ν ( t ): = Vr and ( t) = 1 [between spikes: ν t t 0 s 1 τ = r c 0 O ] τ t t0 m m () t V e + e I( t s)ds, where t 0 is the last time the neuron spiked O t = ]: ( ) 0 After issuing a spike, there is a short period of time, the refractory period, during which no other spikes can be issued. The output of the neuron consists of a continuous time series comprised of spikes. As opposed to ust the integrate-andfire neuron model, the LIF neuron model allows for the application of temporal weighting function to the inputs, providing an additional dimension to characterize the input. 3. OPTICAL IMPLEMENTATION OF A LIF SPLIKING NEURON In our optics-based spike processing model, the LIF neuron is mimicked optically using two key elements: a semiconductor optical amplifier (SOA) and a Ge-doped nonlinear fiber based thresholder. The functional architecture of the integrate-and-fire device consists of three maor processing blocks as shown in Figure 1 (ii) summing, (iv) temporal integration, and (v) thresholding. The input signal is first weighted and delayed as in Figure 1(i), and then spatially summed (Figure 1(ii)). The sampling pulse train shown in (iii) is used to provide spikes to the integrator, while the input signals are integrated in the SOA/EAM (Figure 1(iv)). The green curve corresponds to the dynamic change in the SOA/EAM governed by the spatially-summed input signals. The integrated output is thresholded by a HDF-based loop mirror (Figure 1(v)) and the final output is shown in Figure 1(vi). The equations governing the SOA carrier density and the equations governing leaky integration in a LIF neuron has an exact correspondence [3], which ustifies the use of an SOA as the embodiment of the leaky integrator in this computational primitive. Proc. of SPIE Vol

3 Figure 1. Illustration of a photonic neuron. Input 1 Input N: inputs; W 1 W N : variable weight; T 1 T N : variable time delay; SOA: semiconductor optical amplifier; EAM: electro-absorption modulator; HDF: highly Ge-doped nonlinear fiber. Note: Using an SOA results in an inverted output, while an EAM results in an non-inverted output. The SOA has an exponential recovery behavior that is similar to the integration characteristic that a neuron requires. When an optical pulse is launched into the SOA, the SOA carrier density decreases. In the presence of a pumping current, the carrier density recovers exponentially over time. When a second optical pulse is launched into the SOA before the carrier density completely recovers, it further decreases the carrier density, resulting in a temporal integration of the effects of both input pulses. Figure 1 illustrates the conversion between changes in carrier density to output pulse amplitude through gain sampling The interval between sampling pulses corresponds to the refractory period. We experimentally measured the relative change in carrier density in response to different optical pulses launched into the SOA. (a) (b) Figure 2. The measured SOA response to excitation by multiple pulses. (a) Experimentally measurement of the pulses launched into the SOA (b) Relative change in SOA carrier density as measured using a optical sampling pulses. The SOA recovery time (integrationn time constant) is 180 ps. Figure 2(a) shows a series of optical pulses with different temporal spacing and intensity that are launched into the SOA, while Figure 2(b) shows the relative change in carrier density, represented by the intensity of the sampling pulses. An nonlinear optical loop mirror is used for thresholding in which the fiber loop consists of a short piece of Ge-doped nonlinear fiber and a tunable isolator as a directional attenuator [4]. The optical thresholder has a cubic transfer function Proc. of SPIE Vol

4 which suppresses low-power inputs while saturating at high powers. In other words, the optical thresholder amplitude- discriminatess the input signal and the neuron fires only when the input power exceeds a certain threshold. In the context of the spiking neuron, the thresholder is used to removee the undesiredd weak spikes while equalizing the strong spikes to provide a low noise control for a second-stage neuron. 4. LIGHTWAVE NEUROMORPHIC CIRCUITS We demonstrated several small-scale lightwave neuromorphic circuits to mimic important neuronal behavior based on the optics-based neuromorphic circuit described above. Here, we would like to present two of those lightwave neuromorphic circuits: (i) the auditory localization algorithm of the barn owl, useful for LIDAR localization, and (ii) the crayfish tail-flip escape response, useful for responding to patterns. 4.1 Auditory localization algorithm of the barn owl Figure 3(a) shows a simple diagram of auditory localization. Due to the difference in position of obect 1 and obect 2, there is a time difference between the signals arriving the owl s left sensor and right sensor, denoted as ΔT 2 = (t 1 1a-t 1b ) for obect 1 and ΔT 2 = (t 2a -t 2b ) for obect 2. Therefore, the neuron can be configured to respond to a certain obect location merely by adusting the weight and delay of the neuron inputs. If the weighted and delayed signals are strong enough and arrive within the integration window, the neuron spikes; otherwise no spike results. Figure 3(b)i illustrates the corresponding SOA-based integrator response when the two weighted and delayed signals are too far apart. The stimulated signal cannot pass through the thresholder and therefore no spike is obtained. When the two inputs are close enough, the carrier density reaches the threshold and leads to a spike, as shown in Figure 3(b)ii. This algorithm is useful for LIDAR localization. (a) (b)i (b)ii Figure 3. (a) Schematic illustration of the auditory localization algorithm of the barn owl. (b)i SOA carrier density - when two signals are far apart (no spike). (b)ii SOA carrier density - when two signals are close (spike). Figure 4 is an experiment showing the temporal sensitivity of the spike processing [5]. Figure 4(a) corresponds to the input consists of a number of pulses (signals) with same intensity but with different time interval, i.e. (i) case I: signals that are close together (due to the limited bandwidth of the photodetector, the measured pulsess seems to be a strong pulse) (ii) case II: signals that are further apart. After temporal integration at the SOA and thresholding at the optical Proc. of SPIE Vol

5 thresholder, a spiking output is obtained as shown in Figure 5(b). Due to the gain depletion property of SOA, the spike output is inverted. As we can see in Figure 5(b), no spike is observed when the input signals are close together (case I), while spikes are observed when input signals are furtherr apart (case II). (a) (b) Case I Case III Case II Case I Figure 4. (a) Input to the photonics neuron (i) case I: input signals are close together (measured temporal resolution limited by the bandwidth of the photodetector) (ii) case II: input signals are further apart (b) Photonic neuron output (inverted) (i) case I: no spike when signals are close (ii) case II: spike when signals are further apart. 4.2 Tail-flip escape response of the crayfish Crayfish escape from a predator by means of a rapid tail-flip response. The corresponding neuron circuit is configured to respond to appropriately abrupt stimuli but not respond to stimuli from normal water flow. Since this is a life-or-death mimics the crayfish decision to the crayfish, the response has to be executed quickly and accurately. Our device, which circuit using photonic technology, is sufficiently fast to be applied to defense applications in which critical decisions need to be made quickly while minimizing the probability of false alarm. A potential military application of lightwave neuromorphic signal processing based on escape response could be for pilot eection from aircraft under serious attack. By means of compact optical devices, the latency can be as low as 5000 ps. Figure 5. Experimental results of a photonic feature recognizer based on the crayfish tail-flip abc and ab- ; (b) Output spike case II: escape response (a) Optical inputs to the first integrator; (b) Output spikes case I: recognizing pattern recognizing pattern abc only; (c) Outpu case III: none of the input is recognized. The recognizer consists of two photonic neurons [6]. The first integrator is configured to respond to a set of signals with specific features, while the second integrator further selects a subset of the signal from a set determined by a weighting and delay configuration. It responds only when the input stimuli and the spike from the first integrator arrive within a very short time interval. Figure 5 shows the experimental results of a lightwave neuromorphic feature recognizer based on the tail-flip escape response of the crayfish, the device is configured to respond to different sets of input patterns by Proc. of SPIE Vol

6 simply varying the weights and delays of the neuron inputs. Figure 5(a) corresponds to the input patterns, while the recognizer can be configured to response to pattern abc and ab- (case I: Figure 5(b)) or only to pattern abc (case II: Figure 5(c). However, when the inputs does not match with the recognizer, e.g. spikes from the first neuron arrive too late, i.e. exceed the integration time, the second integrator will not spike (case III: Figure 5(d)). 5. SUMMARY The optical hybrid analog/digital processing device we introduced is a result of the cross-fertilization between spike processing in the fields of computational neuroscience and high speed processing in nonlinear optical device physics. Utilizing the advantages of both fields, a high-speed, broadband, and scalable processing device can be achieved. We believe this technology will have broad applications to high bandwidth signal processing, ultra-fast control loops, ultralow latency response functions, and ultra-fast adaptive processing. REFERENCES [1] M. P. Fok, D. Rosenbluth, K. Kravtsov, and P. R. Prucnal, Lightwave Neuromorphic Signal Processing, IEEE Signal Processing Magazine, vol. 27, iss. 6, pp , November [2] W. Maass and C. M. Bishop, eds., Pulsed Neural Networks (The MIT Press, 1999). [3] D. Rosenbluth, K. Kravtsov, M. P. Fok, and P. R. Prucnal, A High Performance Photonic Pulse Processing Device, Optics Express, vol. 17, iss. 25, pp , December [4] K. Kravtsov, P. R. Prucnal, and M. M. Bubnov, Simple nonlinear interferometer-based all-optical thresholder and its applications for optical CDMA, Opt. Express 15, (2007). [5] K. Kravtsov, M. P. Fok, D. Rosenbluth, and P. R. Prucnal, Ultrafast All-Optical Implementation of a Leaky Integrate-and-Fire Neuron, Optics Express, vol. 19, iss. 3, pp , January [6] M. P. Fok, H. Deming, M. Nahmias, N. Rafidi, D. Rosenbluth, A. Tait, Y. Tian, and P. R. Prucnal, Signal Feature Recognition based on Lightwave Neuromorphic Signal Processing, Optics Letter, vol. 36, iss. 1, pp , January Proc. of SPIE Vol