A NEURODYNAMICAL RETINAL NETWORK BASED ON REACTION-DIFFUSION SYSTEMS

Transcription

1 A NEURODYNAMICAL RETINAL NETWORK BASED ON REACTION-DIFFUSION SYSTEMS Matthias S. Keil and Gabriel Cristóbal Instituto de Óptica (CSIC) Image and Vision Department Serrano 121, Madrid (Spain) Heiko Neumann University of Ulm Department of Neural Information Processing Oberer Eselsberg,D Ulm, Germany ABSTRACT A dynamical model for retinal processing is presented. The model describes the output of retinal ganglion cells whose receptive field is composed of a center and a surround combining linearly. However, in comparison to the classical Difference-of-Gaussian (DOG) model, center and surround are generated in two separate layers of reaction-diffusion systems, through a difference in the speed of activity-propagation between both layers. Thus, intra-layer coupling is based exclusively on next-neighbor interactions. This makes the model suitable for VLSI implementation. Furthermore, the layers are connected by equations with feedback-inhibition to form ON-center/OFF-surround and OFFcenter/OFF-surround receptive fields. The model s output in the early dynamics corresponds to high resolution contrast information, whereas the output at later times can be considered as correlated with local brightness and darkness, respectively. To examine this in more detail, simulations with the Hermann/Hering-grid and grating induction were carried out. 1. INTRODUCTION Our knowledge regarding of how a cortical representation of a visual scene is constructed from the activity of retinal ganglion cells is far from being complete. One reason is that the axons of retinal ganglion cells (constituting the optic nerve) form a bottleneck for the transmission of visual information between the real world and the brain. In the cortex, the representation of visual information is then blown up again: there exists evidence that an orientation and scale selective analysis of the retinal image takes place [1]. Thus, the extraction of object or surface contours seem to play an important role in re-constructing a visual scene. For all that we do not see just contours, but colored surfaces. This raises the question of how the retinal information is used to reconstruct surface properties. A popular idea is to combine a This work has been supported in part by the AMOVIP INCO-DC EU Project. Corresponding author: mat@optica.csic.es set of isotropic (e.g. difference of Gaussian) or anisotropic (e.g. Gabor) filters over a variety of scales. Thereby scales are usually arranged in octave relationship. Furthermore, when using DOG-filters, center size and surround size usually possess a fixed ratio at all scales. We refer to this as classical multiscale analysis. Models based on this idea suggest therefore that perceived brightness is a (often nonlinear) function of contrast, and make no explicit distinction between cortical circuits dedicated to the processing of contrast information and the formation of cortical brightness maps. Only few models exists which explicitly separate contrast- and brightness processing (e.g. [2, 3, 4]). However, models for brightness perception which make use of dynamical retinal processing are, up to our knowledge, not available. The dynamical approach presented here shows how initially contrast-sensitive retinal ganglion cells change their behavior such that they later explicitly transmit brightness information. 2. MATHEMATICAL DESCRIPTION According to the known physiology of spatially linear summing ganglion cells, ON-center-cells respond to stimulation with light in the center of their RF, and are inhibited when stimulated in the surround. The opposite holds true for OFF-cells. This can be modeled with Difference-of- Gaussian filters (DOGs) [5]. Our circuit generates dynamically changing receptive fields (RFs) with center-surround antagonism [6]. The center-surround antagonism is generated by two interacting layers of reaction-diffusion systems. The diffusing quantity may be considered as neural activity. The spatial organization of the receptive field with time is characterized by a difference in the speed of activitypropagation between both layers. Note that lateral spread of information is an ubiquitous feature in retinal processing (e.g.[7],[8]). Exchanging information only between immediate neighbors minimizes wiring costs. Therefore, our circuit may be implemented in VLSI. However, the price to pay for saving To appear in Proc. ICIAP2001, September 26-28, 2001, Palermo, Italy c IEEE 2001

2 hardware is the time which is necessary for the generation of big receptive fields. When considering retinal anatomy, then one might conjecture that biological systems have evolved to a trade off between the avoidance of long processing times and the minimization of connectivity between cells (i.e. increasing dendritic diameters vs. direct electrical coupling). Our model does not perform a multi-scale analysis in the classical sense: the increment in center-size decays with time, which lets the center converge to a finite size ( convergent diffusion ). By way of contrast the surround is permitted to expand infinitely. This may be criticized for being biologically implausible at first sight. However, even if such an unbounded diffusion took place in biological retinae, very big surround sizes which are dynamically constructed would unlikely be reached, because active behavior is usually associated with a change of the retinal image. Center and the surround are described by separate reactiondiffusion systems. The convergent center diffusion decays with time-constant. Thence, parameterizes the spatial organization of the receptive field. This splits up the diffusion process in a slower one for the center, and in a faster one for the generation of the surround. The layer for center diffusion (C-layer, consisting of C-cells) is described by! #"%$'&)(+*,.- 0/ 1 2$' $ : (1) 5426;7<6 = denotes the cell s mean firing rate, the cell s membrane capacitance, >? the leakage conductance, the resting potential, / $> the Dirac delta function (initializing the C-layer at time 1 ), : the excitatory synaptic battery. Stimuli are static and correspond to luminance values (representing gray-levels between zero (darkest) and one (brightest)). " the diffusion coefficient (common to both layers), and - $> A@CB D)E $>; denotes the Laplacian. The S-layer generating the surround is allowed to diffuse freely according to F G F? G H I!H" - G J/ 1 9$K3 5426;7 9$ : G (2) Both layers are combined, in a DOG like fashion, by ONcells L AM? M H! ON PRQS M 9$ T U0$ MKVXW (3) where TY$ V W [ZLE 4\ $ 68] denotes half-wave rectification. The analogous equation for the OFF-cells is F D >? D0H 2 N P^QS_D 9$ T UJ$ D V`W (4) The leakage constant and the inhibitory feedback weight U are common to both equation 3 and equation 4. The in- put is provided by the center-activity T V W and and the surround-activity N T G V W. The output of the ON-center cell is a T M V W, and the output of the OFF-center cell b T D V W. Note the difference to Grossberg s retinal models (e.g. [9, 10, 11]): no driving potential is associated with center and surround inputs. This can be considered as a linear limit case, which can be obtained from the full driving-potential formulation for?dcfe and with gain-factors for center and surround being of the same order of magnitude as?. The normalizing behavior of our cells therefore is not generated, as in Grossberg s models, by driving potentials, but rather through feedback inhibition. This has the advantage that that we get equal response-amplitudes of ON and OFF cells with a luminance step, whereas the Grossbergian formulation always yields higher OFF responses (due to selfnormalization). Although the latter response-type is well suited to reconstruct surface properties at the (model s) cortical level, it nevertheless presents problems with feedback circuits which amplify such asymmetries (see, e.g. [12]). The rectified output of the ON cell model with the input (a) Stimulus (b) 2.5 msec (c) 250 msec Fig. 1. (a) Luminance step (input) (b) rectified ON-cell output at hg iyj msec. (c) rectified ON-cell output at hg j ] msec. shown in figure 1a is visualized at time g ikj msec (figure 1b) and g j ] msec (figure 1c). There we see that ON cell responses at large times are correlated with local brightness, whereas the early dynamics corresponds to contrast information. The whole dynamics is reminiscent of the filling-in process [13], where brightness and darkness information, respectively, starts spreading out from contrast borders and flows into surfaces. The degree of blurring is determined by the center decay. Stopping the center from expanding its receptive field would yield a white surface with sharp contours in figure 1c. 2

3 3.1. The Hermann/Hering Grid (a) 0.5 msec (b) 5 msec (c) 50 msec Fig. 2. ON cell dynamics of the step stimulus in figure 1a with 100% additive Gaussian noise. However, including low-pass filtering smoothes out noise, especially in the late dynamics (figure 2). 3. BRIGHTNESS ILLUSIONS Here, simulations with the Hermann/Hering grid [14, 15] and the grating-induction [16, 17] are presented. Many aspects of these illusions can be explained by models which are based on DOG filters (e.g. [2], [18], [19], [20]). However, none of the models cited above explicitly examined dynamical aspects associated with the illusions, and our model differs mainly in the following (inter-related) points: First, no multiscale analysis in the classical sense is made. As a consequence, ON-cell (OFF-cell) responses at late times are more and more correlated with local brightness (darkness). This characteristic will be examined in more detail in this section. Second, center and surround are combined over a continuous scale-space through leaky integrators (equations 3 and 4). We do not use, for example, a weighted sum of filters at separated scales, nor do we apply rules for brightness interpretation.third, only local interactions between model cells take place. Bigger RFs are created in a continuous manner with the flow of time, as opposed to using a battery of filters with static size and therefore discontinuous scaling. In the Hermann/Hering grid illusion faint spots appear at the crossings (or intersections) of the streets. The effect depends on the width of the streets: narrower streets give rise to a stronger perceived effect (compare figure 3a with 3b). Using the street widths of figure 3, the effect can hardly be seen foveally (fixation of a crossing causes the corresponding spot to disappear). Viewing the grid diagonally reduces the illusion, and this cannot be explained with our model. Rather, it demonstrates that brightness is evaluated in the cortex, because mammalian retinal ganglion cells usually do not reveal such strong dependence on orientation. The orientation-dependence of the illusion can be explained by the fact that natural images contain more Fourier-energy along horizontal and vertical axes [21], what is mirrored also in the primary visual cortex [22]. In addition to the illusory faint spots perceived at the intersections, attentive inspection reveals narrow dark lines or bars that run between the dark blocks (i.e. along the streets). These illusory dark lines are flanked by bright edges at the crossings, where they terminate. These dark channels were accounted for by the inner contrast and the border contrast [23]. The classical explanation of the illusory spots perceived at the street crossings, on the other hand, is based on lateral inhibition [24]. Furthermore there exists evidence that dynamical aspects play a role in the formation of the illusory spots: when viewed under stroboscopic lighting, the illusion is enhanced [25]. Figure 4 shows the ON cell dynamics (equation 3) for the 0.5 msec (a) 0.5 msec 2.5 msec (b) 2.5 msec 5 msec (c) 5 msec 25 msec (a) 25 msec 50 msec (b) 50 msec 500 msec (c) 500 msec (a) 5 pixel wide streets (b) 7 pixel wide streets Fig. 3. Two Hermann grids which were used in the simulations. Original image size was e g e g pixel. Fig. 4. ON cell dynamics for the Hermann-grid stimulus (5 pixel): darker colors correspond to higher activity of ONcells. 3

4 (a) black 0.0 (b) gray 0.5 (c) white 1.0 Fig. 6. Full contrast grating inductions with different background colors. Test field width is 5 pixels, spatial frequency g cycles/image. The numbers refer to background luminances: 0.0 is black, 0.5 is gray, and 1.0 is white (image size e g e g pixel). Fig. 5. ON cell dynamics for the Hermann-grid stimulus (7 pixel): the evolution of a row in time is visualized. This row corresponds to a street. At j ] ] msec, the size of the RF-center is approximately j iyj pixel, and the size of the RF-surround ikj pixel. Note that the inversion-effect here is weak. stimulus corresponding to figure 3b. At ] iyj msec, only a contour map is seen, and no correlation with brightness is obvious. The illusion develops from g ikj to g)j msec, and is stable approximately up to g j ] msec. This can be seen from the two valleys in figure 5, which shows the dynamic of a street: at each time a row-slice was taken from those cells having their receptive field centered in the street. The position of the valleys correspond to two street crossings. The dynamics confirms our hypothesis: the early retinal information is contrast-related, whereas later it is correlated with local brightness 1. Thus, illusory gray spots may form because at street crossings the local brightness information runs behind the brightness information in streets enclosed by blocks. This effect is still visible in the late dynamics, but weaker. occur at gray test-fields, where both ON and OFF cells are responsive (figure 6b). With a black test-field (figure 6a), the centers of ON-cells located in the test field will not be active, and consequently the surround only can further hyperpolarize the cell s membrane potential. The OFF cells, on the other hand, will be active, because they are depolarized by the surround, and no inhibition is imposed on the surround by the center. With a white test field (figure 6c), the opposite situation holds true: OFF cells will be silent, and ON cells are responsive. The dynamics is exemplified in figure 7 with the panel 3.2. Grating Induction Here we examine the model dynamics together with the grating induction [16, 17]. This panel consists of two inducers, separated by a gap, called test-field or inspection area. Each inducer is a sine-wave-grating. Both inducer-gratings normally are in-phase. Inspection of the test-field reveals an (induced) illusory grating in anti-phase to the inducers. The anti-phase character of the induced (illusory) grating is a consequence of the antagonistic center-surround structure of the receptive field. The most efficient modulation should 1 For constructing a complete brightness map the OFF channel also has to be taken into account Fig. 7. Temporal evolution of a test-field slice of ONcells with grating-induction figure 6c, but with a spatial frequency of the inducers cycles/image. Test field luminance was 0.5 (gray). The receptive fields of the ON-cells were centered on the test field. Image size e g e g pixel, test field width 5 pixel. Note the inversion effect in the late dynamics. 4

5 shown in figure 6b, but with inducers spatial frequency cycles/image. Note the peculiar inversion effect in the late dynamics, i.e. previously silent ON-cells (OFF-cells) are getting active, whereas previously active ON-cells (OFFcells) are getting unresponsive. This is because at some time the receptive field center (with convergence-diameter j iyj pixel) grows larger than the width of the test field (5 pixel). This inversion is also revealed in simulations with the Hering/Hermann grid of the previous section, and is conjectured to be connected with the appearance of the dark channels which are perceived in addition to the faint spots at the intersections. With the grating-induction, the inversion is conjectured to be connected to the so-called phantoms [26]. Phantoms are perceived in-phase with the inducers, and seem to rival with the induced anti-phase grating. The control parameter which causes symmetry breaking in this perceptually bistable situation was reported to be the luminance of the inspection area [27]. Phantom visibility was observed to be maximal with black and white test field colors. phase=0, OFF t=63 msec phase=45 o, OFF t=63 msec Fig. 8. Top half: grating induction with 7 pixel test field width and inducer phase differences of ] (left) and j (right). Bottom half: corresponding OFF-cell activity at g ikj msec (original image size g j g j pixels, test field width 5 pixel, test field luminance 0.5, inducer spatial frequency cycles/image). Darker colors in OFF-cell responses encode higher activity, whereas brighter colors in the stimuli stand for higher luminance values. Figure 8 and 9 show the model s behavior on shifting the phase of the inducer-gratings [28]. The model predicts correctly that the induced gratings are getting tilt, and that no more coherent grating is perceived when the inducergratings stand in opposite phase. Other predictions obtained from the dynamics are that the induced grating looses in strength with increasing spatial frequency and when the width of the test-field is increased (not shown) [16]. 4. SUMMARY A dynamical retinal model was presented and it was shown that it could successfully account for the Hermann/Hering phase=90 o, OFF t=63 msec phase=180 o, OFF t=63 msec Fig. 9. Top half: grating induction with 7 pixel test field width and inducer phase differences of ] (left) and e ] (right). Bottom half: corresponding OFF-cell activity at g iyj msec (original image size g j g)j pixels, test field width 5 pixel, test field luminance 0.5, inducer spatial frequency cycles/image). grid and the brightness induction. Thus, the model suggests a novel mechanism for transmitting brightness/darkness information into the cortex, without the need for an extra luminance-driven channel, which contains a low-passed filtered version of the retinal image, as in e.g. [4, 29] (for an alternative approach see [30]). Consistent with the Grossbergian FACADE theory (e.g. [2, 3, 4]), a border map (constituting filling-in compartments) should form before fillingin takes place (otherwise filling-in would generate uniform activity everywhere). Thus, after presenting a stimulus, the immediate information from the optic nerve may be used by the brain to form this border map (with contrast related retinal responses), and thereafter filling-in may be triggered (with brightness related retinal responses). The more time passes (i.e. the longer one looks at an object), the better the resulting cortical representation of surfaces will be, because the brightness estimation generated by our retinal circuit proceeds from more local to more global. 5. REFERENCES [1] D.H. Hubel and T.N. Wiesel, Receptive fields, binocular interaction and functional architecture in the cat s visual cortex, J. Physiol., vol. 160, pp , [2] S. Grossberg and D. Todorović, Neural dynamics of 1-d and 2-d brightness perception: A unified model of classical and recent phenomena, Perception & Psychophysics, vol. 43, pp , [3] A. Gove, S. Grossberg, and E Mingolla, Brightness perception, illusory contours, and corticogeniculate feedback, Visual Neuroscience, vol. 12, pp ,

6 [4] L. Pessoa, E Mingolla, and H. Neumann, A contrastand luminance-driven multiscale network model of brightness perception, Vis. Res., vol. 35, no. 15, pp , [5] R. W. Rodieck, Quantitative analysis of cat retinal ganglion cell response to visual stimuli, Vis. Res, vol. 5, pp , [6] S.K. Kuffler, Discharge patterns and functional organization of mammalian retina, J. Neurophysiol., vol. 16, pp , [7] K.-I. Naka, Neuronal circuitry in the catfish retina, Invest. Ophthal., vol. 15, pp , [8] J.-I. Toyoda, Membrane resistance changes underlying the bipolar cell response in the carp retina, Vis. Res., vol. 12, pp , [9] S. Grossberg, Contour enhancement, short term memory, and constancies in reveberating neural networks, Studies in Applied Mathematics, vol. 52, no. 3, pp , [10] S. Grossberg, E. Mingolla, and J. Williamson, Synthetic aperture radar processing by a multiple scale neural system for boundary and surface representation, Neural Networks, vol. 8, no. 7-8, pp , [11] E. Mingolla, W. Ross, and S. Grossberg, A neural network for enhancing boundaries and surfaces in synthetic aperture radar images, Neural Networks, vol. 12, pp , [12] E. Mingolla, W. Ross, and S. Grossberg, Visual cortical mechanisms of perceptual grouping: interacting layers, networks, columns and maps, Neural Networks, vol. 13, pp , [13] H. Gerrits and A. Vendrik, Simultaneous contrast, filling-in process and information processing in man s visual system, Experimental Brain Research, vol. 11, pp , [14] L.. Hermann, Eine Erscheinung simultanen Kontrastes, Pflügers Archiv für die gesamte Physiologie, vol. 3, pp , [15] E. Hering, Zur Lehre vom Lichtsinne, Gerold, Wien, [16] M.E. McCourt, A spatial frequency dependent grating-induction effect, Vis. Res., vol. 22, pp , [17] J.M. Foley and M.E. McCourt, Visual grating induction, Journal of the Optical Society of America, vol. 2, pp , [18] B. Blakeslee and M.E. McCourt, Similar mechanisms underlie simultaneous brightness contrast and grating induction, Vis. Res., vol. 37, pp , [19] B. Blakeslee and M.E. McCourt, A multiscale spatial filtering account of the white effect, simultaneous brightness contrast and grating induction, Vis. Res., vol. 39, pp , [20] J.A. McArthur and B. Moulden, A two-dimensional model of brightness perception based on spatial filtering consistent with retinal processing, Vis. Res., vol. 39, pp , [21] M.S. Keil and G. Cristóbal, Separating the chaff from the wheat: possible origins of the oblique effect, J. Opt. Soc. of America A, vol. 17, no. 4, pp , [22] C.S. Fumanski and S.A. Engel, An oblique effect in human primary visual cortex, Nature Neuroscience, vol. 3, no. 6, pp , [23] H. Dombrowski, Experimentelle Untersuchungen über das Hering-Ehrensteinsche Helligkeitsphänomen, Ph.D. thesis, Charles University, Faculty of Philosophy, Prague, Czech Republic, [24] G. Baumgartner,, Pflügers Archiv für die gesamte Physiologie, vol. 272, pp , [25] F.L. Kitterle and T.R. Corwin, Enhancement of apparent contrast in flashed sinusoidal gratings, Vis. Res., vol. 19, pp , [26] P. Tynan and R. Sekuler, Moving visual phantoms: a new contour completion effect., Science, vol. 188, pp , [27] K. Sakurai and J. Gyoba, Optimal inspection area luminance for seeing stationary visual phantoms, Vis. Res., vol. 25, pp , [28] Q. Zaidi, Local and distal factors in visual grating induction, Vision Research, vol. 29, pp , [29] J.M.H. du Buf and S. Fischer, Modeling brightness perception and syntactical image coding, Opt. Eng., vol. 34, no. 7, pp , [30] M.S. Keil and G. Cristóbal, How is luminance information passed into the cortex? emergent multifunctional behavior of a simple cell model, SPIE, 1999, vol. 3812, pp