Traffic Sign Detection with VG-RAM Weightless Neural Networks

Transcription

1 Proceedings of International Joint Conference on Neural Networks, Dallas, Texas, USA, August 4-9, 013 Traffic Sign Detection wit VG-RAM Weigtless Neural Networks Alberto F. De Souza, Cayo Fontana, Filipe Mutz, Tiago Ales de Olieira, Mariella Berger, Aelino Foreci, Jorcy de Olieira Neto, Edilson de Aguiar, and Claudine Badue Abstract We present a biologically inspired approac to traffic sign detection based on Virtual Generalizing Random Access Memory Weigtless Neural Networks (VG-RAM WNN). VG-RAM WNN are effectie macine learning tools tat offer simple implementation and fast training and test. Our VG-RAM WNN arcitecture models te saccadic eye moement system and te transformations suffered by te images captured by te eyes from te retina to te superior colliculus in te mammalian brain. We ealuated te performance of our VG-RAM WNN system on traffic sign detection using te German Traffic Sign Detection Bencmark (GTSDB). Using only 1 traffic sign images for training, our system was ranked between te first 16 metods for te proibitory category in te German Traffic Sign Detection Competition, part of te IJCNN 013. Our experimental results sowed tat our approac is capable of reliably and efficiently detect a large ariety of traffic sign categories using a few training samples. I. INTRODUCTION Safety during driing is a ery important researc topic for te automotie industry. One of te tecnologies tat can make cars safer to drie is automatic detection and recognition of traffic signs. Suc tecnology aims to warn te drier of inappropriate actions, e.g., speeding, taking a wrong turn in a one-way street, as well as to elp te drier in difficult situations, e.g., bad weater, tiredness, sleeplessness etc. Altoug te traffic sign detection process could be simplified as te appearance of certain traffic signs are fixed, sometimes een described by law, in real-world situations, gien te substantial appearance ariation, for instance, due to different ligt conditions, weater, iewpoint canges, ageing of te traffic sign and een deformations, simple approaces are not reliable and more robust metods are necessary. Wile umans are capable of detecting te large ariety of existing traffic signs efficiently, automatic systems are still a callenge. Gien te ig industrial releance, automatic traffic sign detection and recognition as been attracting many researcers attentions in recent years [1]. Despite many recent adances, traffic sign detection is still a complex real-world problem, wic makes it one important Manuscript receied on Marc 1st, 013. Tis work was supported in part by Conselo Nacional de Desenolimento Científico e Tecnológico- CNPq-Brasil (grants 55630/011-0, /010-0, and /009-0) and Fundação de Amparo à Pesquisa do Espírito Santo-FAPES-Brasil (grant /009). Alberto F. De Souza, Cayo Fontana, Filipe Mutz, Tiago Ales de Olieira, Mariella Berger, Aelino Foreci, Jorcy de Olieira Neto, Edilson de Aguiar, and Claudine Badue are wit te Departamento de Informática, Uniersidade Federal do Espírito Santo, Vitoria, ES, , Brazil (pone: ; fax: ; s: alberto@lcad.inf.ufes.br, claudine@lcad.inf.ufes.br). application for adanced and autonomous driing systems. A useful detection system needs to cope wit sign rotation, different ligting conditions, perspectie canges, occlusion and all kinds of weater conditions. Gien an image of a scene, te general problem of traffic sign identification is to try and identify one or more traffic signs in te image using a priori information about te sape, color or features present in te traffic signs. Te current solutions in te literature commonly inoles segmentation of traffic signs from te scenes (traffic sign detection), feature extraction from te traffic sign regions, and recognition. In tis paper, we are only interested in te traffic sign detection part of te identification problem. For traffic sign detection, a ariety of tecniques ae been proposed in te literature (see oeriews in [1], [], [3]), wic can be grouped in tree main categories: colorbased metods, sape-based metods and feature-based metods. Color-based metods [4] usually apply color segmentation combined wit edge detection tecniques to find specific sapes corresponding to traffic signs in images. Sape-based metods rely mostly in edge information to extract geometric constraints tat correspond to traffic signs, like circles in te images [5]. In addition, radial symmetry [6] can be employed to detect regular sapes like triangles, squares, octagons, etc. Feature-based metods apply macine learning tecniques to special features detected in te images. Among te most commonly used tecniques are Neural Networks [7], Support Vector Macines [8] and AdaBoost metods [9]. In tis paper, we present a biologically inspired approac to traffic sign detection based on Virtual Generalizing Random Access Memory Weigtless Neural Networks (VG- RAM WNN [10]). VG-RAM WNN are effectie macine learning tools tat offer simple implementation and fast training and test. Our VG-RAM WNN systems model te biological saccadic eye moement system and te transformation suffered by te images captured by te eyes from te retina to te superior colliculus of te mammalian brain. We ealuated te performance of our system using te German Traffic Sign Detection Bencmark (GTSDB) (ttp://bencmark.ini.rub.de) [11]. Our experimental results sowed tat our approac is capable of reliably and efficiently detect a large ariety of traffic sign categories using only 1 traffic sign images for training. Tis paper is organized as follows. After tis introduction, in Section II we briefly discuss te saccadic eye moement system and te transformation suffered by te images captured by te eyes from te retina to te superior colliculus of te mammalian isual system. In Section III, /13/$ IEEE 730

2 we present a biologically inspired VG-RAM WNN arcitecture for traffic sign detection. In Section IV, we describe our experimental metodology and analyze our experimental results. Our conclusions and directions for future work follow in Section V. II. SACCADIC SYSTEM AND MAPPING FROM THE RETINA TO THE SUPERIOR COLLICULUS Te saccadic eye moement subsystem of te mammalian isual system is te main responsible for pointing te foea towards objects of interest [1]. Te foea is te central region of te retina tat as te igest density of receptors and tus affords te greatest isual acuity. Te saccade system produces rapid eye moements (saccades) tat sift te foea rapidly to a isual target (saccade target) in te isual field. Te purpose of te saccade is to moe te eyes as quickly as possible. As tere is no time for large isual feedback to significantly modify te course of a saccade, corrections to te direction of eyes moement are typically made in successie saccades. Te saccadic eye moements are controlled by te midbrain s superior colliculus (SC). Te images captured by te eyes are transformed into electrical impulses by te retina and, troug te optic nere, are projected into te SC and oter cerebral areas [1]. Te neural projection from te retina to SC follows a retinotopic mapping, i.e., neigboring regions in te retina are projected onto neigboring regions of te SC [13]. Before a saccadic moement, cells in te SC are actiated and a winner-takes-it-all beaior leads to te selection of a point in te isual field retinotopically mapped in te SC tis point is te target of te saccade [14]. Figure 1: Log-polar transform. Te mapping from te retina to SC follows a log-polar function [13]. Figure 1 sows te log-polar transform of an image, centered on te point (x c, y c ) tis point corresponds to te center of attention in te isual field. Note tat te circle (in red) in te left image of Figure 1 becomes a straigt line in te rigt image, and te regions around te circle s center (te foea of te model) in te left image occupy a muc larger area in te rigt image. Te matematical modeling of te log-polar transform commonly used in te literature is gien by: R = ( x xc ) + ( y yc ) ρ α log( R) and (1) θ = arctan ( y yc ) ( x x ) c φ α θ. In tis paper, we did not employ te log-polar transform exactly as sown aboe, but a ariant tat was created to emulate more precisely te mapping from te retina to SC. Figure sows tis ariant of te log-polar transform. As Figure sows, neigboring regions in te image around te circle s center (te foea of te model) are also neigbors in te log-polar transform (retinotopy), as occurs in te SC. Tis does not occur in te transform depicted in Figure 1. Figure : Our ariant of te log-polar transform. III. TRAFFIC SIGN DETECTION WITH VG-RAM WNN A. VG-RAM WNN RAM-based neural networks, also known as n-tuple classifiers or weigtless neural networks, do not store knowledge in teir connections but in Random Access Memories (RAM) inside te network s nodes, or neurons. Tese neurons operate wit binary input alues and use RAM as lookup tables: te synapses of eac neuron collect a ector of bits from te network s inputs tat is used as te RAM address, and te alue stored at tis address is te neuron s output. Training can be made in one sot and basically consists of storing te desired output in te address associated wit te neuron s input ector [15]. In spite of teir remarkable simplicity, RAM-based neural networks are ery effectie as pattern recognition tools, offering fast training and test, in addition to easy implementation [10]. Howeer, if te network input is too large, te memory size becomes proibitie, since it must be equal to n, were n is te input size. Virtual Generalizing RAM (VG-RAM) Weigtless Neural Networks (WNN) are RAM-based neural networks tat only require memory capacity to store te data related to te training set [16]. In te neurons of tese networks, te memory stores te input-output pairs sown during training, instead of only te output. In te test pase, wit a distributed neuron memory model, eac neuron searces associatiely its memory by comparing te input presented to te network wit all inputs in te input-output pairs learned; wit a sared neuron memory model, eac neuron searces te memory of all te network s neurons. Te () 731

3 output of eac VG-RAM WNN neuron is taken from te pair wose input is nearest to te input presented te distance function employed by VG-RAM WNN neurons is te Hamming distance. If tere is more tan one pair at te same minimum distance from te input presented, te neuron s output is cosen randomly among tese pairs. Considering a distributed neuron memory model, Table 1 sows te lookup table of a VG-RAM WNN neuron wit tree synapses (X 1, X and X 3 ). Tis lookup table contains tree entries (input-output pairs), wic were stored during te training pase (entry #1, entry # and entry #3). During te test pase, wen an input ector (input) is presented to te network, te VG-RAM WNN test algoritm calculates te distance between tis input ector and eac input of te input-output pairs stored in te neuron s lookup table. In te example of Table 1, te Hamming distance from te input to entry #1 is two, because bot X and X 3 bits do not matc te input ector. Te distance to entry # is one, because X 1 is te only non-matcing bit. Te distance to entry #3 is tree, as te reader may easily erify. Hence, for tis input ector, te algoritm ealuates te neuron s output, Y, as label, since it is te output alue stored in entry #. Table 1: VG-RAM WNN neuron lookup table. Lookup Table X 1 X X 3 Y entry # label 1 entry # label entry # label 3 input label B. VG-RAM WNN Arcitecture for Traffic Sign Detection Our VG-RAM WNN arcitecture for traffic sign detection, named TSD, as a single bidimensional array of m n neurons, N, were eac neuron, n i,j, as a set of synapses, W = (w 1,w,...w w ), wic are connected to te network s bidimensional input, Φ, of u pixels, φ k,l (Figure 3). Te mapping of te elements of Φ onto te center of te receptie field of eac neuron of N follows a log-polar function, wic models te mapping from te retina to SC (Section II). Te synaptic interconnection pattern of eac neuron n i,j (wic consubstantiates its receptie field), Ω i,j,σ (W), follows a bidimensional Normal distribution wit ariance σ centered at ϕ μ k, μ l, were te coordinates μ k and μ l of Φ are gien by te inerse log-polar function of te coordinates i and j of N; i.e., te distribution of coordinates k and l of te pixels of Φ to wic n i,j connects ia W follow te probability density functions: ( k μk ) 1 ( k) e σ k, σ Π ( l μl ) 1 ( l) e σ l, σ Π ω = and μ σ ω =, μ σ (3) (4) were σ is a parameter of te arcitecture, and te coordinates μ k and μ l of te pixel of Φ were te Normal distribution is centered at are calculated by: were u μk = + d cos( θ ) and μ = + d sen θ, l ( ) i m u α m 1 d = and α 1 3n j π m π + ; if k < n n θ =, 3n j π m π + + ; if k > n n and α is te log-factor of te log-polar function and is a parameter of te arcitecture Te Ω i,j,σ (W) synaptic interconnection pattern mimics tat obsered in many classes of biological neurons [1]. It is randomly created wen te network is built and does not cange afterwards; furtermore, altoug random, it is te same for all neurons. Moreoer, te memory of all neurons is sared (Section III.A). Tus, wen te network is trained, eac neuron learns te association between te information collected by its synapses and a gien output, but all neurons sare eeryone else knowledge afterwards. neurons N mincinton cells synapses W input Φ image I n 1,1 > >... >... w 1 w... w W... n m,n > >... >... (5) (6) (7) (8) w 1 w... w W φ 1,1 φ 1, φ 1,3... φ k,l... φ r,s... φ u, crop, scale and filter i 1,1 i 1, i 1,3 i 1,4 i 1,5... i x,y... i ξ,η Figure 3: Scematic diagram of our VG-RAM WNN arcitecture for traffic sign detection. VG-RAM WNN synapses can only get a single bit from te input. Tus, in order to allow our VG-RAM WNN to deal wit images, in wic a pixel may assume a range of different alues, we use mincinton cells [17]. In te proposed VG-RAM WNN arcitecture, eac neuron s synapse, w t, forms a mincinton cell wit te next, w t+1 (w W forms a mincinton cell wit w 1 ). Te type of te mincinton cell we ae used returns 1 (one) if te synapse w t of te cell is connected to an input element, φ k,l, wose 73

4 alue is larger tan te alue of te element φ r,s to wic te synapse w t+1 is connected, i.e., φ k,l > φ r,s ; oterwise, it returns zero (see te synapses w 1 and w of te neuron n 1,1 of Figure 3). During training, te training image is cropped by a square centered at te traffic sign image center. Te crop square size is estimated to contain te traffic sign image and part of te background. We use distinct scale factors for te crop square size to matc arious sizes of traffic sign images. Te image patc extracted from te training image by te crop square is scaled to fit into Φ, filtered by a Gaussian filter to smoot out artifacts produced by te scaling, and its pixels are copied to Φ. Te image patc s center is used as te center of te log-polar function (center of attention in te isual field) tat maps Φ onto N. Eac neuron is ten trained to output a alue different tan zero if te center of its receptie field is witin a circle wit radius r centered at te image patc s center, and zero oterwise, were r is a parameter of te arcitecture. Te trained alue decreases as te center of te receptie field of te neurons moes away from te circle s center until it reaces zero at its border. Tis procedure is repeated for all images in te training set. During training, te scale factor used in te scaling mentioned aboe is set to a alue tat puts te traffic sign image precisely witin te circle of radius r. Figure 4 sows a training instance, were te TSD neural network is trained to detect a traffic sign in a training image. Figure 4(a) sows te training image wit te border and center of te ground trut bounding box marked wit a red square and red cross, respectiely; Figure 4(b) sows te transformed (scaled and filtered) image patc extracted from te training image; Figure 4(c) sows te log-polar mapping of Φ onto N (tis is only for isualization); and Figure 4(d) sows te output of N after training. As Figure 4(d) sows, neurons wit te center of teir receptie fields witin te circle centered at te image patc s center (compare te Figure 4(c) wit te Figure 4(d)) are trained to produce outputs wit alues iger tan zero (wite or gray), wile tose wit te center of its receptie field far from te circle s center are trained to output zero (black). Note tat circles become rectangles after our log-polar transform, and te regions around te center of te transform occupy a muc larger area (compare te representations of te circular traffic sign wit tat of te triangular traffic sign in Figure 4(c)). During testing, te test image is probed by our system at seeral points regularly spaced. We use 1 distinct scale factors for tis probing to matc arious sizes of traffic sign images, mimicking te procedure of looking at te image from different distances. Te probing is made using orizontal and ertical sifts of te center of te log-polar function (center of attention) in te test image proportional to te size of te crop square (window of attention) mentioned aboe, suc tat te square sifts yield partially oerlapped image patces, wic allows detection of traffic sign images in te boundaries of te window of attention. (a) (b) (c) (d) Figure 4: Example of a training instance of our VG-RAM WNN arcitecture for traffic sign detection (TSD). Eac image patc extracted from te test image is scaled (to fit into Φ), filtered (by a Gaussian filter to smoot out artifacts), and its pixels are copied into Φ. Te neurons ten generate teir output according to teir receptie fields. Neurons wit receptie field on regions of te image patc similar to regions of traffic sign images preiously trained generate outputs wit ig alues. After a procedure equialent to te winner-takes-its-all beaior obsered in te SC [14], te neuron wit te igest output is selected and te inerse log-polar function in te coordinates of tis neuron is used to compute te coordinates of a point (saccade target) in te image patc possibly belonging to a traffic sign image. Te log-polar function center (center of attention) is moed (saccade) to tis point in te test image and te neurons outputs are recomputed. Tis procedure is repeated one or more times, or until te log-polar function center does not moe anymore. A matcing score, tat quantifies te similarity between te center of te image patc and a traffic sign, is computed by comparing te image of te output of N after testing to tat after training (Figure 4(d)). If te output of N after testing is similar to te output of N after training, a traffic sign image migt ae been detected in te image patc. Figure 5 sows a testing instance, were neurons in te network, trained to detect traffic signs, generate teir outputs according to te image region monitored by teir receptie fields. Figure 5(a) sows te test image wit te center of attention marked wit a red cross; Figure 5(b) sows te transformed (scaled and filtered) image patc extracted from te test image; Figure 5(c) sows te log-polar mapping of Φ onto N (just for isualization); and Figure 5(d) sows te output of N before te saccade. As Figure 5(d) sows, neurons wit te center of teir receptie fields on a traffic 733

5 sign image generate iger outputs (compare te Figure 5(c) wit te Figure 5(d)). Figure 5(e) to Figure 5() are equialent to Figure 5(a) to Figure 5(d), wit te difference tat tey illustrate te testing instance after te saccade. As Figure 5() sows, te output of N after te saccade is ery similar to te output of N after training (compare Figure 4(d) wit Figure 5()), wic indicates tat a traffic sign image migt ae been detected in te image patc. An animation of a single training and seeral subsequent saccades is aailable at ttp://youtu.be/h_lde8fcbf4. Te TSD degree of belief tat a traffic sign as really been detected in te image patc is estimated using Bayesian inference, as described below in Section III.C. Te wole system final decision is regulated by a tresold: if te degree of belief is larger tan tis tresold, ten a traffic sign as been detected. Neerteless, te detected traffic sign image migt be out of te center of te image patc extracted from te test image (as it is in Figure 5(e)). To precisely find te center of te detected traffic sign image, we employ a second VG-RAM WNN tat explores te symmetry present in te traffic sign image, as described below in Section III.D. C. Degree of Belief in te Traffic Sign Detection Our system maps te matcing score tat quantifies te similarity between te center of te image patc and a traffic sign (Section III.B) into a probability measure. Tis probability measure is expressed as p(d M, S, X, Y), were D is a binary random ariable, and D = True if a traffic sign as been detected and D = False if it as not; and M = {m 1, m,, m M }, S = {s 1, s,, s S }, X = {x 1, x,, x X } and Y = {y 1, y,, y Y } are discrete random ariables tat represent te discretization of te possible alues of matcing scores, scale factors, and saccade target x and y coordinates in te test image, respectiely. Using te Bayes Teorem, te probability tat a traffic sign image as been detected in te image patc (p(d = True)), gien tat te network computed a matcing score in te interal m i, te image patc was extracted from te test image according to te scale factor s i, and te saccade target is witin te range of orizontal coordinates x i and ertical coordinates y i of te test image, can be formulated as: were p( D) p( M, S, X, Y D) p( D M, S, X, Y ) =, p( M, S, X, Y ) p ( M, S, X, Y D) = M D) S D) X D) Y D) and (10) p( M, S, X, Y ) = p( D) M D) S D) X D) Y D) +. + ~ D) M ~ D) S ~ D) X ~ D) Y ~ D) (9) (11) (a) (b) (c) (d) (e) (f) (g) () Figure 5: Example of a testing instance of our VG-RAM WNN arcitecture for traffic sign detection (TSD). To estimate te alues of p(d), p(~d), p(m D), p(m ~D), p(s D), p(s ~D), p(x D), p(x ~D), p(y D), ), and p(y ~D), we used a training subset and a alidation subset. We trained te network wit te images of te training subset, examined te TSD output wit te alidation subset, and estimated te alues of te terms of te Equations (10) and (11) mentioned aboe. Te probability tat a traffic sign as (or as not) been detected, p(d = True) (or p(d = False)), can be estimated as te percentage of saccades tat it (or do not it) te associated ground trut bounding boxes of te alidation 734

6 subset, according to te four constraints described below in tis section. Te probabilities p(m = m i D = True) (or p(m = m i D = False)) can be estimated as te percentage of matcing scores in eac interal m i, gien tat a traffic sign as (or as not) been detected, i.e., D = True (or D = False). Te probabilities p(s = s i D = True) (or p(s = s i D = False)) can be estimated as te percentage of image patces extracted according to a scale factor s i, gien tat D = True (or D = False). Te probabilities p(x = x i D = True) (or p(x = x i D = False)) can be estimated as te percentage of saccade targets witin te range of orizontal coordinates x i, gien tat D = True (or D = False). Finally, te probabilities p(y = y i D = True) (or p(y = y i D = False)) can be estimated as te percentage of saccade targets witin te range of ertical coordinates y i, gien tat D = True (or D = False). A saccade its a ground trut bounding box if four constraints are satisfied. Let d x and d y be distances in te x- axis and y-axis, respectiely, between te saccade target and te center of te ground trut bounding box; w gt te widt of te ground trut bounding box; and w ei te traffic sign estimated widt as a function of s i (w ei = c/s i, were c is a parameter of our system). Te first constraint specifies tat te saccade target must be near te center of te ground trut bounding box, i.e.: wgt d x 0.8 and wgt d y 0.8. (1) (13) Te second tat te image patc size must be close to te size of te ground trut bounding box, i.e.: w 0.7 w w 1.. (14) gt ei gt Te tird tat te matcing score must be greater or equal to 0.3, and te fourt tat te traffic sign image must belong to a pre-defined category of traffic signs. Our system final decision is regulated by a tresold: if p(d = True M = m i, S = s i, X = x i, Y = y i ) is larger tan te tresold, ten a traffic sign as been detected. Te tresold alue can eiter be specified by te system s user or automatically tuned using a training subset and a alidation subset, by arying te tresold alue until te performance of interest in terms of precision and recall is acieed (te alidation subset is part of te training dataset but not of te test dataset [18]). D. VG-RAM WNN for Traffic Sign Centralization We employ a second VG-RAM WNN tat explores te symmetry present in te detected traffic sign images to try and find teir center. Tis second VG-RAM WNN, named TSC, as te same arcitecture of te first one (TSD), preiously presented in Section III.B, and operates in te tree steps described below. In te first step, te center of attention of TSC (te center of its log-polar) is pointed at a traffic sign image detected by TSD (actually, TSD s saccade target). To do tat, an image patc, centered at te target of te TSD s saccade, is extracted from te test image by a crop square using te same scale factor used by TSD for detection. Tis image patc is scaled and filtered, and its pixels are copied to te input Φ of TSC. TSC is, ten, trained to learn te appearance of te image surrounding TSD s saccade target. Te training procedure is identical to te one followed by TSD and described preiously in Section III.B, except by te parameter r tat, in te case of TSC, it is set in suc a way tat TSC s neurons learn to output alues different tan zero in a smaller region of te traffic sign image tat TSD as possibly detected, instead of te wole traffic sign image. In te second step, te image patc is swapped orizontally and te TSC neurons generate teir output according to teir receptie fields. If TSD s saccade it te center of a traffic sign image, its swapped image will appear ery similar due to te traffic sign image symmetrical form. Terefore, in tis case, te TSC neurons will sow actiation ery similar to te one learned and a TSC saccade will not moe its center of attention. In te oter and, if TSD s saccade it a point not in te center of te traffic sign image, te image region tat will appear similar will be a region symmetric to te one learned by TSC. Terefore, te TSC neurons tat will sow actiation will be tose tat ae receptie field in regions were te swapped image is symmetrical to te learned image. A TSC saccade will, ten, moe its center of attention to tis region. Te distances in te x-axis, d x, and y-axis, d y, between te current center of attention (te target of TSC saccade), (x s, y s ), and te target of te TSD saccade, (x c, y c ), are computed and saed ( d x = xs xc e d y = ys yc ). Considering te symmetrical appearance of traffic sign images, it is expected tat TSC s saccade in tis step is mostly orizontal (i.e., d y 0 ). In te tird step, te image patc is swapped ertically and te same procedure is followed. Te distances in te x- axis, d x, and y-axis, d y, between te current center of attention (te target of TSC saccade) and te target of te TSD saccade are computed and saed. In tis case, it is expected tat TSC s saccade is mostly ertical (i.e., d x 0 ). Actually, if d y 0. d x and d x 0. d y, our system consider tat te image patc contains a traffic sign image and tat its center as been found at te point d d x y ( xc, yc ). In tis case, te crop square s center is moed to tis point and te traffic sign detection process is considered complete. Te traffic sign bounding box can ten be easily computed using tis point coordinates and te current scale factor. If te first TSC saccade is not mostly orizontal or te second is not mostly ertical, TSC neurons memories are deleted so tat tis procedure can be repeated starting from a point cosen randomly near (x c, y c ) and a iger scale factor. So, wile d > 0. d and d > 0. d and a y x x y 735

7 maximum number of iterations as not been reaced, te wole process is repeated. If te maximum number of iterations is reaced, our system considers tat te image patc does not contain a traffic sign image. An animation of te TSC operation is aailable at ttp://youtu.be/sz9w1xbwjqe. IV. EXPERIMENTAL EVALUATION AND DISCUSSION A. Experimental Metodology To ealuate te performance of VG-RAM WNN on traffic sign detection, we used te German Traffic Sign Detection Bencmark (GTSDB) (ttp://bencmark.ini.rub.de) [11]. Te GTSDB dataset contains 600 images in te training dataset and 300 images in te test dataset. All images ae a resolution of pixels. Eac image migt or migt not contain traffic signs, wic are categorized into proibitory, danger and mandatory categories tat are, in turn, subcategorized into 1, 15 and 8 subcategories, respectiely. training subset (1 samples from 0-99 range) ground trut dataset ground trut, estimates and stores te terms of Equations (10) and (11) in te Inference Table. Figure 7 sows te flow cart of te test pase of our system. In tis pase, TSD uses as input a training subset and a test subset (see Figure 7). Te training subset is composed of only 1 traffic sign images, te same used in te alidation pase. Te test dataset is composed of all 300 images (0-99 range) of te GTSDB test dataset, or te last 300 images ( range) of te GTSDB training dataset. We trained TSD te same way as in te alidation pase and tested it wit te wole images of te test dataset. TSD output was fed to te Bayesian Inference Module, tat, in tis pase, tanks to te Inference Table built in te alidation pase, is able to output a tuple tat includes p, wic is a sort for p(d = True M = m i, S = s i, X = x i, Y = y i ), and te same members of te tuple it receied from TSD (see Figure 7). If tis tuple is aboe a tresold, it is fed to TSC, oterwise it is discarded. TSC tries and finds te center of te traffic signs it receies or discards te tuple as well. Te traffic signs properly centered by TSC are outputted as bounding boxes in te test image tat can be compared wit te GTSDB dataset ground trut. training subset (1 samples from 0-99 range) Inference Table alidation subset ( range) images images TSD (ImgId, m, s i, x c, y c ) terms of Eqs. (10) and (11) Bayesian Inference Module Inference Table Figure 6: Flow cart of te alidation pase of our system. Figure 6 sows te flow cart of te alidation pase of our system, wic is used for estimating te alues of te terms of te Equations (10) and (11). In tis pase, TSD uses as input a training subset and a alidation subset (see Figure 6). Te training subset is composed of only 1 traffic sign images selected from witin te first 300 images (0-99 range) of te GTSDB training dataset. Tese 1 traffic sign images are randomly taken from tose images tat contain traffic signs of te proibitory category. Te alidation subset is composed of te remaining 300 images ( range) of te GTSDB training dataset. We trained TSD wit te 1 traffic sign images of te training subset and tested it wit te wole images of te alidation subset. TSD outputs a (ImgId, m, s, x c, y c ) tuple for eac test instance, were ImgId is te image identification number, m is te matcing score, s is te scale factor, and x c and y c are te coordinates of te saccade target (we used a single saccade for eac test instance). Tese tuples are fed to te Bayesian Inference Module, wic accumulates tese tuples for te wole alidation subset and, after tat, using te GTSDB training test dataset images coordinates of te traffic sign bounding box TSD images TSC (ImgId, m, s i, x c, y c ) (p, ImgId, m, s i, x c, y c ) (ImgId, s i, x c, y c ) Bayesian Inference Module Tresold Figure 7: Flow cart of te test pase of our system. B. Experimental Results TSD and TSC arcitectures ae 5 parameters: (i) te number of neurons, m n; (ii) te number of synapses per neuron, W ; (iii) te size of te network input, u ; (i) te standard deiation, σ, of te two-dimensional Normal distribution followed by te synaptic interconnection pattern of te neurons, Ω; and () te log-factor, α, of te log-polar function tat maps Φ onto N. We ae used te following parameters in our experiments, wic were cosen in an ad oc manner: (i) number of neurons equal to 65 49; (ii) number of synapses per neuron equal to 56; (iii) size of te network input equal to 01 01; (i) σ equal to 5; and () α equal to. We ae used 1 different scale factors, wic were cosen in suc way tat te largest traffic sign in te GTSDB database (18 18 pixels) could fit into te receptie field of te TSD neurons trained wit a alue different tan zero in te case of te smaller scale factor; and te smaller traffic sign image (16 16 pixels) could fit in te receptie field of te TSD neurons in te case of te 736

8 igest scale factor. To ealuate te contributions of eac module of our traffic sign detection system, we examined its performance (i) considering only TSD, (ii) considering TSD and te Bayesian Inference Module, and (iii) considering te wole system. Figure 8 sows te performance of our system in a precision recall cure for te GTSDB training dataset in te (i), (ii) and (iii) scenarios, named Matcing score, Probability and Probability+Symmetry scenarios, respectiely. Te grap of Figure 8 as tree cures, one for eac scenario. As te grap of Figure 8 sows, te wole system performs te best (Probability+Symmetry scenario), i.e., te area under te precision recall cure is te largest, wen using te wole system. Howeer, te performance of te system using te scenarios Matcing score and Probability are, at first glance, unexpected. Te grap of Figure 8 sows tat te area under te precision recall cure for te Probability scenario is smaller tan tat of te Matcing score scenario and te reerse would at first be expected. Howeer, tis appens because te Bayesian Inference Module remoes most of te tuples it receies een using a low probability as tresold. In te Matcing score scenario, on te oter and, many tuples are accepted as alid traffic signs and some of tem end up contributing to te recall of tis reduced ersion of our system. One may reason tat, in spite of tat, tis ersion sould be preferred and te wole system sould not include te Bayesian Inference Module. Howeer, tis would put a large burden in te TSC module, since, in te Matcing score scenario, muc more tuples would be submitted to it. 1,00 0,90 0,80 0,70 0,60 0,50 0,40 0,30 0,0 0,10 Matcing score Probability Probability+Symmetry 0,00 0,00 0,5 0,50 0,75 1,00 Figure 8: Performance of our system: precision recall GTSDB training dataset. Te difference in area between te Probability+Symmetry and te Probability cures demonstrates te benefits of using te TSC module for performing symmetry correction most tuples tat would be considered false posities are corrected by TSC and contributes for te better oerall recall sown by te Probability+Symmetry cure. We ae submitted te results of our system for traffic sign detection to te GTSDB ealuation webpage on February 8t, 013. Our system was ranked between te first 16 metods for te proibitory category, sowing an area of 85.1% under te precision recall cure. Te grap in Figure 9 sows te data we collected from te GTSDB webpage red cure wit squared data points togeter wit te performance of our system wit te training dataset preiously sown in Figure 8 blue cure wit diamond data points. As te grap in Figure 9 sows, te performance in te training set is consistent wit te performance in te testing set. Validation Test 1,00 0,90 0,80 0,70 0,60 0,50 0,40 0,30 0,0 0,10 0,00 0,00 0,5 0,50 0,75 1,00 Figure 9: Performance of our system: precision recall GTSDB test dataset. We ae also measured te performance of our systems in terms of time. Running in a Dell Alienware Aurora R1 macine, wit an Intel Core i7-930 quad-core processor (8M Cace,.8 GHz) and 1 GB of RAM, TSD can be trained in 3.85 seconds (1 traffic signs) and performs a saccade in 0.4 seconds on aerage, wile TSC performs te wole process required for symmetry detection and correction in 0.18 seconds on aerage. Terefore, if te wole system detects a traffic sign in a single TSD saccade, te total detection time of a traffic sign is 0.60 s = 0.4 s s (te consumed by te Bayesian Inference Module is negligible). Howeer, te system may not detect a traffic sign wit a single TSD saccade. Actually, to coer a single image, in our experiments we performed 780 saccades because we use seeral scale factors for te igest scale factor 16 saccades are necessary. Neerteless, te system can be used in fewer and coarser scale factors, say te smallest scale factor, wic requires a total of 6 TSD saccades. In tis case, using current desktop computers, our wole system can operate at a rate of about one image eac 3.6 s. It is important to mention tat our implementation can be optimized for taking adantage of ardware accelerators, suc as GPU, FPGA or digital signal processors, improing te time performance figures just mentioned, and, in suc case, we beliee it can be used on-line. 737

9 C. Discussion Te traffic sign detection performance of our system is te result of two factors. First, eac synapse of TSD and TSC collects te result of a comparison between two pixels, wic is executed by its corresponding mincinton cell. Our configurations for bot systems ae undreds of synapses per neuron and tousands of neurons. Terefore, during testing, undreds of tousands of suc comparisons are performed on eac input image and te results are cecked against equialent results learned from training images. Tis amount of pixel comparisons allows not only for ig discrimination capability but also generalization. Second, tanks to te synapse interconnection patterns and log-polar arcitecture, eac neuron of TSD or TSC monitors a specific region of te traffic sign, wic reduces te oerall impact of occlusion, and arying color and illumination conditions on te performance of tese systems. We beliee tat our system could sow muc better traffic sign detection performance if proper parameter tuning were employed we adjusted all system parameters in a ad oc manner and if a large number of training images were used we used only 1 traffic sign images in all experiments. We were unable to perform parameter tuning tests wit more training samples, and tests wit oter categories of traffic signs due to time constraints. V. CONCLUSIONS AND FUTURE WORK In tis paper, we present a new approac for traffic sign detection based on Virtual Generalizing Random Access Memory Weigtless Neural Networks (VG-RAM WNN). Our experiments sowed tat VG-RAM WNN can be employed for traffic sign detection wit good accuracy, despite a ery small number of training samples. Te main adantage of VG-RAM WNN against oter neural network approaces employed for traffic sign recognition is its simple implementation and fast training and test. For future work, we plan to tune te parameters of our VG-RAM WNN arcitecture and increase te number of training examples, wic can improe its performance een furter. We also would like to ealuate te proposed system on traffic signs of Brazil s road enironment. Proceedings of te nd International Conference on Industrial Electronics, Control, and Instrumentation, pp , [8] S. Maldonado-Bascon, S. Lafuente-Arroyo, P. Gil-Jimenez, H. Gomez-Moreno, and F. Lopez-Ferreras, "Road-sign detection and recognition based on support ector macines," IEEE Transactions on Intelligent Transportation Systems, ol.8, no., pp.64-78, 007. [9] X. Baro, S. Escalera, J. Vitria, O. Pujol, and P. Radea, "Traffic sign recognition using eolutionary adaboost detection and forest-ecoc classification", IEEE Transactions on Intelligent Transportation Systems, ol.10, no.1, pp , 009. [10] I. Aleksander, From WISARD to MAGNUS: A family of weigtless irtual neural macines, in RAM-Based Neural Networks, J. Austin, Ed. Singapore: World Scientific, pp , [11] S. Houben, J. Stallkamp, J. Salmen, M. Sclipsing, and C. Igel, Detection of traffic signs in real-world images: te German Traffic Sign Detection Bencmark, in Proceedings of te International Joint Conference on Neural Networks, 013 (Submitted). [1] E. R. Kandel, J. H. Scwartz, and T. M. Jessell, Principles of Neural Science, 4t ed. McGraw-Hill, 000. [13] N. Tabareau, D. Bennequin, A. Bertoz, J. Slotine, and B. Girard, Geometry of te superior colliculus mapping and efficient oculomotor computation, Biological Cybernetics, ol. 97, pp. 79-9, 007. [14] R. A. Marino, T. P. Trappenberg, M. Dorris, D. P. Munoz, Spatial interactions in te superior colliculus predict saccade beaior in a neural field model, Journal of Cognitie Neuroscience, ol. 4, no., pp , 01. [15] I. Aleksander, Self-adaptie uniersal logic circuits, IEEE Electronic Letters, ol., no. 8, pp. 31-3, [16] T. B. Ludermir, A. C. P. L. F. Caralo, A. P. Braga, and M. D. Souto, Weigtless neural models: a reiew of current and past works, Neural Computing Sureys, ol., pp , [17] R. J. Mitcell, J. M. Bisop, S. K. Box, and J. F. Hawker, Comparison of some metods for processing grey leel data in weigtless networks, in RAM-based Neural Networks, J. Austin, Ed. Singapore: World Scientific, pp , [18] F. Sebastiani, Macine learning in automated text categorization, ACM Computing Sureys, ol. 34, no. 1, pp. 1-47, 00. REFERENCES [1] S. Escalera, X. Baró, O. Pujol, J. Vitrià, and P. Radea, Traffic Sign Recognition Systems, Springer Series: SpringerBriefs in Computer Science, 011. [] D. Garila, Traffic sign recognition reisited, in Proceedings of te Mustererkennung 1999, 1. DAGM-Symposium, pp ,1999. [3] Y. Li, Real-time traffic sign detection: an ealuation study, in Proceedings of te 0t International Conference on Pattern Recognition (ICPR), pp , 010. [4] Y.-Y. Nguwi and A. Z. Kouzani, Detection and classification of road signs in natural enironments, Neural Computing and Applications, ol. 17, no. 3, pp [5] T. M. Nguyen, S. S. Auja, and Q. M. J. Wu, "A real-time ellipse detection based on edge grouping," IEEE International Conference on Systems, Man and Cybernetics, pp , 009. [6] N. Barnes, A. Zelinsky, and L. S. Fletcer, "Real-time speed sign detection using te radial symmetry detector," IEEE Transactions on Intelligent Transportation Systems, ol.9, no., pp.3-33, 008. [7] Y. Aoyagi and T. Asakura, "A study on traffic sign recognition in scene image using genetic algoritms and neural networks", in 738