Sentient Autonomous Vehicle using Advanced Neural net Technology

Transcription

1 Sentent Autonomous Vehcle usng Advanced Neural net Technology J.B.Sddharth Jonathan, Arvnd Chandrasekhar and T.Srnvasan, Department of Computer Scence and Engg, Sr Venkateswara College of Engneerng, Srperumbudur, Inda. Abstract Over the past decade, the feld of automated ntellgent transport systems has been the focus of rgorous research. Ths paper proposes Sentent Autonomous Vehcle usng Advanced Neural net Technology (SAVANT), an automated transport system wth sgnfcant advantages over prevous attempts n ths feld. The system uses a mult-layer feed-forward neural network wth back propagaton learnng. In addton, the desgn of SAVANT nvolves the convergence of a plethora of technologes lke a Global Postonng System (GPS), a Geographc Informaton System (GIS), and laser rangng. SAVANT can gude a moble agent through a hostle and unfamlar doman after beng traned by a human user wth doman expertse. One of the many areas n whch SAVANT scores aganst the competton s that the system s completely doman ndepent and ncurs substantally less processor overhead. SAVANT thus provdes more functonalty even though t requres consderably less nput as compared to other attempts n ths feld. Ths reducton n the sze of the nput vector translates nto more effcent and faster processng. Another of SAVANT s hallmark features s ts ablty to negotate turns and mplement lane-changng maneuvers wth a vew to overtakng obstacles. It does ths by employng a novel technque, Selectve Net Maskng. A smulaton of SAVANT s neural network was performed on a varety of network topologes, and the best network selected. Index Terms Back propagaton learnng, Frontal and Sde Impact Collson Vectors, Selectve Net Maskng I. INTRODUCTION Man has long dreamed of desgnng machnes that are capable of operatng themselves, one of the basc mlestones to develop truly ntellgent systems that can ntutvely learn how to perform specfc operatons and execute them to perfecton. One feld n whch several forays have been made to ths s that of automated transport systems. The prmary test of any ntellgent navgaton system s ts ablty to negotate a standard road or hghway envronment successfully by dentfyng curves n the road and obstacles n ts path and take approprate acton n each case. Changng lanes or overtakng slower vehcles s also a desrable feature. A large number of the systems proposed to date [1]-[4] use a vdeo camera feed as nput. In [1]-[3], ths nput s fed to a base neural network. The vdeo feed ncurs massve processor overhead, thereby ncreasng the complexty and sze of the network and reducng the effcency of the system as a whole. The desgn of a system that rvaled [1] but dd not use a neural network was presented n [4]. However, all these systems are constraned by the doman n whch they operate. Another ssue plagung these systems s that they are not lghtng ndepent, as they operate based on a camera feed. B. Fresleben et al presented, n [5], an nterestng system that reled on strategcally placed sensors nstead of a vdeo feed. We use a smlar concept n our approach. Ths paper proposes SAVANT, a hghly advanced system wth consderable advantages over prevous attempts n the feld of ntellgent transport systems. SAVANT uses a multlayer feed forward neural network wth back propagaton learnng. In addton, the desgn of SAVANT nvolves the fuson of a varety of technologes lke a Global Postonng System (GPS), a Geographc Informaton System (GIS) and laser rangng. SAVANT can gude a moble agent through a hostle and unfamlar doman after beng traned by a human user wth doman expertse. Ths system ncurs a lot less processor overhead compared to the other approaches dscussed. SAVANT thus provdes more functonalty even though the nput vector s far smaller than n earler systems. Ths reducton n the sze of the nput vector translates nto more effcent and faster processng. Another salent feature of SAVANT s ts ablty to change lanes and pass a slower vehcle or a statonary obstacle ahead of t. The rest of the paper s organzed as follows. Secton II descrbes the prevous work n ths feld. The proposed system and ts desgn s presented n Secton III. All algorthms relevant to the operaton of SAVANT are dealt wth n Secton IV. In Secton V, the performance analyss of SAVANT wth a number of network topologes s addressed and the best topology of those tested s subected to more rgorous analyss. Fnally, Secton VI summarzes the paper and gves nsght nto possble further development and future drectons of ongong work. II. RELATED WORKS Each of these followng attempts has had ts share of success accompaned by specfc drawbacks. Here, we consder four exstng approaches:

2 A. ALVINN and MANIAC ALVINN (Autonomous Land Vehcle In a Neural Network) and MANIAC (Multple ALVINN Networks In Autonomous Control), developed at Carnege Mellon Unversty to control the NavLab vehcles there, are outftted wth computercontrolled steerng, acceleraton and brakng. Sensors nclude color stereo vdeo, scannng laser range fnders, radar and nertal navgaton. Both systems requre tranng by a human drver for about fve mnutes (followed by applcaton of the back propagaton algorthm for ten mnutes) before they are ready to drve. The sgnal from the vehcle s vdeo camera s preprocessed to yeld an array of pxel values that are connected to a 30x32 grd of nput unts n the neural network. ALVINN computes a functon that maps from a sngle vdeo mage of the road to a steerng drecton. The output s a layer of 50 unts, each correspondng to a steerng drecton. ALVINN s performance s mpressve n the cases where t has been traned. It s unable to drve on a road type for whch t has not been traned, and s also not very robust wth respect to changes n lghtng condtons or the presence of other vehcles. MANIAC (Multple ALVINN Networks In Autonomous Control) s composed of several pre-traned ALVINN networks, each traned for a sngle road type that s expected to be encountered durng drvng. The superstructure combnes data from each of the ALVINN networks and does not smply select the best one. The output from the ALVINN networks can be taken from ether ther output or hdden unts. B. ELVIS ELVIS s a road-followng system based on ALVINN usng the same nput and output, but wthout usng a neural network. Lke ALVINN, ELVIS observes the road through a vdeo camera and observes human steerng response through encoders mounted on the steerng column. ELVIS learns the egenvectors of the mage and steerng tranng set va prncpal component analyss. These egenvectors roughly correspond to the prmary features of the mage set and ther correlatons to steerng. Road-followng s then performed by proectng new mages onto the prevously calculated egenspace. Whle ELVIS prncpal component analyss mnmzes the total error n the mage reconstructon and steerng vector, ALVINN drectly mnmzes the steerng output alone.although the speed of tranng of the two systems s roughly equal, ALVINN s faster at run-tme because t requres far fewer calculatons. The prmary advantage of ELVIS s ts smplcty and lack of pre-defned structure. C. Sensor networks One of the prmary dsadvantages of a system usng a vdeo camera feed s the enormously hgh processng power requred. The performance can be mproved by usng a sensor network that deps on a number of non-vdeo sensors such as range fnders on the vehcle to sense the envronment around the vehcle and take approprate acton. SAVANT s essentally a sensor network, such as that descrbed n [5]. III. THE DESIGN OF THE PROPOSED SYSTEM SAVANT brngs together a plethora of technologes, namely a Global Postonng System (GPS), a Geographc Informaton System (GIS), a laser transcever bank and a multlayer feed forward neural network. A. GPS The GPS n SAVANT obtans the coordnates of the locaton of the vehcle. Ths nformaton s relayed to the GIS n order to obtan nformaton necessary for the operaton of SAVANT. B. GIS The characterstc of a GIS that s most relevant to our dscusson s the fact that one can pont at a locaton, obect, or area on the screen and retreve recorded nformaton about t from off-screen fles. Wth specfc reference to SAVANT, the data from the GIS that are relevant are () road locatons/maps, () road angles, () road wdths, and (v) Fg. 1. General organzaton of SAVANT. sgnals.. Hghly dynamc nformaton such as the color of the sgnal at the ntersecton ahead can be obtaned by mantanng a connecton to the local traffc polce records. All other data s readly avalable from standard GIS. Thus, based on the current locaton of the vehcle, all necessary detals can be obtaned. In bref, we see that the GPS s nstrumental n pnpontng the precse locaton of the vehcle, whle the GIS provdes real tme nformaton based on ths locaton. C. Laser transcever banks The transcever bank conssts of arrays of laser beam emtters and detectors. In a standard confguraton, lght s sent from the source by an emtter. When the beam reaches an obect, t s reflected back to the source. A detector at the

3 source receves the reflected beam and ndcates that an obect s present. Based on the tme between emsson and detecton T, and the speed of the laser beam S, the dstance D of the obect from the transcever bank s determned. D = S T / 2 (1) ( ) The system possesses two banks of transcevers. The frontal transcever bank provdes a Frontal Impact Collson Vector (FICV) whch holds the nformaton necessary to detect and avod an obstacle n front of SAVANT. The transcever banks at the sdes of the vehcle provde a Sde Impact Collson Vector (SICV) whch holds the nformaton necessary to detect the presence of obstacles that are parallel to SAVANT n the adacent lanes. The SICV s used durng the process of overtakng by changng lanes. The neural network uses the collson vector nformaton to take approprate acton (such as applyng the brakes). D. Multlayer Feed Forward Neural Network The network under study here has two layers and one hdden layer. Its nput vector s comprsed of FICV Current speed Traffc sgnal nformaton Road wdth (left and rght) The output vector s comprsed of Steerng angle Accelerator pressure Brake pressure Based on nput, approprate output values are generated by the neural network on the bass of the weghts, thereby gudng the vehcle along the rght drecton at the approprate speed. 1) SAVANT algorthm IV. ALGORITHMS functon SAVANT() defne: network, a multlayer network nput, nput vector output, output vector DataAcquston() network NeuralNetworkLearnng() nput values from the nput sensors output RunNetwork(network, nput) f (RoadAngle = 0) and (output.steerngangle 0) output Overtakng(output) output Turn(RoadAngle, output) f Implement the output vector ndefntely Ths s the prmary drver functon of SAVANT. Frst, the tranng set sample data s acqured (DataAcquston). Ths s followed by the neural network learnng phase where the lnk weghts are updated based on the nput-output sample sets (NeuralNetworkLearnng). After ths comes the real-tme applcaton of SAVANT. The system contnuously reads the nput from the varous sensors, derves the approprate outputs by runnng the neural network and mplements these outputs. 1) DATA ACQUISITION algorthm functon DataAcquston() Read nput from FICV Speedometer readng Traffc sgnal nformaton GIS (Road wdths) Read the human drver s response regardng Steerng angle Accelerator pressure Brake pressure Formulate the tranng set entry whch comprses of the nput vector and the Drver response. Add the tranng set entry to the tranng set table. untl of tranng run return Ths s the ntal perod where a human drver trans the vehcle. In effect, the system ust observes the human s actons under dfferent nput condtons. 2) NEURAL NET REAL TIME LEARNING algorthm functon NeuralNetworkLearnng( ) returns a network defne: network, a multlayer feed forward Neural Network wth randomly assgned weghts sample a structure wth content Input, nput vector Output, output vector, the learnng rate for each sample n tranng set sample.input nput vector sample.output output vector network BackPropUpdate(network, sample, α )

4 untl network converges return network The (nput, output) pars obtaned from the human drver durng the data acquston phase are now appled to the network to set t up. The network lnk weghts are updated by means of Back Propagaton. 3) BACK PROPAGATION functon BackPropUpdate (network, example, α ) returns a network nputs: network, a multlayer network example, a structure contanng vectors nput and output α, the learnng rate O RunNetwork(network, example.nput) Err T O W W + α a Err g' n ( ) for each subsequent layer n network do W k g' return network ( n ) W W + α I k k Back Propagaton s an effectve method of dvdng the contrbuton of each weght to the error. We try to mnmze the error between each target output and the output actually computed by the network. Here, Err represents the dfference between the output O and target T whch s the desred output. I ndcates the nput to unt, W the weght on the lnk from unt to unt, α the constant learnng rate and g' refers to the a the actvaton value of the unt. ( ) dervatve of the actvaton functon of the weghted sum of nputs to the unt. s the product of g'. 4) TURNING algorthm n Err and ( ) functon Turn(θ, output) returns output nputs: θ, The turnng angle n degrees. A value n the range of 0 to 180 degrees mples a rght turn and a value from 180 to 360 degrees mples a left turn. output, output vector flag 1 f θ s dentfed to be a rght turn n flag 0 f flag = 1 magntude 360 θ magntude θ lmt celng(magntude/10) terator 0 whle terator < lmt f (flag = 0) 10 output.steerngangle = output.steerngangle = 10 terator terator + 1 Implement the output vector lmt magntudemod10 terator 0 whle terator < (lmt 1) f (flag = 0) 1 output.steerngangle = output.steerngangle = 1 terator terator+1 mplement the output vector f (flag=0) output.steerngangle = 1 output.steerngangle = 1 return output The GIS nput for road angle s montored external to the neural net at all tmes. If the road angle s not zero, the neural net output for steerng angle s subected to Selectve Net Maskng, an mplementaton technque by whch specfc nodes of the neural network are overrdden whle the rest of the neural network operates as normal. Here, the steerng angle output s completely determned by the turnng system and overrdes the exstng value calculated by the neural network. The system negotates turns frst n steps of 10 and then n steps of 1 to acheve the desred turn angle accurately. 5) OVERTAKING algorthm functon Overtakng(output) nput: output, output vector defne: network, a multlayer network nput, nput vector flag, a bnary varable ndcatng an ongong ncomplete overtakng maneuver SICV, nputs from the sde mpact laser bank f(sicv 0)

5 output RunNetwork(network, nput) return output f push output.steerng drecton onto stack flag 1 mplement the output vector output RunNetwork (network, nput) f (nput.ficv = 0) f (Stack s not empty) output.steerng drecton (-1 top of stack) Pop the topmost element off of the Stack flag 0 f f (output.steerng drecton 0) and (nput.ficv 0) push output.steerng drecton onto stack untl flag=0 return output responded correctly to as a functon of the sze of the tranng set used to tran the network. The results are shown n Fg. 4. When the system fnds that the neural net output for steerng angle and the road angle are both non-zero, t realzes that an opportunty to overtake exsts. It frst checks ts SICV to see f there are any obstructons to ts sde. If not, the system turns n the requred drecton by 10 degrees. It sets a flag to ndcate that t has not straghtened yet and pushes the steerng angle on to the stack. It does ths so that once the car has turned by the approprate amount to overtake the other car by changng lanes, t can straghten tself by poppng the contents of the stack one by one and mplementng the complement of the steerng drecton. The flag s reset once the stack s empty, and the system then resumes ts usual modus operand. V. PERFORMANCE ANALYSIS We compared three scenaros wth varyng neural network topologes as shown n Table I. These topologes dffer only n the number of nodes n the hdden layer. TABLE I NUMBER OF NODES PER LAYER IN EACH SCENARIO Scenaro Input layer Hdden layer Output layer We frst compare the three neural net topologes on the bass of the number of epochs they take to stablze and the fnal error at whch they stablze. The back propagated error as a functon of the number of epochs of tranng n each of the scenaros s presented n Fg. 2 and 3. On analyzng the graphs, we found that the network represented by Scenaro 2 stablzed the fastest. We then plotted the percentage of standard test set entres ths network Fg. 2. Back propagated error as a functon of the number of epochs of tranng n the ntal stages of tranng n (a) Scenaro 1, (b) Scenaro 2, and (c) Scenaro 3.

6 Fg. 3. The back propagated error as a functon of the number of epochs of tranng n each of the scenaros. Fg. 6. The percentage of the test set values for whch the network responds correctly as a functon of the number of entres n the tranng set. VI. CONCLUDING REMARKS We present a possble alternatve approach to the above. Here, the traffc sgnal status s not part of the nput set. Instead, a Real Tme Operatng System s used to overrde the neural network when a red sgnal s detected. When the GIS ndcates a red sgnal, control s temporarly wrested from the neural network. The RTOS executes an approprate nterrupt handler the stuaton (say, by slowng down). When the nterrupt servce routne completes executon, control s transferred back to the neural network. Selectve Net Maskng can be used to overrde specfc nodes of the neural network lke the brake pressure node. Ths enables the neural network to retan over all control, whle stll allowng the RTOS to handle red traffc sgnals by takng approprate acton. Another possble drecton of further work s to develop ads to support such ntellgent systems, along the lnes of the methods dscussed n [11]. SAVANT marks a mlestone n the evoluton of ntellgent transport systems. Its numerous advantages lke doman ndepence, ease of tranng, low processor overhead, reducton n sze of the nput vector and the ablty to handle complex operatons lke overtakng and turnng make t truly revolutonary. Whle there are, as always, certan avenues for mprovement or modfcaton, SAVANT lays the groundwork on whch several smple yet effcent ntellgent transportaton systems can be based n the future. The technology s very exctng and promsng. It s only a matter of tme before the automoble gants wake up to the neural net revoluton and employ systems smlar to SAVANT to manufacture cars that are truly autonomous n every sense of the word. REFERENCES [1] C. E. Thorpe, M. Hebert, T. Kanade and S. Shafer, "Toward Autonomous Drvng: The CMU Navlab," IEEE Expert, V 6 #4, August [2] D. A. Pomerleau, Neural Network Percepton for Moble Robot Gudance, Kluwer Academc Publshers, 1993, ISBN [3] T. M. Jochem, D. A. Pomerleau and C. E. Thorpe, "MANIAC: A Next Generaton Neurally Based Autonomous Road Follower," Proceedngs of the Internatonal Conference on Intellgent Autonomous Systems, [4] J. Hancock and C. Thorpe, "ELVIS: Egenvectors for Land Vehcle Image System," Proc. of IEEE/RSJ Internatonal Conf. on Intellgent Robots and Systems, Pttsburgh, PA, 1995, pp [5] B. Fresleben and T. Kunkelmann, "Combnng Fuzzy Logc and Neural Networks to Control an Autonomous Vehcle," Proceedngs of the 2nd IEEE Internatonal Conference on Fuzzy Systems, IEEE Press, 1993, pp [6] R. Aufrere, J. Gowdy, C. Mertz, C. Thorpe, C. Wang, and T. Yata, "Percepton for collson avodance and autonomous drvng," Mechatroncs, Vol. 13, No. 10, December, 2003, pp [7] M. Bertozz and A. Brogg, "GOLD: A Parallel Real-Tme Stereo Vson System for Generc Obstacle and Lane Detecton," IEEE Transactons on Image Processng, [8] J. Hancock, M. Hebert, and C. Thorpe, "Laser Intensty-Based Obstacle Detecton", Proceedngs 1998 IEEE/RSJ Internatonal Conference On Intellgent Robotc Systems (IROS '98), Vol. 3, October, 1998, pp [9] C. Wang, C. Thorpe and A. Suppe, "Ladar-Based Detecton and Trackng of Movng Obects from a Ground Vehcle at Hgh Speeds," IEEE Intellgent Vehcles Symposum (IV2003), June, [10] C. Thorpe, R. Aufrere, J.D. Carlson, D. Duggns, T.W. Fong, J. Gowdy, J. Kozar, R. MacLachlan, C. McCabe, C. Mertz, A. Suppe, C. Wang and T. Yata, "Safe Robot Drvng," Proceedngs of the Internatonal Conference on Machne Automaton (ICMA 2002), September, [11] P. Grffths, D. Langer, J. A. Msener, M. Segel and C. Thorpe, "Sensor Frly Roadway and Vehcle Systems," IEEE Instrumentaton and Measurement Technology Conference, Budapest, Hungary, May 2001.