Improvement of the Vehicle License Plate Recognition System in the Environment of Rain and Fog Zhun Wang 1, a *, Zhenyu Liu 2,b

Transcription

1 Internatonal Conference on Informaton Technology and Management Innovaton (ICITMI 05) Improvement of the Vehcle Lcense Plate Recognton System n the Envronment of Ran and Fog Zhun Wang, a *, Zhenyu Lu,b Department of Informaton,Shenyang Unversty of Technology,Shenyang,0870,Chna Department of Informaton,Shenyang Unversty of Technology,Shenyang,0870,Chna a emal: zhunwang3456@63.com, b emal:lu_zhenyu049@sna.com Keywords: Lcense Plate Recognton; Gamma Correcton; Double Color Space; Neural Network Recognton; Adaptve Fuson Algorthm Abstract. Lcense Plate Recognton (LPR) s one of the key technologes of the ntellgence of communcaton management. A certan amount of dffcultes to lcense plate recognton are caused by the envronment of ran and fog. Lcense plate recognton system for ths knd of envronment s studed n ths paper, based on the theory of dgtal mage processng, computer vson and pattern recognton technology. In order to mprove the ablty to dentfy the lcense plate, Gamma correcton algorthm and the denosng algorthm of color mage are added to lcense plate locatng method based on color. In the pre-processng of the segmentaton method based on connected area detecton, a knd of double color space bnarzaton method s proposed by the artcle. Fnally a knd of adaptve fuson algorthm based on BP, RBF, GRNN neural network s proposed to fnsh the recognton for lcense plate character. Experment shows that the method adopted by ths system appled n the bad envronment of ran and fog acheves good recognton effect. Introducton Along wth the socal lfe rhythm speedng up, the vehcle popularzaton has become nevtable trend of socal development. In a large vehcle management system, lcense plate recognton technology s a key technology n ntellgent traffc management system, whch s wdely appled to hghway, ntellgent buldng entrances, arport, parkng management and so on. In a lcense plate recognton system, due to the nfluence of the weather condton, especally ran and fog, there are many factors that can affect the recognton results of the system, such as background complexty ncreasng, lght ntensty weakenng, lcense plate staned and so on. The mage processng part of the system and lcense plate recognton algorthm need to adapt to the condton mentoned above. In vew of the lcense plate recognton of the specal condtons, targeted research and mprovement on lcense plate recognton from three phases of lcense plate locaton, lcense plate segmentaton and lcense plate recognton by ths artcle. Lcense Plate Postonng There are many knds of algorthms of lcense plate locaton., such as frequency doman analyss method based on DFT transform, locaton dentfcaton method based on neural network, lcense plate locaton method based on Hausdoff and so on. The above methods can extract lcense plate mages under certan condtons, but often fal to have an deal effect of postonng n weather condtons of ran and fog, where there are many nterference, such as blurred mages, the change of lght and soled lcense plate. They don t have very strong adaptablty to the envronment and the characterstcs of domestc lcense.in ths paper, a lcense plate locatng method based on color and texture have been adopted, accordng to the characterstcs of domestc lcense plate. Lcense plate locatng s completely splttng the lcense plate area mage from the background of a vehcle mage, whose basc dea s to fnsh extracton of lcense plate area mage accordng to the dfferent characterstcs of lcense plate regon and background regon. But vehcle mages are blurred on the whole n the weather of ran and fog, and ther pxels are low, makng the dfference and the border of lcense plate regon and background regon weaken and fuzzy, so as to ncrease the dffculty of lcense plate locatng. In order to weaken the dsturbance of envronment of ran and fog 05. The authors - Publshed by Atlants Press 670

2 to lcense plate mage postonng, and to ncrease the contrast of lcense plate regon and background regon,gamma algorthm and Gaussan denosng algorthm of color mage are added n the bass postonng algorthm. The process of postonng s shown n Fg.. The orgnal lcense plate mage Image enhancement The coarse postonng lcense plate segmentaton The accurate postonng Fg. Flow chart of postonng Gamma correcton s a knd of parameter values, whch s used to express the nonlnear characterstcs of the CRT (cathode ray tube) dsplay. In addton, the sense of human vsual system to the brghtness or RGB sgnals of three colors follow the law of logarthmc, whch s not a lnear relatonshp. Gamma correcton method s ntroduced as a transfer functon n order to overcome the nonlnearty. In general, when Gamma correcton value s greater than, hghlght part of the mage s compressed and the shade part s extended; when the value s less than, the hghlght part s extended and the shade part s compressed. Under the nterference of the weather of ran and fog, mage s more fuzzy compared wth the normal mage, whose color s relatvely shallow, whch s not conducve to lcense plate postonng based on color and texture. In the system, the parameter of Gamma correcton s set to., whch darken the mage on the whole and deepen the color of blue lcense plate compared wth the color of background part,thus n favor of the extracton of lcense plate mages. Contrast of before and after the mage enhancement of lcense plate mages s shown n Fg.. (a) before mage enhancement (b) after mage enhancement Fg.. contrast of before and after the mage enhancement of lcense plate mage After mage enhancement, the mage ntensfed by Gamma algorthm s transformed from the RGB color space to the HSV color space. Then the tonal component matrx and the saturaton component matrx of the mage are nput nto flter n order to remove ther Gaussan nose. The sze of denosng template s taken for [3,3], whch can meet the requrements of two aspects of denosng performance and mage resoluton. After the bnarzaton of the tonal component and the saturaton component of the mage, we AND them to each other. Then the coarse postonng s fnshed by a seres of morphologcal processng, ncludng openng operaton, closng operaton and convex hull operaton. In vew of the factor of the wear and tear of lcense plates and the effect of lght n the weather of ran and fog, the threshold of bnarzaton of the saturaton component matrx s set as 0.6. After coarse postonng n color space, the poston of connected doman whose area s overlarge s removed, f there exsts, by connected components detecton, n order to rule out the stuaton that vehcle appearance s blue. Fnally accurate postonng of lcense plate mage s fnshed by testng the poston of the connected doman whose area s largest n ths mage. Affected by the shootng angle, the lcense plate mage may be tlted at an angle. After lcense plate postonng, the angle of nclnaton of the plate s determned by testng the angle of x axs and the axs of the ellpse that has the same standard second-order central moment as the plate regon. Fnally the lcense plate mage s rotated at the tested angle for tlt correcton. 67

3 Character Segmentaton A character segmentaton method based on double color space and connected component detecton s presented by ths artcle. Under the weather of ran and fog, the fuzzness of the boundary of lcense plate characters and blue background can be caused by fog and water vapor between shot and lcense plate. And a certan degree of dstorton of character area may be made by refracton effect of lght presented by water drops attached on lcense plate. In addton, segmentaton results are more lkely affected by the dust and drt of lcense plate tself n the envronmental condtons of ran days. Due to the nfluence of the above n the weather of ran and fog, some algorthms that can acheve correct segmentaton n fne weather, where gudelnes of Identfyng characters and background may fal n such bad weather, lead to segmentaton fault. To solve ths problem,we must as far as possble reduce the nfluence of ran and fog for lcense plate mages before segmentaton. Therefore mprovement of pre-processng of segmentaton algorthm s necessary. At frst, the lcense plate mage s transformed from the RGB color space to the HSV color space, and then we bnary the saturaton component. The threshold value of bnarzaton s set by automatcally computng wth a statstcal method, rather than drectly determned as a fxed value or average, lke most of the lcense plate recognton systems. Threshold calculaton formula s: th 0 s = m +. () In the formula, th stands for the threshold; m s the mean value of the saturaton component matrx, and s s the mean square error value of the saturaton component matrx. The mean square error value s ntroduced nto threshold calculaton formula, n order to mnmze the nterference of envronment, makng the character outlne more clearly. It s proved that the method of threshold determnaton has an obvous mprovement effect on the process of segmentaton and recognton under the weather of ran and fog n the experment. Then we bnary the red component of lcense plate mage n the RGB color space, for blue background of lcense plate and whte characters reach the maxmum degree of dfferentaton n the red component. Threshold s determned by computng wth a statstcal method lkewse. Threshold calculaton formula s: th 0 s = m +. () In the formula, th stands for the threshold; m s the mean value of the red component matrx, and s s the mean square error value of the red component matrx. In the envronment of ran and fog, whte characters are more lkely to be affected by dust and drt, whch brngs about the change of ther color component. Therefore the mean square error s multpled by 0., n order to adapt to the change caused by dust and drt on whte characters and ncrease the accuracy of the bnarzaton. Then we AND the two results of the bnarzaton respectve n two color space to each other. The nose of threshold mage caused by water drops, dust and drt s further elmnated by medan flterng. The ntermedate regon of lcense plate s extracted wth pror knowledge, and then, n order to elmnate the nfluence of rvet area, the regon outsde the upper and lower boundary of the characters s removed, by detecton of area and shape of connected regons. Lcense plate characters, n addton to the Chnese character, are segmented by usng sortng of area and centrod as well as segmentaton method based on connected component detecton; the Chnese character s segmented out by pror knowledge. Fnally all characters are scaled to a unfed standard sze. Lcense Plate Character Recognton There are several frequently used character recognton method: neural network pattern recognton, statstc pattern recognton, structural pattern recognton, fuzzy pattern recognton, and so on. Neural network pattern recognton can deal wth some problems where envronmental nformaton s very complex, and nference rules are not clear, and background knowledge s not clear. Ths method allows mages have relatvely large mage defect and dstorton, and has certan characterstcs of 67

4 ntellgent processng. In the envronmental condtons of the fog and ran, the lcense plate mages tend to have larger defects and dstorton, so neural networks are adopted by ths artcle for character recognton. But each neural network has ts respectve adaptablty and lmtaton. For example, as a most wdely used neural network, BP neural network has the problem of easly fallng nto local mnmum. In order to acheve hgh recognton rate n the complex envronment, ths paper proposed an adaptve fuson algorthm based on BP neural network, RBF neural network, and GRNN neural network to realze character recognton. Ths algorthm can process characters on the lcense plate mages wth large nose by ntellgent processng, and can also avod some stuatons that a sngle neural network fal to dentfy characters or has dentfcaton errors, thus further mprove the accuracy of lcense plate character recognton. Ths paper adopts characters of the grd feature extracton approach, whch drectly nputs elements of the whole gray scale mage nto neural network, gettng rd of the feature extracton and enhancng the ant-ammng performance of the network. Ths approach s conductve to the network to keep hgh recognton rate. BP neural network The BP neural network s a knd of nverse learnng algorthm of mult-layer network. Its network model s composed of neuron nodes of several layers, shown as Fg.3, ncludng the nput layer, hdden layers, and the output layer. Each layer s composed of multple neurons. nput layer hdden layer hdden layer output layer nput output nput output nput output Fg.3 BP network structure (k- (k) y s set as the nput to the th neuron of the k-th layer, and the y s set as the output. The relatonshp between the nput and output s: y ( k ) = f ( (3) Before the applcaton of BP neural network, t need to be traned self-learnng. The process of learnng algorthm of BP neural network s: () The ntal values are set(connecton weghts w and threshold value θ ); () A set of tranng data s nput nto the network; (3) Forward calculaton from the nput to the output; y = f ( s (4) The errors of output data of each neuron n the output layer are calculated; δ = f ( s ) = f '( w (5) The errors of each neuron n the last layer are calculated; (6) The connecton weghts are modfed; 673 ) = δ f ( = N k f w '( W s m ( k ) y y ( m ) y ( k ) ( m ) ( m 0 ) w θ θ θ k ) ) (4) δ ) (5) (6)

5 w = ηδ y (7) ( m ) w ( t + ) = w ( t) + w ( t) (8) (7) Return to step untl the network convergence, that s, E< e 0. S-type functon s the exctaton functon of the BP neural network. When constructng the BP neural network n the expermental system, set the learnng rate of tranng as 0.5, the maxmum number of teraton as 000, the mnmum mean square error as 0.00, and the factor of momentum as 0. BRF neural network x c x c y x m c n y k Fg.4 RBF network structure Multvarate nterpolaton of radal bass functon(brf) network, belongs to the type of feed-forward neural networks, whch can approxmate any contnuous functon wth arbtrary precson so as to be partcularly suted to solve classfcaton problems. Ths network s a knd of feed-forward neural networks of excellent performance, wth compact topologcal structure, fast convergence speed, and structure parameters whch can acheve separately learnng., and fundamentally solve the problem of local optmal of BP network. RBF network and fuzzy logc can complement each other well, thus mprove the abltes of learnng and generalzaton of neural network. Its structure s shown as Fg.4. As s wdely used as the radal bass functon of network, Gaussan functon s set as the radal bass functon of the RBF neural network n ths artcle. Its formula s: P C R ( P) = exp (9) σ The self-organze algorthm s adopted by ths artcle to fnd the locaton of RBF centers, whch s one of the many knds of learnng algorthm of RBF neural network. Ths algorthm ncludes two stages, supervsed learnng and self-organzed learnng. Gaussan functon s radally symmetrc, and s proportonal to the response accordng to the output of the hdden layer of RBF neural network. When the nput s the same as the radal dstance of the center of bass functon, ther outputs of the nodes of hdden layer are the same. And the center of hdden layer nodes C and the radus of mplct functon δ are needed to worked out.. The center of bass functon of the RBF network s worked out by k-mean clusterng algorthm, whose process s: () The center of clusterng s ntalzed, that s, randomly select I dfferent samples from tranng samples are randomly selected as the ntal center t (0) (=,,,I), accordng to the experence. And the number of teraton steps n as 0; () Tranng samples X k are randomly nput nto the network ; (3) The center whch s nearest to tranng samples X k s found, that s, the (X k ) that comply wth the followng formula s found: ( X ) = arg mn X t ( n), =,, I In the formula, t(n) s the nth center at the nth teraton. (4) The center s adusted by the followng formula: k k (0) 674

6 t ( n) + η ( X t ( n + ) = k t ( n)), 当 =,, I t ( n) other () (5)The udgment s made that the process of learnng all the tranng samples wth the results of changeless dstrbuton have been fnshed or not. If yes, the process of learnng s ended up; otherwse, n=n+,and return to (). The calculated t(,, I) are the fnal center of bass functon. The fundamental prncple of determnng the radus of mplct functon s to make the sum of recevng domans of all the hdden unts covers the whole sample space. After the sample classfcaton, radus of mplct functon s calculated accordng to K-nearest algorthm: δ = K x( k) C K k= () The RBF neural network s traned, by usng lnear least square method, and then the connecton weghts between the hdden layer and the output layer are worked out. The number of neurons n the hdden layer s set as 5, and the largest number of neurons s set as 400. At frst sample mode counter p and tranng tmes q are set as. The error E s ntalzed as 0, and the error threshold of the experment E mn as Then the ntal learnng rate s set as 0.6. The network s general error s: GRNN neural network Generalzed regresson neural network(grnn) s a knd of new neural network proposed by Amercan scholar Donald F.Specht n 99, one knd of radal bass functon(rbf) neural network. Set on the bass of P~n non-parametrc estmaton, the network can estmate contnuous varables and converges n the basc regresson surface. Set on the bass of mathematcal statstcs, GRNN neural network s also the extended form of probablstc neural network(pnn). GRNN neural network can solve nonlnear problems well, wth strong ablty of nonlnear mappng and hgh degree of fault tolerance and robustness. The structure of GRNN network s shown as Fg.5. nput pattern sum output E = P P p= ( E p ) (3) x P y S x y P S x n y k P m S Fg.5 GRNN network structure GRNN neural network s manly composed of four parts of structure: nput layer, model layer, summaton layer, and output layer. Wth clear meanng of possblty and completely specfed structure and connecton weghts, GRNN neural network overcomes the dffcultes on desgn and tran of general RBF network, and though lack of sample data, GRNN network can also acheve excellent predctng results, wth faster tran speed and stronger nonlnear mappng capablty. The network can fnsh tranng wthn a moment wth a hgh fttng precson. The expresson of the output of GRNN network s: 675

7 Y( X) = n = n D Y exp( ) σ D exp( ) σ = T D = [( X X ) ( X X )] (5) In the formula, X and Y ( =,, n) are the nputs and outputs of the th sample, respectvely. And n s the number of samples. X stands for the nput vector, and Y(X) stands for the output of the network when the nput s X.σ stands for the wdth of sensng feld. Dfferent tranng effect s realzed by adustng the value of σ. Newgrnn s the constructor functon of GRNN neural network, whose format s: [ net, tr] = newgrnn( P, T, SPREAD) (6) SPREAD, the smoothng parameter of bass functon, remans the default.0. On the bass of three methods of character recognton by neural network, adaptve fuson algorthm based on the three neural network s proposed by ths artcle. adaptve fuson algorthm The three neural networks above are traned by a large number of mage fles, producng three traned neural networks respectvely named ym_bp.dat, ym_rbf.dat and ym_grnn.dat. Three groups of recognton results are obtaned by nputtng the mage after postonng and segmentaton to the three traned networks n the system. The nformaton fuson follows the prncple as follows: If results of three three groups are the same, the fnal result s determned as the only result; If results of only two of the three groups are the same, the fnal result s determned as the same result of the two groups; If results of three groups are dfferent, then the fnal result s determned as the result of GRNN neural network whch has the hghest accuracy n the three networks. The logc dagram of recognton s shown as follows: A A A A A A B A (7) A B C XRGNN (4) Test results The algorthms of all modules have been realzed through programmng n Matlab., and the experment effect s shown as Fgure

8 (a) The orgnal lcense plate mage (b) Lcense plate mage (c) Lcense plate characters after segmentaton (d) The result of lcense plate recognton Fg.6 The result of experment 805 lcense plate pctures n dfferent perod of tme and dfferent angle of nclnaton are collected by shootng wth dgtal camera and surfng on the nternet, n whch there are 377 lcense plate mages shootng n the weather condtons of ran and fog. All 805 lcense plate mages are composed of 0 fgures, 4 Englsh letters and Chnese characters ncludng Gu, Su, Lao, Jng, Mn,Yue,Qong, J, E, Xang, J, Gan, Meng, Nng, Qng, Lu, Jn, Shan, Hu, Chuan, Jn, Xn, Zhe, Yu. Typcal lcense characters are selected from the mage lbrary to nput the three neural networks after segmentaton for tranng of networks, untl the three networks acheve ther states of convergence. Then the overall performance of ths lcense plate recognton s tested. All the 377 lcense plate mages n the weather of ran and fog n the mage lbrary are nput to the expermental system to test the system performance, n whch there are 369 lcense plate mages of correct recognton, and 8 of error recognton. The rezults of the experment are shown n Table. Expermental results shows that the lcense plate recognton system we buld for the weather of ran and fog have acheved 97.8% of hgh recognton rate. The correct recognton means that no error character exsts n the result of recognton. In order to further study ts performance, ths system s compared wth another lcense plate recognton system wth no mprovement. It s found that the effect of the segmentaton acheves bg mprovement, and the recognton algorthm acheves hgher character recognton rate n our system. But as the prce of mprovement of recognton rate, the speed of recognton has been a lttle slower. And t s found n the results that 4 hghly reflectve 677

9 photos shot at nght have been not correctly recognzed, accountng for half of the number of error recognton mages. So results are not satsfactory when hghly reflectve mages are processed by the system, whch s the defect of ths system. Table Lcense plate recognton result statstcs System Total Number of correct recognton accuracy Concluson recognton The mproved lcense plate % recognton Contrastng system % In vew of the dffcultes caused by the weather of ran and fog to lcense plate recognton, some mprovements and nnovaton are made n mage enhancement, lcense plate postonng, pre-processng of lcense plate segmentaton and fuson of multple neural networks. It s proved that ths system of lcense plate recognton has acheved good recognton performance through the experment. Acknowledgements [] Industral proect of Shenyang technology bureau, research and development of mult-channel vdeo enhancement system, proect number: F [] Educaton department of laonng provnce, proect number:0404 References [] Langlang He. Study and Realzaton of A Knd of Lcense Plate Recognton Technology n Chnese [D]. South Chna Unversty of Technology, 04. [] Zhfan Feng. Study of Lcense Plate Recognton Based on Graphc Processng and BP neural network n Chnese[D]. Wuhan Unversty of Scence and Technology, 0. DOI:0.7666/d.y [3] Wewe Chen. System of Lcense Plate Recognton and Informaton Management Based on Neural Network n Chnese[D]. Wuhan Unversty of Technology, 00. DOI:0.7666/d.y [4] Dongan He, Nan Geng, Ykuan Zhang. Dgtal Image Processng. X an: X'an Electronc Sence &Technology Unversty Press, 008 [5] Lele Zhang. Study of Lcense Plate Recognton Based on Radal Bass Functon n Chnese[D]. X an Unversty of Scence and Technology, 03 [6] L Wang, Xaohua Wang. Applcaton of Neural Network n Lcense Plate Recognton n Chnese [J]. Mcrocomputer and Its Applcatons[J],0, 30(5):38-40 DOI: (0) [7] Xaochun Zhang, Kachun Ren. Study of Lcense Plate Recognton Based on RBF Neural Network n Chnese[J]. Chnese Journal of Automotve Engneerng, 0, ():47-5 DOI: /.ssn [8] Fuqang Xu, Tngtng Zheng. Functon Approxmaton Based on Generalzed Regresson Neural Network(GRNN) n Chnese[J]. Journal of Chaohu Unversty,00,(6):-6 DOI: (00) [9] Yan Zhang, Anhu Ren. Study of Multple Features and Lcense Plate Recognton System Based on BP Neural Network n Chnese[J]. Scence Technology and Engneerng, 0,() DOI: (0) [0] Zhbang Xu, Xaoyng Sh. Desgn and Realzaton of Automatc Lcense Plate Recognton System Based on RBF Neural Network n Chnese[J]. Journal of Nanchang Unversty,009,3():47-50 DOI: (009)

10 [] Defeng Zhang. Applcaton Desgn of Neural Network n MATLAB n Chnese[M]. Beng: Chna Mechne Press,0: