2010 Fifth International Conference on Frontierof Computer Science and Technology Research of improving the accuracy oflicense plate character segmentation Shuang Qiao l, Yan Zhu l, Xiufen Li l, Taihui Liu 2,3, Baoxue Zhang l,* lnortheast Normal University, Changchun,130024, China 2Changchun Institute ofoptics, Fine Mechanics andphysics, Chinese Academy ofsciences, Changchun, China 3Beihua University, Jilin, China Qiaos81O@nenu.edu.cn, luke_ciom@yahoo.com.cn Abstract In order to improve the accuracy of license plate character segmentation, this paper strengthens preprocessings before segmentation. In character binarization, use improved Bernsen method to reduce noise interference; use line-shape filter to remove plate frames. Then character segmentation approach combines projection and inherent characteristics of character, can better solve character adhesion and fracture problem. For tilted characters only in horizontal direction, even without tilt correction, this method can achieve better segmentation. Experiment results shows the accuarcy ofcharacter segmentation reaches 97%. Keywords: License plate character segmentation, Binarization, Bernsen method, Line-shape filter, projectioon. 1. Introduction Vehicle license plate recognition system, as one of crucial technology in intelligent transportation syatem, plays an important role in Traffic management automation. It contains license plate location, character segmentation and character recognition. The former two parts are core technology. After locating plate, character segmentation will directly affect the quality ofcharacter recognition accuracy. The common character segmentation methods are vertical projection algorithm, clustering analysis, connected area analysis, etc[1][2]. But in actual * Corresponding author applications, there are many disturbances, such as uneven illumination, contrast-poor, plate frames, rivet and especially declining. Therefore, this paper strengthens preprocessings before segmentation. In binary of license plate, improve the local bernsen threshold method[3], and the binarization effect is good. Meanwhile, use line-shape filter to remove plate frames, which does not depend on the plate tilt correction. For tilted characters only in horizontal direction, no need to tilt correction, combine projection and character'inherent characteristics and achieve better segmentation. 2. Preprocessing Because the following segmentation and recogmtlon is based on the plate character, it's necessary to preprocess the plate image before segmentation, including binariztion and frames removal. 2.1 Plate binarization The purpose of plate binarization is to separate characters from background well and reduce noise interference. This paper improves traditional Bernsen method, combines global and local threshold method, and results to good binary effect. Bernsen method is a kind of common local threshold method. It obtains threshold by neighborhood grayscale and maintains stroke of text image well. Considering point (x,y) as the center of template, with width 2W, algorithm describes as follows: (I)calcuate threshold ofeach point. 978-0-7695-4139-6/10 $26.00 2010 IEEE DOl 10.1l09/FCST.201O.75 489 IEEE computer society
T(x,y)=O.5( max f(x+k,y+l) + min f(x+k,y+l)) (1) (2)if f(x,y»t(x,y), then b(x,y)=l. Otherwise, b(x,y)=o. where, W is estimated biggest width ofcharacter stroke. f(x,y)is gray value of point (x,y) in original image, T(x,y) is the corresponding local threshold. max means maximum gray value in the template, min means minimum gray value. But if there is no target or all targets in the template window, the heterogeneity of background gray and some noises will cause the threshold drastic change, so the background point may mistake for target or target mistake for backgroud, Namely artifacts and stroke fracture phenomena. Usually, in order to reduce the artifact and stroke fracture phenomena, format(1) is improved to format(2). T(x,y)=a( max f(x+k,y+l) + mm f(x+k,y+l)) (2) where a is decided by experience, adjusted between 0.4 and 0.6. Meanwhile, according to actual character width, adjust parameter W. In the paper, the plate image is binarized using Otsu[4] and Bernsen method respectively, part of results are shown as Fig 1. Here, it mainly causes artifacts. That is to say, in some area the threshold is too small, so the mutative background is mistaken for character. So it's necessary to enlarge part ofthreshold values. Through the statistics, the proportion of character pixels in the whole located plate image is around 25%. Considering noises, the proportion is up to 30%. So there must be a gray value K as global threshold. Suppose h(x,y) is the histogram of plate image, K is defined as follows: 255 2.: h(x,y) k={gl max [ >30%]} (3) O:S:g:S:255 2.: h(x,y) i=o Note that the above K corresponds to the situation that the gray value of chatacter is larger than that of background. As for the contrast situation, need to reverse the gray value firstly. Of course, if we only use global threshold K to binarize plate image, it's hard to solve uneven illumination or variation of background gray. So this paper combines threshold K and Bernsen to binarize image, described as follows: (1) calculate thresholdk by format(3). (2) use Bernsen method to obtaint(x,y). (3) For each points, take maximum between K and T(x,y) as final thresholdto complete binarization. In actual application, the parameter Wand a is adjusted. Here, W is 7 and a is 0.55. The binary images by improved Bernsen method show in Fig 2. iiii.iii:iiiii.i:i...uljlmimjmd:m (a) (b) (c) Figure 1. Binary method comparison. (a )gray image. (b) Otsu method.(c) traditional Bernsen method. From Fig. 1, when plate character and background has high contrast, Otsu performs well. But if plate is located improperly, containing much non-plate information or the contrast is lower, it's difficult for Otsu to separate the characters from plate background. So it's not wise to use Otsu here. Then Bernsen'threshlod is decided by neighborhood grayscale. It's good to separate character from background. But it exists artifacts and stroke fracture phenomena. So Bernsen needs to be improved. (a) (b) Figure 2. plate binarization by Bernsen method. (a)traditional Bernsen (b)lmproved Bernsen Seen from Fig 2, improved Bernsen can reduce distrubances from plate backgroud and reach good effect. 2.2 Plate frames removal 490
Usually plate frames removal is based on horizontal projection. But it relies on plate tilt correction too much, Shown in Fig 3. If plate is corrected properly, its projection distributes regularly. The projectin exists valley in the spacing between frames and characters. Through this feature can remove frames effectively. But with improper tilt correction, there is no regular in horizontal projection. Then it's not easy to remove frames. Therefore, a new frame removal method is proposed, indepence of the plate tilt correction. mimi. (a) plate with tilt correction &Jd (b) plate without tilt correction Figure 3. Horizontal projection of plate For horizontal frames, they are is continuous in horizontal direction and its length is larger than character width at least. So line-shape filter is used to remove upperframes and lower ones. Line-shape filter, namely, is a line for filter. For horizontal filter, its width is decided by the maximum width of plate character. Here, we set it as 1.5 times of maximum character width. Scan every point in each line. When the length of continuous white points is large than filter width, these points are considered as frames, and to be removed. Considering the possible location of frames, the line-shape filter just need to scan the upper and lower area of plate, rather than the whole plate. It can prevent adhesive characters from beening removing. Use line-shape filter to filter Fig 3(b) horizontally, result shows in Fig 4. - Figure 4. Horizontal filter Seen from Fig 4, after horizontal filter, frames are not completely removed. There are still many small lines, whose height are no more than two pixels. So apply vertical filter around the possible location of frames. Detailes are: Scan every point in each column. When the length ofcontinuous white points is less than 2 pixels, those are residual frames and to be removed. Result after vertical filter shows in Fig 5. Figure 5. Vertical filter This method reduces dependence of tilt correction. As long as the plate inclination is not very serious, it can remove the plate frame effectively. Besides, because part of chinese character and digit" l"in the plate easily confuse with vertical frames, here we don't remove vertical frames in the plate. Vertical frames are to be processed in the following segmemtation. 3 Character segmentation According to prior knowledge of license plate, this paperuse projection[5] [6] [7] to segment characters. It can better solve character adhesion and fracture problem and are not sensitive to inclination. For tilted characters only in horizontal direction, even without tilt correction, this method can achieve better segmentation. It consists of two steps: coarse segmentation and fine segmentatin. 3.1 Coarse segmentation First step is coarse segmentation. Here we take Fig 3(b)for example, without tilt correction. Vertical projectin is carried to the preprocessed plate, result shows in Fig 6. Then we get several blocks with projection nonzero. They may be plate characters or noises, called candidate characters. Firstly, record start and end locations for every candidate character in the horizontal direction. Then apply horizontal projection to every candidate character and record vertical start and end locations. Finally, each candidate character can get a coarse bounding box. Figure 6. Vertical projection of plate image Here, to avoid the interference of residual frames, find the vertical start and end location from a third upward and Two-thirds of plate height downward and stop atthe first zero projection. Meanwhile, the space mark betweenthe second and third character canbe removed according its width and height. Besides, the chinese character such as "jl["and "te" may be discontinuous, and so do the broken characters. And there is still the disturbance of vertical frames. So the precessing of the most left and right block of the plate is beyond this part and described in the following fine segmentation. 3.2 Fine segmentation 491
As mentioned above, fine segmentation mainly solve three problems: (l)the rivet and part of horizontal frames may connect with characters, so the bounding box from coarse segmentation is not ideal. The upper and lower boundaries need adjustment. (2)A few characters may be broken or adhesive, so fragmented characters merging and connected character splitting must be done. (3)For the first and last character, the vertical frames may connect with it. Also, part of chinese character and number "l"in the plate easily confuse with vertical frames. It's necessary to distinguish the plate character and noises. For the first problem, make use ofthe consistency of character position. Specifically, characters ususlly distribute horizontally or slightly tilt. So the upper and lower boundaries for each character is regular and monotonous, ascending or descending, rather than frequent trend variation. According to this, adjust the upper and lower boundaries. Take Fig 7 for example. For the second character, its upper boundary is abnormal, needing adjustment. Suppose the upper boundary of the first and third character is ttl and tt3. Then the upper boundary ofthe second character should betweenttl and tt3, and exists at the position whose horizontal projection is the smallest. Adjusted boundary is shown in Fig 7(d). For the second problem, based on the average width and spacing of characters. Firstly calculate the average width and spacing of character, ignoring the small-width ones such as digit "1". When the width of projection blocks is large than 1.5 times of average width, then character adhesion comes up. According to average width, we can forecast the position range where the first character ends. Then the position with minimum vertical projection is thought to be the end boundary. Then the connectd character is split, seen in Fig 8(e). As for the brokencharacter, each widthofthe two fractures is less than average width, and the spacing is also small. But the total width of the two fractures is around average width. According to these, merge the broken character. For the third problem, it's the average width to decide whether to update the start position of the first character and the end position of the last character. Meanwhile, find out the biggest spacing among vertical projection, which can determine the character distribution. That is two characters in the left and five characters in the right. Then the fine segmentation is finished, result shows infig 7 and Fig 8. 4 Experiment result The experiment images are captured ina toll station BE -- (a)binary plate image (b)preprocessing (c)coarse segmentation(d) fine segmentation Figure 7. Example 1 of character segmentation (a)binary plate image (b)preprocessing (c)coarse segmentation (d)vertical projection (e) fine segmentation Figure 8. Example 2 of broken character segmentation in natural conditions. After plate location, the proposed method is applied to the plate images, including preprocessing and segmentation. 226 of 232 images can be segmented successfully, with accuarcy 97.41%. In our experiment, a PC with Intel CPU (T2 130, 1.86GHZ) and 1GB RAM is used. Programming in the VC++6.0[8], the total processing time is arouond 40ms. Mostly, the error is caused by fuzzy image, whichi is hard to separate characrer from background. Part of the experiment results are shown in Fig 9. - BIID (a) binary plate (b)segmentation Figure 9. Part of segmentation results 492
References [l]anagnostopoulos C-N.E., Anagnostopoulos I.E., Psoroulas I.D., et al, "License plate recognition from still images and video sequences: A Survey", IEEE transaction on Intelligent transportation system, 2008,9(6),377-391. [2] Li B., Zeng Z.-Y, Dong H.-L., "Automatic license plate recognition system", Applied Mechanics and Materials, 2010, 438-444. [3]Liu Hongwei, "License plate location and recognition", changchun, Jilin university, 2008. [4]Zhao Jianlei, "Research of the vehicle license plate recognition system", Jinan, Sandong university, 2008. [5]He x., Zheng L., WU Q., "Segmentation of characters on car license plates", Proceedings of the 2008 IEEE 10th Workshop on Multimedia Signal Processing, 2008, 4665111, 399-402. [6]Nomura S., Yamanaka K., Katai 0., et ai, "A novel adaptive morphological approach for degraded character image segmentation", Pattern Recognition, 2005,38 (11),1961-1975. [7] Li H., Zhang H., "Improved projection algorithm for license plate characters segmentation", 2009 2nd International Conference on Intelligent Computing Technology and Automation, 2009, 528834,534-537. [8]Qiu shi ke ji, The typical algorithm and its implementaion of digital image processing by Visual C++[M], 2006, Beijing University of Posts and Telecommunications, Beijing, China. 493