An Approach to Korean License Plate Recognition Based on Vertical Edge Matching

Transcription

1 An Approach to Korean License Plate Recognition Based on Vertical Edge Matching Mei Yu and Yong Deak Kim Ajou University Suwon, , Korea Abstract License plate recognition (LPR) has many applications in traffic monitoring systems. In this paper, a vertical edge matching based algorithm to recognize Korean license plate from input gray-scale image is proposed. The algorithm is able to recognize license plates in normal shape, as well as plates that are out of shape due to the angle of view. The proposed algorithm is fast enough, the recognition unit of a LPR system can be implemented only in software so that the cost of the system is reduced. 1 Introduction Vehicle license plate recognition (LPR) is one form of automatic vehicle identification (AVI), which not only recognizes and counts vehicles, but also distinguishes them as a unique. LPR has many applications in traffic monitoring fields. It can save time and alleviate congestion by allowing motorists to pass toll plazas or weigh stations without stopping. It can save money by collecting and processing vehicle data without human intervention. It can also improve safety and security by helping control access to secured areas or assisting in law enforcement. License plate extraction is the key step within a license plate recognition system, which influences the accuracy of the system significantly. Some approaches to license plate extraction in color image are presented because license plates have appointed colors in their backgrounds and characters [ 1,2]. However, colors of license plates in images vary greatly due to different light conditions under which the images are taken, and vehicles may have similar color to the background of license plate. Thus, it is not easy to segment license plate directly by using information on plate s colors. Additionally, color image processing usually takes longer time than gray-scale image processing. Histogram based approaches are widely used gray-scale image processing methods in many LPR systems [3]. It is based on the fact that characters on license plate usually make strong gray level variations. However, other characters or reflection on vehicle surface will disturb license plate extraction. Moreover, it is difficult to extract license plate exactly if the plate is out of shape due to the angle of view, especially for Korean license plates that consist of two rows. Hough transform is another used approach to license plate extraction, but it requires huge memory and computational effort [4]. By contrast, license plates usually have relatively clear outline, which can be used to separate license plates from vehicle bodies. Therefore, a vertical edge matching based algorithm is proposed to extract Korean license plate from gray-scale image in this paper, and then license plate segmentation and character recognition are also discussed. 2 Korean License Plate Extraction 2.1 Edge Detection For a given gray-scale image { Gi, ; 1 I is K, 15 j< L}, its corresponding edge image can be obtained by using Prewitt or other masks. Fig. l(a) shows three input images, in which the second license plate is out of shape due to the angle of view. Fig. 1 (b) gives the edge images obtained from Fig. 1 (a), where there are many edges, both horizontal and vertical edges, which make the edge image complex. It is noticed that most of vehicles usually have more horizontal lines than vertical lines. If the two vertical edges of a license plate can be detected correctly, four comers of the plate can then be located, so that the license plate can be extracted exactly from the input image even if it is out of shape. Thus, as an alternative, only the vertical edges of input image are used to extract license plate, which are represented by where D, is vertical edge intensity function of the input image at pixel (i, j), while {Ei, j; 1< is, 15 jll } denotes the corresponding binary vertical edge image. Tin Eq.(2) is a threshold, and defined by T={? if otherwise TD = Dm + Do, (3) where D, and D, denote the mean and standard deviation of {Q, j; 15 ilk, 12 jll}, respectively. To (empirically, T0=26) is the lower limit of the threshold. The vertical edge images of Fig.l(a) are shown in Fig.l(c), which are simpler than Fig. l(b), and easier for post-processing /00/$ IEEE 2975

2 (a) Input gray-scale images (b) Edge images (c) Vertical edge images ~~ ~ (d) Results of size-and-shape filtering (e) Extracted license plates Fig. 1 Examples of license plate extraction. 2.2 Size-and-Shape Filtering Binary size-and-shape filter is very useful in pattern recognition, because it is usually needed to recognize objects with special shapes in images. As a pre-processing, the filter removes objects that do not satisfy some specific features, so makes it easy to recognize objects in postprocessing. Seed filling algorithm presented by Smith [5] is faster than the simple recursive algorithm by decreasing the recursion depth. The algorithm proceeds as follows: the contiguous horizontal run of pixels containing the starting seed is filled in. Then the row above the just-filled run is examined from right to left to find the rightmost pixel of each run, and these pixel addresses are stacked. The same is done for the row below the just-filled run. When a run has been processed in this manner, the pixel address at the top of the stack is used as a new starting seed. When the stack is empty, the algorithm terminates. In a binary image, a region is defined as a set of white pixels that are eight connected with each other. To filter the image, each region is browsed by using the seed filling algorithm, during which the information about the region is collected according to the specified features, such as width, height, coordinates of significant points, slope, etc, with respect to the interested object If the region does not satisfy the specified features, it is removed as noise by filling it with black. Otherwise, the region is kept and its information is also recorded for post-processing. For the binary image {E,,,}, the size-and-shape filter based on seed filling algorithm is described as follows: 1) Search the entire image row by row, for each white pixel E,,, in image, if it has not been checked, then run over the eight connected white region by using seed filling algorithm in which E,,, is adopted as the first starting seed of the region. 2) Check whether the region have specified features. If it does not satisfy some predefined restricted conditions, then fill the region with black, that is, remove the region as noise, since it is impossible to be the region of interest (ROI); Otherwise left it in the image as a possible interested object and record its information for pos t-processing. After that, mark whole pixels of this region as checked pixels. 3) Continue to scan the image row by row to find another unchecked white pixel as the first starting seed of a new region, until all white pixels in the image have been checked. On other point of view, the size-and-shape filter is in fact a region identification method that labels region of interest. It is able to process binary imag,e as well as multi-level image, where background is represented by zero pixels, and objects by non-zero values. Compared with some two-pass labeling algorithms, in which the eniire image is scanned for two times, the proposed algorithm accomplishes filtering and labeling in one pass since label collision is avoided due to different scan order. In Korean license plate extraction, after the vertical edge image has been obtained, it is filtered so as to remove the edges that are impossible to be the vertical edge of a license plate. Before filtering, morphological operation dilation is applied as a pre-processing. In the vertical edge image, an edge area is defined as a se1 of white pixels that are eight connected neighbors with each other. For each edge area, we check its size and shape by the seed filling based filter. If the edge area is smaller than a predefined size, or does not form a beeline whose slope is within a predefined interval, then it is removed, since it is impossible to be the vertical edge of a license pliite. Otherwise, the coordinates of the top and bottom pixels of the edge area are recorded for edge matching in post-processing. Since the vertical edges of a license plate may be cut off in the vertical edge image, edge areas are also recorded as one edge area if they have similar slope and one s top is quite close to another s bottom. The results of the size-and-shape filtering of Fig.l(c) are shown in Fig.l(d). It is clear that the filter has removed much noise in Fig. l(c). 2976

3 2.3 Edge Matching and License Plate Extraction The ratio of width to height of Korean license plate is about 2: 1, it can be used to judge whether two edge areas are the pair of vertical edges of a license plate. Consider that the real ratio in the image may deviate from the standard value due to the angle of view, the possible range of the ratio is adopted from 1.4: 1 to 3.3: 1. It is assumed that license plate does not lean quite a lot in input image, therefore, the vertical coordinates of the two vertical edges of a license plate should have small difference. It is restricted that the difference should be within half of the plate s height. Additionally, the two vertical edges of a license plate will have similar height within the vertical edge image, which is also a restricted condition of the edge matching. Here, the height ratio of plate s two vertical edges is from 0.8 to 1.2. Let L be an edge area, X and Y be its corresponding horizontal and vertical coordinates, Top, Bottom and Middle represent its top, bottom and middle respectively and the top has lower vertical coordinate than the bottom. N be the total number of edge areas in a vertical edge image. Then the edge matching is described as follows. for i=l to N-1 do 1 forj=i+l to N do I1 Do edge areas Liand Li have similar height? Height(Li ) if Height(Li) and then Height (L ) Height(Lj) I1 Do Li and Lj have similar vertical coordinates? if Middle (Li) > Top (Lj) and yhfidd,e (Li) < Bot,om (Lj) then { ratio = IxTop (Li ) - xt0p (Lj )[ + IxBottom (Li ) - XBottom (Lj 1. Height(Li) + Height(Lj) I1 Is ratio of width to height satisfies the condition? if ratio>l.4 and ratio<3.3 then { Extract the region according to the pair of Li and Lj, and check whether it is license plate region during plate segmentation and recognition; if the extracted region is plate region then exit; I 1 Here, edge areas in vertical edge image are matched with each other according to the above restricted conditions. If a pair of edge areas satisfies the conditions, it is regarded as the possible vertical edges of license plate, otherwise, another pair is checked. After the possible vertical edges of license plate have been found, the region is extracted and continuously checked whether it is real license plate region during plate segmentation and recognition. For example, after license plate is segmented, the percentage of character regions (white pixels) on a license plate is about from 10% to 40%. That is, if the percentage of character regions in the 9 possible plate region is lower than 10% or higher than 40%, it can not be the real license plate region. Thus, another pair of edge areas that satisfy the restricted conditions of edge matching is searched. When license plate has been extracted, it is normalized and sharpened into a 200x 100 gray-scale image {Pi, j}, as shown in Fig. l(e). 3 License Plate Segmentation After license plate extraction, the normalized plate {Pi.j} is segmented into a binary image {B, j}. In Korea, the most common vehicle license plates are private plate and business plate. Their backgrounds are green and yellow, while the characters are white and dark blue, respectively. Thus, the segmentation for them is a little bit different. For private plates, consider that the characters have higher luminance than the background, the segmentation is represented by B.. = 1, if P.. > T 1, ~ - P, l<i<w, l<jih, (4) where the threshold Tp is defined by T~ = P, + p0, in which the two terms are For business plates, the threshold Tp is defined by T = P, - p0, and the pixels whose gray levels are larger P than the threshold are set to be 0, while the others are 1, because in this case the dark blue characters have lower luminance than the yellow background. Different from license plates of many other countries, Korean license plate has two rows with the size of 335mmx170mm. The upper row consists of two small Korean characters of region name followed by one or two numbers of class code. The lower row is one Korean character and four big numbers to indicate the usage and serial number, respectively. Since luminance of different part of license plate may be not uniform because of the light condition, a license plate is separated into three or four parts when it is segmented. These parts are the part of region name and class code, the part of usage code, and the parts of serial number. Fig.2 gives histograms of three extracted license plates, in which the luminance of the last two plates is not uniform. The three histograms labeled by zero are that of the entire license plates. The others labeled by nonzero numbers are with respect to the labeled parts in license plate images. The license plates in Figs.2(a)-(b) are divided into three parts, while the one in Fig.2(c) is broken into four parts. For each part, the threshold Tp is calculated separately (5) 2977

4 according to Eq.(5) and Eq.(6), where W and H denote the width and height of each part. The first binary plates in the third row of Figs.2(a)-(c) are obtained by using Tp of entire plate as the threshold. It is seen from Figs.2(b)-(c) that some information on characters is lost in the two binary plates. By contrast, the second binary plates are obtained by using local thresholds of three or four labeled parts, where the lost information in the first binary plates is still kept. After license plate has been segmented into binary image, size-and-shape filter is used to remove noise on the plate, and the third binary plates give the results. Then character segmentation can be done by using the horizontal and vertical histogram of the binary license plate image, combined with the knowledge of standard characters' position on a license plate. Korean characters are normalized into 20x20 binary image, while numbers are 8x16. By global Tp; By three local Tp; Filtered plate; (a) License plate with uniform luminance 4 Character Recognition Template matching for character recognition is straightforward and can be reliable. Since characters on license plates have the same font, ternplate matching is employed for character recognition because of its relatively lower computational efforts compared with neural networks. Template matching is also more tolerant to noise than structural analysis method. In Korea, old license plates use 15 region names and 70 usage codes, while in new license plates, the corresponding numbers are 16 and 25. The total 16 region names are M Q, F&t, uta, UT-, 2+, YF!, 39, 3s, 27, 52, g?, 6%, 321, 354, S&t, and MI?. Korean characters of region name is divided into two sets, one for the first characters, denoted as set SI, the other for the second characters, denotes as S,. Similarly, we can also define the other two sets S, fix usage code and S, for "0"- "9" ten numbers. Since there are only 60 usage codes appearing in our 710 test images, the set S, just includes 60 characters. In the experiments, we use two templates associated with number "3" clue to its different font in old and new license plates. Teimplates of the four sets of characters are shown in Fig.3. Each unknown character is compared with the templates in the corresponding set, and the character is regarded as the one whose template has thc smallest distance from the unknown character. When recognizing region name, some rules are also used to raise the recognition accuracy. s,:- Fig.3 Templates of characters on Korean license plates. By global T,; By three local T,; Filtered plate; (b) License plate with non-uniform luminance I IO (c) License plate-with non-uni'form luminance Fig.2 License plate segmentation. I 5 Experiments and Analysis Experiments have been implemented to test the efficiency of the above vertical edge matching based algorithm to recognize Korean license plate in input gray-scale image, the size of which is 493x373. The test images are taken under various light conditions, such as sunshiny, raining, snowing, and cloudy days. The license plates include both old and new ones. To test the performance of the proposed algorithm under different situations, the experiments are implemented in the following six aspects: (1) license plates in normal shapes, (2) license plates that are out of shape or leaned due to the angle of view, (3) license plates which have similar color to vehicle bodies, (4) damaged or bent license plates, (5) dirty license plates, (6) degraded images (including under or over exposed images, and blurred images). 2978

5 More examples of Korean license plate extraction are given in Figs.4-8, in which the four rows correspond to the input gray-scale images, vertical edge images, results of size-andshape filtering, and the extracted license plates, respectively. It is seen that the proposed algorithm can extract license plates in normal shape as well as skewed license plates, as shown in Fig.1 and Figs.4-5. The proposed algorithm is able to extract license plates that have similar color to vehicle bodies. The algorithm also shows its efficiency in recognizing damaged or dirty license plates, as shown in Figs.6-7. Finally, Fig.8 gives some degraded test images, in which the first two images are under and over exposed, while the last one is blurred. The proposed algorithm succeeds in extracting license plates from these degraded images. Table 1 gives the rate of license plate extraction and characters recognition of the proposed approach to Korean license plate recognition. Here, license plate recognition is thought as a failure so long as one character of license plate is incorrectly interpreted. The second row in the table shows the number of correctly recognized license plates (or characters) and the total number of license plates (or characters). It is shown that the proposed approach has correctly extracted 699 license plates from total 710 test images. The 3-6 columns in the table give the character recognition rates of the 699 extracted license plates. For the total 710 input images, the number of correctly interpreted license plates is 670, and the recognition rate is about 94.37%, as shown in the last column in Table 1. The average processing time of these 710 images is about 0.28 second per image, which include license plate extraction and character recognition, when implemented on PC 586(300MHz) in Borland C language. The experimental results show that the shortcoming of the proposed license plate extraction mainly comes from unclear edges of license plates. Incorrect license plate segmentation also causes license plate to be missed even though its edges are clear in the image. When edges of other objects intersect the vertical edge of license plate in image, the size-and-shape filter becomes another factor of unsuccessful extraction since it may remove the plate s edge as noise. In character recognition, it is seen that compared with old license plate, new Korean license plate is easier to be recognized because of the new font. The Korean characters of usage code are another big problem since some of them are quite similar with each other, and no rules can be used to determine an unknown usage code, as what we do in region name recognition. Thus, the reduction of usage code in new license plate design is advisable for LPR system. vertical edge matching based algorithm to recognize Korean license plate from input gray-scale image is proposed. The algorithm is able to recognize license plates in normal shape, as well as plates that are out of shape due to the angle of view. The proposed algorithm is fast enough, the recognition unit of a LPR system can be implemented only in software so that the cost of the system can be reduced. References [I] Eun Ryung Lee, Pyeoung Kee Kim, and Hang Joon Kim, Automatic recognition of a car license plate using color image processing, IEEE International Conference on Image Processing 1994, vol. 2, pp ,1994 [2] Sang Kyoon Kim, Dae Wook Kim, and Hang Joon Kim, A recognition of vehicle license plate using a genetic algorithm based segmentation, IEEE International Conference on Image Processing 1996, pp , 1996 [3] Dong Uk Cho, and Yong Hwan Cho, Implementation of pre-processing independent of environment and recognition of car number plate using histogram and template matching, The Journal of The Korean Institute of Communication Sciences, vol. 23, no. 1, pp , 1998 [4] G. M. Kim, The automatic recognition of the plate of vehicle using the correlation coeficient and Hough transform, Journal of Control, Automation and System Engineering, vol. 3, no. 5, pp , 1997 [SI Smith A. R., Tint fill, Computer Graphic, vo1.13, no.2, pp , Conclusion In this paper, a size-and-shape filter based on seed-filling algorithm is presented so as to remove some noisy areas that do not satisfy some restricted conditions. Then a Fig.4 License plates in normal shape. 2979

6 ~~~ Fig.5 License plates that are leaned or out of shape. Fig.6 Damaged or bent license plates. Fig.7 Dirty license plates. Fig.8 Degraded images. License Plate Character Recognition for Extracted License Plates Extraction Region Name Class Code Usage Code Serial Number Correct Recognition Recognition Rate 98.45% 99.86% 98.84% 98.86% 99.61% 2980