STRATEGIES FOR FAST LICENSE PLATE NUMBER LOCALIZATION

Transcription

1 46th International Symposium Electronics in Marine, ELMAR June 2004, Zadar. Croatia STRATEGIES FOR FAST LICENSE PLATE NUMBER LOCALIZATION Balazs Enyedi, Lajos Konyha, Csaba Szombathy, Kalmin Fazekas Budapest University of Technology and Economics H Budapest, Goldmann ter 3. Department of Broadband Infocommunications and Electromagnetic Theory Media Technology Laboratory, enyedi@mht.bme.hu Abstract Though several effective image processing afgorithms are created nowadays, their application is ofen limited by huge computation requirements, making real time operation impossible. Generally speaking, algorithms of only higher complexity can provide better and more accurate solutions, but these require significantly more computational time and po werfil resources. Procedures based on any kind of segmenting are complicated and diverse, B special example for this is license plate number recognition. Two major issues can be dstinguished in license plate number identification, i e. the localization of the number plate in the picture and the recognitin of characters within the identified area, where the country specific signs and registration number types must also be handled Neither of the tasks mentioned is easy, requiring considerable processing time, but some measurements indicate that finding and extracting characters often takes times fonger than recognizing them. The localization strategies introduced in the following are intended to reduce this difference. Keywords: license plate number search, segmenting, histogmm, real-time 1. INTRODUCTION Cars are used in all weather conditions, therefore their number plates get dirty, scratched, and may often be bent. More to the point, cars might pass the camera at a high speed, resulting a dim picture, lost edges and contours, and images are often skewed due to the road or the camera, and many cases the recorded picture is noisy. Although the optimal camera position would be right in front of or behind the cars, most cases the pictures are taken from one of the side of the road or from above, distorting significantly the ratios of the images. In addition, the text is hidden in the picture, therefore the registration number has first to be found during the recognition procedure. 2. THE EXISTING NUMBER PLATE FINDING METHODS 579

2 46th International Symposium Electronics in Marine, ELMAR June 2004, Zadar. Croatia Due to the limitations of the extent of this paper the comprehensive introduction of all related methods is not possible, therefore only those of greater importance shall be discussed to give the Reader an overview on the currently applied procedures. There are several ways to classify the number plate finding methods, in the following one aspect shall be considered. A major field of development is the application of learning algorithms, which require several test images and user interaction to become operational. These algorithms provide a result that is decided for adequacy by the user, and, on the basis of this, modifications are made that the system learns and takes into consideration in the following decisions. It is important to ensure precise initial conditions and learning procedure, since these will later affect the efficiency of the algorithm. Such procedures are the ones based on neural networks. [7]. Another major field of development is related strictly to image processing, which does not require the time-consuming learning process. The corresponding parameters must precisely be set also in this case, but there is no learning process behind. Another important aspect of distinguishing between the existing methods relies on the fact that some algorithms are number plate specific, i.e. they are based on the recognition of the special characters and number plate standards applied in a given state, while others provide more general results, independent of the country. Special elements of number plates can be as follows: Size, ratios Colors, e.g. characters on white or yellow background, colorful characters Indication of country and region \ Character types Arrangement of characters Searching algorithms mainly rely on color information and special signs. Widely used procedures that are solely based on image processing are as follows (these usually apply several steps and/or perform several different procedures consecutively): Hough transform Top-Hat and Bottom-Hat filtering (highlights the black-white transitions) [6] Binary morpholgy algorithm (for example: classical Otsu method) [5] Edge finding methods (Sobel, Kirsch, Stochasic, Laplacien, Marr (zero crossing), Roberts, Prewitt, Canny operators) [4] [5] [2] Procedures based on the color of the background and characters Detection of special characters [3] Region-growing algorithm (RGA): By using a recursive region-growing algorithm, the dark regions (license plate symbols) surrounded by light areas (background of the license plate) can then be classified. Each region has a unique position and dimensions. [I] Checking: color, size, ratio Amalgamating regions 3. FAST LICENSE PLATE POSITIONING ALGORITHMS Most of the license plate finding algorithnk apply a certain combination of the above mentioned procedures, performing the individual steps consecutively. They require long 5 80

3 46th International Symposium Eledronics in Marine, ELMAR June 2004 Zadar. Croatia computational time, which results long execution. The obtained result is highly dependent on the picture quality, because the reliability of the algorithms degrades severely in case of noisy, complex images containing a lot of details. Unfortunately this fact cannot be overcome by the procedures, instead, the precise positioning of the camera can help the issue: the car must be photographed in such a way that the environment is excluded as possible and the license plate is as big in the picture as possible. Setting the size is especially hard in case of fast cars since the optimal instant of exposure cannot be guaranteed. The procedures elaborated and tested by our group shall be presented in the following. The major focus was on improving reliability and speed. The more the environment around a car is excluded, the better results they provide. This requirement is solely dependent on the positioning of the camera License number localization on the basis of edge finding The algorithms of this group are based on the observation that number plates usually appear as high contrast (black-and-white or yellow-and-white) areas in the picture. Letters and numbers are placed in the same row (height), therefore many changes can be observed in the horizontal intensity. A sensible solution is the detection of the changes in the horizontal intensity, because the rows including the number plate contain several harsh changes. Accordingly, the algorithm first determines the extent of changes in the intensity in every row, while in the second step the largest coherent area including the biggest changes is found. The license plate in question is very likely to be contained in these rows, but its horizontal direction must also be determined. The change values calculated previously can be used for this purpose. The strongest transitions occur at the letters (black characters on white background), this is where the biggest change can be expected within a row. A simple, effective and fast edge-finding algorithm must be used in the first step, which considerably highlights the characters at the license number, while has smaller effect on the other parts of the image. High-pass filtering has proven to be the most optimal method for this purpose. The high-pass filter s impulse response can either be finite (FIR) or infinite (IIR), which has to be chosen with the consideration of the properties of the problem. Filters of fmite, short impulse response require less computation, i.e. filtering can be performed faster. The execution can also be accelerated by selecting filter characteristics of lower complexity. Filters with an impulse response length of 7 have proven to be adequate according to our experiments. The result obtained with filtering can be seen in Fig. 1. I -====- I Fig. I. 581

4 46th International Symposium Electronics in Marine, ELMAR June 2004, Zadar, Croatia In the second step, the license plate has to' be located using the image obtained with high-pass filtering. The results of filtering are summarized for each row, then, on the basis of the statistical properties of the values obtained for each row, the height of the license plate in the image is determined (Fig.2). To accelerate the process, simply the row with the highest amplitude is selected (this is where the license plate is the most likely to be). Moving downand upwards from this line, the bottom and top side of the license plate is searched for using the following method: first the points featuring the half of the maximal value are located, then the movement is continued until the first local minimums are reached. These two minimal values might be considerably differenf therefore the alignment is performed on the basis of the bigger _- one. To obtain a more precise result, the histogram curve is flattened witli a lowpass filter, eliminating several local minimums. Fig.3. Fig. 4. The horizontal position of the license plate is to be found in the third step of the algorithm. This is done similarly to the previous case. The results in the image obtained with high-pass filtering are summarized for each column but only in the previously found field. No maximal values are investigated here, since the curve always has a minimum in the empty fields between two characters; therefore, the limit lines are determined simply on the basis of the averages (Fig. 3.). The estimated location of the license plate is depicted in Fig. 4. The license number has been split up to several independent blocks and some false results have also been obtained. Fig.5. - License plate located with a simple edgefinding algorithm I i In the last step, the possible location must be selected from these. It is obvious in the figure that the individual areas are closer to each other where the license plate must be located, while the distance between the false results is larger. Therefore, areas near to each other are merged. A maximal distance estimated on the basis of the license plate's expected size must be determined for this procedure License plate location on the basis of domains The drawback of the previously described method is that in case of images comprising many edges, i.e. containing a lot of details (e.g. a complex background, Fig. 6. and 7.), the hystogram does not sufficiently emerge from its environment, sometimes the edges around it have more emphasis (Fig. 8.). To avoid this, at the generation of the hystogram, the application of a window aligned to the width of the license plate is advisable. The size of this window is estimated on the basis of the expected dimensions of the license plate; if it was I 582

5 46th lnternalional Symposium Electronics in Marine, ELMAR-2004, June 2004, Zadar, Croatia chosen to be as wide as the image, the result of the previous algorithm would be obtained, while in case of selecting it too small, the license plate would not be located correctly. Only the horizontal values falling within the window are added up at histogram generation, and the maximal value is found for each row by shifting the window. As a result, the license plate emerges from its environment more efficiently (Fig. 9.). Fig. 6. Fig i Fig. 8. Fig The licensephte located with the window method The efficiency of the algorithms can be improved by further signal processing. Two simple methods are the investigation of ratios and areas. The fact that the ratio of the license plates height to their width falls within a certain range is made use of. If the found license plate does not fulfill this criterion, than the search procedure must be continued elsewhere. During the investigation of the areas, the domains that are too small or large for processing are also omitted, even if their ratios prove to be acceptable. 4. CONCLUSION The algorithm was implemented on i386 architecture (executed on a P4 2.4 GHz processor, on Windows XP platform), without speed optimization. Our results are as follows: 583

6 461h International Symposium Electronics in Marine, ELMAR-2004, June 2004, Zadar. Croatia The reliability of the algorithm does not improve in case of higher resolution, in fact, it actually degrades for bigger images, because the high-pass filter, the window size, the size of the license plate and the ratio parameters have been set for images of 320 x 240 pixels. The license plate is small in images with lower resolution, therefore it cannot be found safely. As a result, it is recommended to convert the images to be processed to this size before the execution of the algorithm. Often the same procedure must be performed on many data during image processing. In this case the benefits offered by certain architectures can be exploited. For example, Intel Pentium 4 processors can perform 4 floating point multiplications simultaneously by the SSE2 instruction set (this is advantageous in case of digital filtering). A further possibility for increasing the speed is to manually optimize the usage of registers. The algorithm was tested for several images of different noise level (Fig. 11.). The results reflected that the procedure is almost ideal below a certain noise level, while it degrades severely beyond this threshold. The reason is that the histogram curve does not fall below the 50% of its maximal value if the noise exceeds this limit. In case the system is to be operated in a noisy environment, this threshold can be modified, extending this way the domain of reliable operation. REFERENCES Optimization of vehicle licence plate segmentation and symbol recognition, R.P. van Heerden and E C. Botha, Department of Electrical, Electronic and Computer engineering, University of Pretoria, South Africa A Robust License-Plate Extraction Method under Complex Image Conditions, Sunghoon Kim, Daechul Kim, Younbok Ryu, and Gyeonghwan Kim, Dept. of Electronic Engineering, Sogang University, Seoul, Korea Multi-National Integrated Car-License PIateRecognition System Using Geometrical Feature and Hybrid Pattern Vector, Su-Hyun Lee, Young-So0 Seok and Eung-Joo Lee, Dept. of InformatiodCommunication Eng., TongMyong Univ. of Information Technology License plate recognition system, David Chanson and Timothy Roberts, Department of Electrical and Electronic Engineering, Manukau Institute of Technology, Auckland License Plate Recognition - Final Report, Pierre Ponce, Stanley S. Wang, David L. Wang Automatic Car Plate Recognition Using a Partial Segmentation Algorithm, Fernando Martin, David Borges, Signal Theory and Communications Department, Vigo University, Pontevedra, Spain ReFeRend YRendszdm Felismero Rendszer ), Fajt Piter,. Vacz Istvan, paper for student s scientific contest and conference, Technical College of Budapest