Automatic license plate recognition

Transcription

1 Automatic license plate recognition Mohamed El-Adawi Helwan Faculty of Engineering Hesham Abd el Moneim Keshk Helwan Faculty of Engineering Mona Mahmoud Haragi Helwan Faculty of Engineering Abstract: Automatic license plate recognition (LPR) is one of the techniques that can be used for the identification of vehicles. The purpose of this paper is to investigate a suitable way to recognize the registration plate from an image of vehicle, The proposed algorithm recognizes the Arabic (Hindi)digits and words within the plate area. The obtained recognition ratio is about 80% for a 65 random images included in the database. The algorithm failed only in images that include abnormalities such as fixing screws, or a major slant in the plate. keywords: thresholding, edge detection, dilation, watershed transform, template matching, OCR. 1.Introduction There has been a number of software products that can be used for LPR mainly from USA and Europe.V.Kamet and S.Ganesan [1] used the Hough transform (HT) for line detection to locate the boundary line of the vehicle license plate. They first applied a modified Sobel operator for edge detection and then an image thresholding is done depending on the average brightness of the image. HT is applied to the entire image, and peaks are searched in the Hough space in which the lines are identified in real space. Color and texture of the license plate have been used to identify the plate lines. J.A.G.Nijhuis [2] incorporated fuzzy features of the license plate, using yellow rectangular area and black characters. He used a discrete time cellular neural network (DTCNN) as the feature extractor and multi-layered perception (MLP) as the recognizer. Yap Keem Siah, Tay Yong Haur, Marzuki Khalid and Tahir Ahmed [3] used an iterative threshold to fully exploit the large contrast between the black color and background and with color characters of the license plate, They used the blob analysis method to identify the connected regions of pixels in the image and then the basic features(such as height, width, top and left) of the region is calculated. These regions are commonly know as blobs, They used a fuzzy artmap neural to cluster these basic features into different classes, The clustering produced blobs that satisfy the original features, which are considered as the characters of the license plate. Xulio Fernandez and Fernando martin [4] used the character location method to license plate recognition, They designed an algorithm to locate all zones in the image that may contain characters. Their model is based on derivatives, where they computed the horizontal Sobel gradient of the whole image and search for high positive values followed by a high negative ones. They look for rectangular zones in which they have a repetition of the pattern. The distance between the positive values and the negative ones must be approximately constant, Their algorithm tries to find the possible rectangular that enclosed the plate. Another algorithm proposed by J.R.Cowell.[5] where he applied a global threshold to the whole image and finds the plate using template matching and growing regions techniques. This method required a fixed size for the plate area and also the same degree of illumination, which can not be satisfied easily. Su-Hyun Lee,Young-Soo Seok and Eung-Joo Lee[6] proposed license plate recognition system for multinational vehicle license plate using geometric features along with hybrid and seven segment pattern vectors. In the proposed system, they suggested to find horizontal and vertical relation after going through preparation process with inputted real-time license plate image of Korea and Japan, and then to classify license plate with using characteristic and geometric information of license plates. It classifies the extracted license plate images into letters and numbers, such as local name, local number, classification character and license consecutive numbers, and recognize license plate of Korea and Japan by applying hybrid and seven segments pattern vectors to classified letter and number region. David Chanson and Timothy Roberts [7] used Image processing techniques such as edge detection, thresholding and

2 resampling in their algorithm to locate and isolate the license plate and the characters. Template matching was the character recognition method that was used to successfully recognize the license plate. Jaime José Laracuente-Díaz and Manuel Toledo-Quiñones [8] determined to partion the image that contains the license plate, the different objects present in the image are found and labeled. Those that have geometrical characteristics similar to those of license plate characters are selected. If the relative position of the selected objects is consistent with a license plate, it is assumed that the plate has been found. 2.LPR advantages and applications One of the advantages of LPR systems is that all cars already have or at least are supposed to have the required identification which is the registration plate. This means that there is no need to add transmitters or special signs to the vehicles, which in turn results in savings towards the implementation of the system. Another advantage is that the image of the car and may be even the image of the driver can be stored and retrieved if there is a need for evidence in disputes or crime situations. LPR systems have several applications ranging from allowing car park entry to detection of stolen cars. Long term parking at the airport is also a good candidate for the use of an LPR system as the actual time of entry can be retrieved in the event of a lost ticket that was issued days ago or in case there is a ticket fraud situation. In a border control situation, the LPR system can be used to double check that the car is allowed to cross and shorten the time it takes to go through the border, hence shortening the typical long queues. The LPR system can also be used as a clever way of directing traffic at the entrance of an organization s property. For instance, vehicles entering could have their plates read and they could be directed to the suitable lanes depending on their privileges as specified in a database. An elaborate LPR system can also be used to monitor traffic on the roads in order to detect stolen cars or cars with unpaid fines. 3.Proposed VLP recognition algorithm In such algorithms, one of the most difficult tasks is to locate the license plate, which could be any where in the image. We make the following assumptions about plates proposed in this algorithm : 1.The plates have a rectangular shape with two rows of characters. 2.The plate has dark character on a bright white back ground. 3.The width-height relationship of the license plate is approximately known in advance. 4.The orientation of the license plate is approximately aligned with the axes(vertical and horizontal). 5.Lines of image must not be optically distorted. We have been working with color images captured with a digital camera and transferred to computer memory. Our images consist of a rectangular array of 640*480 pixels. Fig.1 shows a sample of such images. Fig.1 Car image with recognizable LP 3.1 Extraction of the license plate The purpose of this stage is to locate the plate that encloses the license number. The whole idea depends on the boundary of the license plate only because there are a huge data in the image. The algorithm tries to decrease the amount of data in the image and avoid the unwanted areas to extract only the area which have the same properties of the license plate. Our proposed algorithm goes as follow: 1- Thresholding of the input image. After turn the input color image to a grayscale, we compute a global threshold level that can be used to convert a gray image to a binary image. In a grayscale image, the thresholding transformation sets each gray level that is less than or equal to a chosen value T ( the threshold value ) to 0 and each gray level greater than T to 1.The result is a black and white image with 0 representing black and 1 representing white. In our algorithm the threshold value is not constant, Iit depends on the average brightness of the image. We used the otsu s method [9] to determine that value.figs.2 and 3 show the applications of different threshold values to convert the image into binary image.

3 Fig.2 The computed threshold is 127 Fig.3 The computed threshold is Apply a thin edge detector If an image consists of objects of interest on a contrasting background, an edge is a transition from background to object or vice versa. The total change in intensity from background to object is called the strength of the edge. There are many methods of performing edge detection. We detected the edges here by using the spatial filtering. We used convolution kernel for horizontal direction [-1 1]and for vertical direction is [-1;1].Those values will help us to detect any thin edge in the image as shown in fig.4 4- Reject small lines that are impossible to be a part of the plate borders in both directions. For each point checking if there is a new point in the same line within certain distance. If it is, the line segment is extended to include the new point; otherwise a new line segment is started. In this project the maximum distance, was chosen as the value which gave the best results for the training images. Also, as some of the line segments are potentially very short, specifying a minimum line segment length is advantageous and helps reduce computation in later steps. The long lines segments from the image have been extracted. The short lines segment in the letters and in the top part have been eliminated. Fig.6 shows the output of the line extract process. Fig.6 Image after limination of short lines and extraction of long lines 5- Now we have a group of lines in horizontal and vertical direction with different lengths. By adding the two images, we obtain many regions with different areas. We used the watershed transform to segment those regions each with it s properties. Fig.7shows the result of the adding. Fig.4 Horizontal and vertical edge detector 3- Dilation the image. The obtained image after edge detector is one pixel thickness for each edge. If there is any ripples on any line, then the line will appear as a broken one. To avoid this problem we will make a dilation for all lines in both horizontal and vertical directions as shown in fig.5 Fig.5 Dilation in both directions Fig.7 Adding image The watershed transform is the method of choice for image segmentation in the field of mathematical morphology. The watershed transform can be classified as a region-based segmentation approach. The intuitive idea underlying this method comes from geography, It is that of a landscape or to pographic relief which is flooded by water. Watersheds being the divide lines of the domains of attraction of rain falling over regions. An a alternative approach is to imagine the landscape being immersed in a

4 lake, with holes pierced in local minima. Basins (also called catchments basins') will fill up with water starting at these local minima, and, at points where water coming from different basins would meet, dams are built. When the water level has reached the highest peak in the landscape, the process is stopped. As a result, the landscape is partitioned into regions or basins separated by dams, called watershed lines or simply watersheds. When simulating this process for image segmentation, two approaches may be used: either one first finds basins, then watersheds by taking a set complement; or one computes a complete partition of the image into basins, and subsequently finds the watersheds by boundary detection. To be more explicit, we will use the expression `watershed transform to denote a labeling of the image, such that all points of a given catchments basin have the same unique label, and a special label, distinct from all the labels of the catchments basins, is assigned to all points of the watersheds. Fig.8 shows the output of the watershed transform [10]. technique, which depends strongly on comparing each portion of the investigated image to a template image(shown in fig.9).the true license plate in the image is found as the region bearing the most resemblance to the template. Fig.9 the proposed template image A common way to perform this task is to use a cross correlation scheme[11].fig.10(a) shows an example of a true plat be selected, and the corresponding a mesh of the cross correlation factor where the maximum value is Fig.10(b) shows an example of a true plat be selected, and the corresponding a mesh of the cross correlation factor where the maximum value is Fig.10(a) Cross correlation with true plate 9000 Fig.8 watershed transform output 6- As mentioned, the output from watershed segmentation consists of a list of regions, of which one hopefully contains the license plate. Two steps are performed in order to select the best candidate. Area: The candidate region with small area or which have very large area will be voided, Height- width ratio: Although some times a region with the license plate will contain some edges around the actual plate, the ratio provides a mean of sorting out a lot of regions that could not possibly contain a license plate. 7- In some cars we found that the back light indicators have almost the same characteristics of the plate (area and ratio between width and height) as shown in fig.8(regions number 1 and 3). To distinguish between such false areas and the true license plate (as shown in fig.8 region number 2 ), we used a template matching Fig.10(b) Cross correlation with false region 3.2 character segmentation: 1.The next step after the identification of the rectangular plate is to isolate its characters or digit contents. All plate rectangle are assumed to contain two rows of characters. For the image, we need to isolate the two lines as shown in fig.11. Fig.11 the extract plate and the required isolation

5 To do this we calculate the horizontal summation of all black pixels in each row. The peaks of the summation will identify the two rows of characters as shown in fig.12. The two arrows point to the location of the number and registration parts. Fig.14 The two types of plate registration 4.Experimental results: The system was tested for 65 images under several conditions. The recognition ratio was very good (almost 80%).The 20%wrong identification came from the fixation screws of the plate that was recognized as zero. If the distance between camera and car was larger than certain limit, the system fails to correctly identify the plate. Fig 15(a)and fig.15(b)shows the application of the proposed algorithm and the response of the program was to identify the plate number and its registration. Fig.12 The horizontal projection 2.After isolating the two rows of characters within the plate area, we take the digits row alone and try to isolate each digit. 3.On the digits row we vertically count the number of black pixels in each column. This is actually a vertical projection. The peaks of the projection will represent the existence of a digit, and the valley will represent the isolation or boundary between these digits. Fig.13 shows this projection. Fig.15(a) Fig.13 The vertical projection 3.3 Character identification: The final task of the LPR is the recognition of the plate contents after the isolation process in the pervious step. We used a 3 layer neural network trained with the back propagation algorithm to identify the following objects in each plate :- 1.The digits from 0 to 9. ". ملا آي " word 2.The 3.The two word القاهرة" "and الجيزة" "that exits in all cars that are registered in either Cairo or El_giza. " ملا آي القاهرة " the We used ten plate from registration and another ten from the " registration as the training set ملا آي جيزة " for the neural network. The remaining plate images were used for testing. The performance of the neural network was very good. Fig.15(b) 5.Interpretation of results: We test the algorithm on 65 various images, the results are as follows: 89% success rate for license plate recognition, the algorithm failed in recognition the plates which has a major slant. 93% success rate for character recognition of the extracted plates, the failed occurs for the plate which has fixing screws in the character position. 96% success rate for character recognition of all the characters on the extracted plates, the illumination and

6 the blurring of the characters was the main reason of the failed cases. 60Comparsion with other similar projects: Although all the mentioned methods deal with the English letters and Arabic numbers which are mainly different with the characteristics of the Arabic letters and Hindi numbers, we can say that our work has a good ratio (80%) against to (70% ) success rate mentioned in [7], (83.5 %) in [6] and (74% ) in [8]. 7.Conclusion Our system was able to identify license plate with a good accuracy. The source of errors was due to mainly the fixation screws of the plate, or due to a major slant in the plate axis. The algorithm will be extended to include different shape of plates with different registration. 8.References: [1]Kamat,V.and Ganesan,S, An efficient implementation of the Hough transform for detecting Vehicle license plates using DSP S Cellular Neural Networks and their application proceedings. The IEEE 31st annual 1997International Carnahan Conference,1997,pp [2]Nijhuis J.A.G,Brugge,M.H.,Helmholt K.A., lincese plate recognition using DCTNNs, Security Technology,1997,Proceedings 1998 fifth IEEE international workshop on publish,1998,pp [3] Yap Keem siah,tay Yong Haur,Marzuki Khalid and Tahir Ahmed, Vehicle licence plate recognition by fuzzy artmap neural network, Centre for Artificial intelligence and robotics (CAIRO),Faculty of Electrical Enginerring,University Teknologi Malaysia,Jalan Lumpur,54100 Kuala Lumpur. [4] Xulio Fernández Hermida,Fernando Martín Rodríguez,José Luis Fernández Lijó,Fidel Pita Sande,Miguel Pérez Iglesias.Departamento de Tecnologías de las Comunicaciones.E.T.S.I.T.Universidad de Vigo. Ciudad Universitaria s/n Vigo (Pontevedra).Spain. An O.C.R.for V.L.P. [5]J.R.Cowell. Syntactic Pattern Recognizer for vehicle identification Numbers.Image &Vision Computing (1995). [6] Multi-National Integrated Car-License Plate Recognition System Using Geometrical Feature and Hybrid Pattern Vector Su-Hyun Lee,Young- Soo Seok and Eung-Joo Lee Department of information/communication Eng.,TongMyong Univ.of Information Technology [7] License plate recognition system,david Chanson and Timothy Roberts,Department of Electrical and Electronic Engineering,Manukau Institute of Technology,Auckland. [8] Artificial Vision for Vehicle Monitoring at Tren Urbano Stations Jaime José Laracuente- Díaz and Manuel Toledo-Quiñones Electrical and Computer Engineering Department University of Puerto Rico,Mayagüez Campus Mayagüez,Puerto Rico [9] ]N.Otsu, A Threshold Selection Method from Gray-Level Histograms, IEEE Transactions on Systems,Man,and Cybernetics,vol.9,no.1,pp.62-66,1979 [10] Luc Vincent and Pierre Soille, Watersheds in Digital Spaces:An Efficient Algorithm Based on Immersion Simulations, IEEE Transactions of Pattern Analysis and Machine Intelligence,vol.13,no.6,June 1991,pp [11] J.P.Lewis, Fast Normalized Cross Correlation,Industrial Light&Magic. /nvisioninterface/nip.html [12] Serra,J.,Image Analysis and Mathematical Morphology,Vol.2:Theoretical Advances.Newyork:Academic,1988 [13]Rafael C.Gonzalez,Richard E.Woods. Digital Image Processing 1993,Addison-Wesley