VEHICLE LICENSE PLATE DETECTION ALGORITHM BASED ON STATISTICAL CHARACTERISTICS IN HSI COLOR MODEL Instructor : Dr. K. R. Rao Presented by: Prasanna Venkatesh Palani (1000660520) prasannaven.palani@mavs.uta.edu
INTRODUCTION With constantly increasing traffic on roads, there is a need for intelligent traffic management system. License plate (LP) detection is widely used for detecting speeding cars, security control, traffic law enforcement and electronic toll collection. License plate detection can be performed via various approaches such as [1] Vector quantization Gabor transform Optical character recognition Neural networks.
INTRODUCTION License plate detection is a two step process Detecting the plate. Character recognition to identify the characters on the plate. This project discusses a method to select automatically statistical threshold value in HSI color space for detecting candidate regions. This will lead to a framework which unifies detection, tracking and recognition of license plates.
RGB AND HSI MODEL RGB Model In the RGB color model, different colors can be reproduced by additively combining red, green, and blue in different ways. In a general sense, the RGB color model describes our perception of color. Three types of receptors in the retina of the human eye have peak sensitivities corresponding to these three primary colors. (Fig 1). The RGB color model represents colors within a cubic volume defined by orthogonal Red, Green, and Blue axes. Black is at the origin of the coordinate system (R=G=B=0). White is at the opposite corner of the cube (R=G=B=255). The diagonal connecting the black and white corners (gray dashed line) contains the range of neutral gray levels.
HSI Model The HSI color model, represents colors within a double-cone space. (Fig 2). The vertical axis is intensity, which represents variations in the lightness and darkness of a color. The 0 intensity level is black; full intensity is white. HSI values elsewhere along the intensity axis represent different levels of gray. On any horizontal slice through the model space, the hue (or color of the color) varies around the slice, and the saturation (the purity of the color) increases radially outward from the central intensity axis. In the HSI color model, intensity makes no contribution to the color.
ALGORITHM FOR DETECTING LICENSE PLATE REGION (FIG.3) The algorithm for license plate detection consists of three parts Candidate regions are identified using HSI color model The geometrical properties of the license plate such as area, bounding box, aspect ratio, are used to filter the candidate region. The candidate region is determined by decomposing the predetermined alphanumeric character.
STEPS FOR LICENSE PLATE DETECTION As shown in Fig 4, the license plate detection involves the following steps a) Input image b) Color segmentation result c) Detected candidate after filtering d) Candidate region detection.
COLOR SEGMENTATION 1) Input Image (RGB) is converted to HSI color model through the following transformation operations.[1][6] (Fig.5)
RGB-HSI CONVERSION
COLOR SEGMENTATION-BINARIZATION Binarization results in a image whose pixels have only two possible values,0 (black) and 1 (white) (Fig 6). The thresholding method used here is the Otsu algorithm [12][13], which assumes that the image to be thresholded contains two classes of pixels then calculates the optimum threshold separating those two classes so that their combined spread is minimal.
MORPHOLOGICAL PROCESSING After segmentation, there may be some noise in the image such as small holes in the candidate regions. This could be resolved with morphological processing. Mathematical morphology operations are based on the shape in the image and not pixel intensities. There are two basic morphological operations [5] Dilation. Erosion. Dilation allows objects to expand while erosion shrinks the objects by eroding the boundaries. These operations can be modified by proper choice of the structuring element which determines how many objects will be dilated or eroded. Structuring element is simply a matrix of 0s and 1s that could be of any arbitrary shape and size.
MORPHOLOGICAL PROCESSING In MATLAB one can define neighborhood of desired size for the structuring element such as square, rectangle, diamond etc. Preferably rectangle is used as the neighborhood for the structuring element of size 6x4. In the project, closing operation is used which is dilation followed by erosion. Removal of small holes plays an important role in obtaining the rectangular license plate. Figure 7 shows morphological operation on binary image.
EDGE DETECTION, LABELING AND FILTERING Edge detection is one of the important tasks for digital image processing. Edge points possess high gradient difference in the local neighborhood. They are used for feature extraction in image processing. Considerably the image date is reduced after edge detection. Some popular edge detectors are [7], Sobel. Robert s Cross. Prewitt. Kirsch. Laplace. Marr-Hildreth. Edges are defined as intensity gradients within the image.
EDGE DETECTION, LABELING AND FILTERING Sobel Operator In this project, Sobel edge detection operator is performed. Sobel operator is a combination of two operators one which can detect horizontal edges the other which can detect vertical edges. It is a 3x3 neighborhood based gradient operator. The result of the edge image generated by Sobel operator is shown in Fig 8
EDGE DETECTION, LABELING AND FILTERING After detecting the candidate regions using color segmentation and edge detection, features of the region are to be extracted to differentiate LP regions from others. The next step of the algorithm is to label each of the connected components. Before that the image is smoothened so as to reduce the number of connected components (Fig 9). To smooth a image is to create an approximate image that attempts to capture important patterns in the data, while leaving out noise or other fine-scale structures/rapid phenomena. The connected components are labeled.
EDGE DETECTION, LABELING AND FILTERING, EXTRACTING REGION OF INTEREST It works on one connected component at a time and can move all over the image. During this step, the connected components are processed to filter the license plate region from candidate like regions using license plate characteristics like rectangular shape, aspect ratio. This could be used to extract regions in the input image so as to obtain the rectangular region of interest as shown in figure 10.
CHARACTER RECOGNITION Candidate Decomposition involves the following steps Extracting character region. Normalization of candidate region. Comparing extracted alphanumeric against predetermined alphanumeric character in the license plate region.
CHARACTER EXTRACTION The region of interest is divided in equal section, such that each contains an alphanumeric character. Template of all alphabet and numbers can be collected from the database that is of size 42x24 pixels. The 2-D cross correlation coefficient is calculated between each of the 36 templates with the character recognized in order to identify the character. There are a total of 36 templates which include all the 26 alphabets (A-Z) in English along with numbers (0-9). While computing the 2-D cross correlation coefficient, it should be ensured that image size of the character recognized and the template size should be the same. If they are not of the same size, then there is a need to resize.
EXPERIMENTAL RESULTS AND CONCLUSIONS All simulations are performed on Pentium IV processor with Dual core of 3.4 GHz with 2GB RAM under MATLAB. The images were taken using 14 MP camera that are of size 4288 x 3216 pixels. Due to memory requirements of MATLAB, the images are reduced to 500 x 436 pixels. The images were taken at a distance of few meters from the car and the camera focused the license plate region. An accuracy of 96% was achieved in identifying the license plate for a total of 35 images. In [1], the accuracy is reported as 94% for detecting the license plate under different illumination conditions. A common drawback of this method is the failure to detect the boundaries of the license plates. This typically occurs when the license plate and the car are of the same color. This can be negotiated by adjusting the threshold value in the color segmentation process. In future, the scope of the project can be extended to consider independent orientation.
EXPERIMENTAL RESULTS AND CONCLUSIONS It is observed that techniques that are based on a combination of edge statistics and mathematical morphology features gave good results [15] for binary image processing. A drawback of this technique is that when edge detection is used to detect the license plate region, it may be too sensitive to unwanted edges due to high local variance that are not of desired interest. In Hongliang et al [17], a hybrid extraction algorithm based on edge statistics and morphology was developed. This approach consisted of four basic sections: vertical edge detection, edge statistical analysis and morphology based license plate extraction. It was able to achieve an average accuracy of 99.6% for locating a vehicle license plate. The digital images were obtained from a fixed distance, angle and therefore candidate regions in a specific position were given high priority. A prior knowledge can enhance the accuracy to a great extent. Accordingly to [18], if the vertical edges of the car image are extracted with most of the background edges removed, the plate area can easily be located in the whole edge image. The success rate reported in this paper was found to be 97 %. There is still a need for a common platform where different methods can be evaluated based on performance, execution time, percentage of false alarm, percentage of detection and recognition rate.
REFERENCES 1. Automatic number plate recognition: http://en.wikipedia.org/wiki/automatic_number_plate_recognition#cite_note-2 2. Y. Wang et al, Study on HSI color model-based fruit quality evaluation, International Congress on Image and Signal Processing (CISP), Digital Object Identifier:10.1109/CISP.2010.5647943, Vol. 6, Page(s). 2677 2680, 2010. 3. Z. Xiehua and Z. Shen, A robust license plate detection algorithm based on multifeatures, The 2nd International Conference on Computer and Automation Engineering (ICCAE), Digital Object Identifier: 10.1109/ICCAE.2010.5451298, Vol. 5, Page(s): 598 602, 2010. 4. H. Lim et al, An efficient method of vehicle license plate detection based on HSI color model and histogram, Next-generation applied intelligence, Digital Object Identifier: 10.1007/978-3-642-02568-6, Vol. 5579/2009, Page(s). 66-75, 2009. 5. K. Deb, S. Kang and K. Jo, Statistical characteristics in HSI color model and position histogram based vehicle license plate detection, Intelligent service robotics, Digital Object Identifier: 10.1007/s11370-009-0043-x, Vol. 2, Page(s). 173-186, 2009. 6. W. Wang, Reach on Sobel operator for vehicle recognition, International Joint Conference on Artificial Intelligence, IJCAI '09. Digital Object Identifier: 10.1109/JCAI.2009.54, Page(s). 448 451, 2009.
REFERENCES 7. C. Yan et al, Edge detection of images based on improved Sobel operator and genetic algorithms, International Conference on Image Analysis and Signal Processing, IASP 2009. Digital Object Identifier: 10.1109/IASP.2009.5054605, Page(s): 31 35,.2009, 8. M.Donosser, C.Arth and H.Bischof, Detecting, tracking and recognizing license plates. in: Y.Yagi, S.B.Kang, I.S.Kweon and H.Zha, (eds.) ACCV2--7, Part II. LNCS, Vol. 4844, Page(s) 447-456. Springer, Heidelberg,.2007. 9. Y. Li and M. Wang. Novel and fast algorithms of license plate locations and extractions, IEEE International Conference on Information and Automation (ICIA), Digital Object Identifier: 10.1109/ICINFA.2010.5512273,Page(s). 2462-2465, 2010. 10. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, Third Edition. Prentice Hall, N.J., 2008. 11. O.Serkan and E.Ergun, "Automatic vehicle identification by plate recognition," Proceedings of world academy of science, engineering and technology, Vol. 9, Page(s). 222-225, 2005. 12. J. Zhang, J Hu, Image segmentation based on 2D Otsu method with histogram analysis, International Conference on Computer Science and Software Engineering (CSSE), Digital Object Identifier: 10.1109/CSSE.2008.206, Vol. 6, Page(s). 105 108. 2008.
REFERENCES 13. T. Yupeng et al, Automatic thresholding of gray-level pictures using two-dimension Otsu method, International Conference on Circuits and Systems (ICCAS), Digital Object Identifier.10.1109/CICCAS.1991.184351, Vol.1, Page(s). 325 327, 1991. 14. M.M.I.Chacon and S.A.Zimmerman, "License plate location based on a dynamic PCNN scheme," Proceedings of the international joint conference on neural networks, Vol. 2, Page(s). 1195-1200, 2003. 15. C.P.Marques et al, License vehicle plates localization using maximum correlation, Structural, syntactic, and statistical pattern recognition lecture notes in Computer Science, Springer Berlin / Heidelberg, Vol. 3138/2004, Page(s). 470-476, 2004. 16. C. Anagnostopoulos et al, License plate recognition from still images and video sequence; A Survey, IEEE Transactions on Intelligent Transportation System, Vol. 9, No. 3, Page(s). 377-391, 2008. 17. B. Hongliang et al, A hybrid license plate extraction method based on edge statistics and morphology, Proceedings of 17 th International Conference on Pattern Recognition, ICPR 2004, Vol. 2, Page(s). 831-834, 2004. 18. L. Lou et al, An efficient method of license plate location, 1 st International Conference on Information Science and Engineering(ICISE), Vol. 8, Page(s). 770-773, 2009.