Shadow And Highlight Invariant Colour Segmentation Algorithm For Traffic Signs

Transcription

1 Shadow And Highlight Invariant Colour Segmentation Algorithm For Traic Sign Haan Fleyeh Department o Computer Engineering Dalarna Univerity orlänge, Sweden hl@du.e Abtract Shadow and highlight repreent a challenge to the computer viion reearcher due to a variance in the brightne on the urace o the object under conideration. Thi paper preent a new colour detection and egmentation algorithm or road ign in which the eect o hadow and highlight are neglected to get better colour egmentation reult. Image are taken by a digital camera mounted in a car. The RG image are converted into HSV colour pace and the hadow-highlight invariant method i applied to extract the colour o the road ign under hadow and highlight condition. The method i teted on hundred o outdoor image under uch light condition and it how high robutne; more than 95% o correct egmentation i achieved. Keyword colour egmentation, colour detection, road ign outdoor image. I. INTRODUCTION Colour repreent an important part o the inormation provided to the driver to enure the objective o the road ign. Thereore, road ign and their colour are elected to be dierent rom the nature or rom the urrounding in order to be ditinguihable. Detection o thee ign in outdoor image rom a moving vehicle will help the driver to take the right deciion in good time, which mean ewer accident le pollution, and better aety. Shadow and highlight repreent a big challenge to computer viion reearcher. In the irt cae dierent part o the object are expoed to dierent illumination level and in the econd cae the object relect ome o the light o the illuminant directly to the viewer. The aim o thi paper i to tudy thee two cae and develop a colour egmentation algorithm which i invariant to hadow and highlight. Identiication o road ign i achieved by two main tage: detection and recognition. In the detection phae, the image i pre-proceed, enhanced, and egmented according to the ign propertie uch a colour or hape. The output i a egmented image containing potential region which could be recognized a poible road ign. The eiciency and peed o detection are important actor which play a trong role in the whole proce becaue it reduce the earch pace and indicate only potential region. In the recognition tage, each o the candidate i teted againt a certain et o eature to decide whether it i in the group o road ign or not, and then according to thee eature they are claiied into dierent group. Thee eature are choen in order to emphaize the dierence among the clae. Colour repreent the baic cue or road ign detection; however hape are ued by ome reearch group to detect road ign. Fleyeh [1-] developed dierent colour egmentation algorithm to work in dierent light condition including poor light condition uch a now all and og. Colour contancy i ued in thi algorithm to correct the colour o the ign in uch poor light condition. Shadeed et al. [4] propoed a colour egmentation algorithm in which RG image wa converted into HSV and YUV colour pace applying proper threhold to H and UV value and the two reult were combined by an AND operation. énallal and Meunier [5] developed a computer viion ytem which wa embedded in a car and capable o identiying road ign. Segmentation wa achieved by the RG colour pace. de la Ecalera et al. [6] built a colour claiier baed on two look-up table derived rom hue and aturation o an HSI colour pace. Fang et al. [7] developed a road ign detection and tracking ytem in which the colour image rom a video camera were converted into the HSI ytem. Colour eature were extracted rom the hue uing a two-layer neural network. The remaining o the paper i organized a ollow. Section 2 decribe the propertie o the road ign. Section how the diicultie behind working in the outdoor cene and the eect o dierent actor on the perceived image. Section 4 decribe how the colour varie in the outdoor image and the parameter aecting thi and in ection 5, there i a decription o the propertie o the hue and how it change due to light variation. Section 6 decribe the theoretical background o the eect o hadow and highlight on colour. Section 7 how the colour egmentation algorithm, and ection 8 how the reult and uture reearch. II. TRAFFIC SIGNS: PROPERTIES Road and traic ign have been deigned uing pecial hape and colour very dierent rom the natural environment, which make them eaily recognizable by driver [8]. Thee may be principally ditinguihable rom the natural /06/$ IEEE CIS 2006

2 and/or man-made background [9]. They are deigned, manuactured and intalled according to tringent regulation [10]. They are deigned in ixed 2-D hape like triangle circle octagon or rectangle [11, 12]. The colour are regulated to the ign category (red = top, yellow = danger [1]. The inormation on the ign ha one colour and the ret o the ign ha another colour. The tint o the paint which cover the ign hould correpond to a peciic wavelength in the viible pectrum [10, 14]. The ign are located in well-deined location with repect to the road, o that the driver can, more or le expect the location o thee ign [1]. They may contain a pictogram, a tring o character or both [14]. The road ign are characterized by uing ixed text ont and character height. They can appear in dierent condition including partly occulted, ditorted, damaged and clutered in a group o more than one ign [11, 14]. III. TRAFFIC SIGNS: POTENTIAL DIFFICULTIES Due to the complex environment o road and the cene around them, the detection and recognition o road and traic ign may ace ome diicultie. The color o the ign ade with time a a reult o long expoure to unlight, and the reaction o the paint with the air [15, 16]. Viibility i aected by weather condition uch a og, rain, cloud and now [16]. The color inormation i very enitive to the variation o the light condition uch a hadow cloud and the un. [15-17]. It can be aected by the illuminant color (daylight, illumination geometry, and viewing geometry [18]. The preence o object imilar in color to the road ign in the cene under conideration, like building or vehicle. Sign may be ound dioriented, damaged or occulted. I the image i acquired rom a moving car, then it i oten uer rom motion blur and car vibration. IV. COLOUR VARIATIONS IN THE OUTDOOR IMAGES One o the mot diicult problem in uing colour in outdoor image i the chromatic variation o daylight. A a reult o thi chromatic variation, the apparent colour o the object varie a daylight change. The irradiance o any object in a colour image depend on three parameter: The colour o the incident light: Daylight colour varie along the CIE curve. It i given by: 2 y = 2.87x.0x or 0.25 x 0. 8 (1 The variation o daylight colour i a ingle variable which i independent o the intenity. The relectance propertie o the object: The relectance o an object ( i a unction o the wavelength o the incident light. It i given by:. ( = e( φ( Where e ( i the intenity o the light at wavelength, and φ ( i the object albedo at each wavelength. The camera propertie: The oberved intenitie depend on the len diameter d, it ocal length, and the image poition o the object meaured a angle a o the optical axi. Thi i given by: 2 E( = L(.( π / 4( d / co(4a ( According to thi equation, the radiance L ( i multiplied by a contant which will not aect object oberved colour. y cancelling the camera len chromatic aberration, only the denity o the oberved light will be aected. A a reult, the colour o the light relected by an object located outdoor i a unction o the temperature o the daylight and the object albedo, and the oberved irradiance i the relected light urace caled by the irradiance equation [18, 19]. V. HUE PROPERTIES AND ADRIFT DUE TO ILLUMINATION CHANGES Hue play a central role in the colour detection. Thi i becaue it i invariant to the variation in light condition a it i multiplicative/cale invariant, additive/hit invariant, and it i invariant under aturation change. ut the hue coordinate i untable, and mall change in the RG can caue trong variation in hue [1], and it uer rom three problem. Firtly, when the intenity i very low or very high, the hue i meaningle. Secondly, when the aturation i very low, the hue i meaningle. Thirdly, when the aturation i le than a certain threhold, the hue become untable. Vitabile et al. [14] deined three dierent area in the HSV colour pace: The achromatic area: characterized by or v 0.2 or v The untable chromatic area: characterized by and 0.2 v The chromatic area: characterized by 0. 5 and 0.2 v 0.9. In order to obtain robutne againt change in external light condition thee area hould be taken into conideration in any colour egmentation ytem. VI. REFLECTANCE WITH WHITE ILLUMINATION A. The relection model Conider an image o a certain urace patch which i illuminated by an incident light with a certain Spectral Power Denity (SPD denoted e (. Thi image i taken by a camera (2

3 with RG enor characteried by their pectral enitivitie C ( or C = { R, G, }. The Cth enor repone o the camera i given by: C = mb + m C ( e( c C ( b e( c ( d ( d or C = { R, G, } where c b ( and c ( are the body and urace albedo repectively, i the wavelength at which the enor repond and n, v are unit vector repreent the direction o the normal vector to the urace patch, direction o the ource o illumination, and direction o the viewer, repectively, igure 1. (4 In thi equation, C w i the repone o the RG enor under the aumption o white light ource, and k C i given by: kc = C ( cb d (6 where k C i the compact ormulation depending on the enor and the urace albedo only. I the aumption o white illumination hold then R( d = G ( d = ( d = (7 and the relection o the urace can be given by: Cw = emb kc (8. Eect o hadow on colour invariance aed on the dicuion in the ormer ection, colour invariant againt hadow (urace illuminated by dierent level o brightne i teted or dierent colour pace. Conider the body relection term in equation (8 C b = emb kc or Cb = { Rb, Gb, b} (9 Figure 1. Traic Sign Scene. Furthermore, the term m b and m denote the geometric dependencie on the body and urace relection component, repectively [20]. Auming that urace albedo c ( i contant and independent o the wavelength, and white illumination i ued (white illumination mean equal energy or all wavelength within the viible pectrum, then e ( = e and c ( = c which are contant. The enor repone can be modiied a: Cw = emb kc C ( d (5 or Cw = { Rw, Gw, w} where C b i the enor repone or the RG under the aumption o a white light ource. According to equation (9, the colour perceived by the enor depend on three actor: Senor repone and urace albedo repreented by k C. Illumination intenity e. Object geometry (urace orientation and illumination direction m b. The ollowing colour pace are examined or variation o brightne. 1 RG colour pace: The RG colour pace i eaily aected by all o the actor mentioned above ince C R = emb (10

4 C G = emb (11 C = emb k (12 Each o the RG channel i aected by the enor repone, urace albedo, illumination intenity, urace orientation, and illumination direction. 2 The intenity I: The intenity I i deined a: R + G + e mb ( + k I = = (1 According to thi equation, the intenity I depend on the illumination intenity, the object geometry, and the enor and urace albedo. In other word intenity i variance to illumination intenity and it i aected by hadow. The Nrgb colour pace: The Nrgb colour pace i inenitive to urace orientation, illumination direction, and illumination intenity a: k R r = k R + k (14 g = k R + k (15 k b = k R + k (16 4 Hue and Saturation: The hue and aturation are invariant againt variation o brightne which mean that they only depend on the enor repone and urace albedo a long a the white illumination condition hold. The hue angle and aturation are given a: [( + ( ] (17 θ = co 2 ( + ( ( min( k,, 1 R k S = (18 ( + k object itel. The irt component i called the highlight and it i generated becaue the urace o the object work like a mirror relecting the light rom the ource to the viewer. The contribution o the body relection component C b and the urace relection component C i added together a hown i equation (8, and hence, the meaured colour repreent the contribution two component. Teting the dierent colour pace according the aorementioned argument how that only the hue eature i invariant to the highlight component o the perceived colour. In the ollowing are ome teting reult. 1 RG colour pace: The RG colour pace i eaily aected by highlight component ince Rw = emb (19 Gw = emb (20 w = emb k (21 Each o the RG channel i aected by the enor repone, urace albedo, illumination intenity, urace orientation, and illumination direction. It i alo aected by the highlight component. 2 The intenity I: The intenity I i deined by: Rw + Gw + w I = + e mb ( + k = (22 According to equation (22, the intenity I depend on the illumination intenity, the object geometry, and the enor and urace albedo. Furthermore, it i alo aected by the highlight component. The Nrgb colour pace: The Nrgb colour pace i enitive to the highlight becaue it depend on the contribution o the urace relection component a: Thi mean that hue and aturation are only aected by enor repone and urace albedo, i.e. they are invariant to hadow. C. Eect o highlight on colour invariance Under certain condition when the viewer i ituated in a poition o that the angle between the vector nv and n hown in igure 1 are approximately equal, the viewer receive two component o light. The irt component i a light rom the ource relected on the urace o the object to the viewer, and the econd component repreent the light relected by the e mb r = + emb ( + k (2 e mb g = + e mb ( + k (24 emb k b = + e mb ( + k (25 4 Hue and Saturation: Following the ame procedure ued in the aorementioned colour pace hue i the only invariant eature or urace

5 relection component and hence invariant or highlight, and hence it depend only on enor repone and urace albedo. For the cae o the hue angle: 2 ( + ( ( [( + ( ] θ = co (26 For the cae o aturation, it i variance to the urace relection component, and hence it can not be ued or developing any robut algorithm to highlight. min( Rw, Gw, w S = 1 + emb ( k R + k R G where (27 = em k + em v c (28 w b R = em k + em v c (29 w b G = em k + em v c (0 w b Table 1 how a comparion among dierent colour model and their abilitie to be invariant to dierent imaging condition. In thi table, the letter Y denote an invariant colour model to the imaging condition and N denote enitivity o the colour model to that imaging condition. VII. COLOUR SEGMENTATION ALGORITHM The Swedih National Road Adminitration deined the colour ued or the ign [21] in CMYK colour pace. Thee value are converted into Normalized Hue and Normalized Saturation a hown in Table 2. The colour egmentation algorithm i carried out by taking RG image uing a digital camera mounted on a moving car. The image are converted to HSV colour pace. The hue, aturation, and value are normalized into [0,255]. The HSV colour pace i choen becaue the Hue eature i invariant to hadow and highlight. While normalized Hue i ued a a priori knowledge to the algorithm, normalized Saturation and Value are ued to peciy and avoid the achromatic ubpace in HSV colour pace. When the hue value o colour o the pixel in the input image i with the peciied colour range peciied in Table 2, and it hue value i not in the achromatic area, then the correponding value in the output image i et to white. The output image i then divided into a number o 16x16 pixel ub-image and ued to calculate the eed or the region growing algorithm. A eed in initiated i the number o white pixel in the output image i above a certain threhold level. Region growing algorithm i then applied to ind all the object in the output image which are big enough to initiate at leat one eed. Noie and other mall object are rejected becaue o the region growing algorithm. Thi ha an advantage that no more iltering i needed to delete thee objected and the remaining object are only the one which can be ued or recognition. The colour egmentation algorithm i given below: Convert the RG image into HSV colour pace. Normalize the grey level o every pixel in the H image rom [0,60] to [0,255]. Normalize the grey level o every pixel in the S image rom [0,1] to [0,255]. Normalize the grey level o every pixel in the I image rom [0,1] to [0,255]. For all pixel in the H image o I (H_pixel_value >240 AND H_pixel_value <= 255 OR (H_pixel_value >= 0 AND H_pixel_value < 10 Then H_pixel_value := 255 For all pixel in the H image o I correponding S_ pixel_value < 40 Then H_pixel_value := 0 For all pixel in the H image o I correponding ( V_ pixel_value < 0 OR ( V_ pixel_value > 20 Then H_pixel_value := 0 Divide the H image into 16x16 pixel ubimage. For every ubimage o Calculate number o white pixel o I number o white pixel >= 60 Then put a white pixel in the correponding poition in the eed image. Ue eed image and H image, apply region growing algorithm to ind proper region with ign. VIII. RESULTS AND CONCLUSIONS One o the mot diicult job in computer viion i to deal with outdoor image. The main reaon behind thi i the uncontrollable environment including dierent illumination level and dierent illumination geometry. In thi paper a new algorithm or colour egmentation or traic ign recognition i preented. The method i baed on invoking the HSV colour pace. It i proved that hue and aturation are invariant to the eect o illumination variation and hence they can be ued to develop hadow-invariant algorithm or colour egmentation. It i alo proved that hue i invariant to the eect o highlight and can be ued to develop highlight-invariant colour egmentation algorithm. The algorithm decribed in thi paper invoke merely hue or colour egmentation, but aturation and value eature o the HSV colour pace are ued to deine the chromatic ubpace in which hue can be ued. The method i teted on more hundred o image under dierent light condition (unny, cloudy, oggy, and now condition and dierent background. They etablihed very good robutne.

6 Segmentation i ucceully achieved in more than 95% o the cae under invetigation. It ail when the pixel under conideration i not in the chromatic ubpace o the HSV colour pace. In igure 2, there are ome example. The original image are hown in column a, and the reult o egmentation uing one hadow-highlight variant algorithm are hown in column b, while column c how the reult o hadowhighlight invariant algorithm. The algorithm i able to delete all the noie and mall object which can not be recognized a traic ign. Thi reduce the chance or ale alarm and make the detection better. Combining thee reult with hape recognition o the road ign and pictogram recognition, which are part o the uture work, will give a good mean to build a complete ytem which provide the driver with the inormation about the ign in real time a part o the intelligent vehicle. TALE I. NORMALIZED HUE AND SATURATION. Colour Normalized Hue [0,255] Normalized Saturation [0,255] Red Yellow 7 20 Green Light lue Dark lue ACKNOWLEDGMENT The author want to expre hi incere thank to Proeor Mark Dougherty at Dalarna Univerity or the ruitul dicuion and or hi time which i given to me reely. [1] H. Fleyeh, "Color detection and egmentation or road and traic ign" preented at 2004 IEEE Con. on Cyberneti and Intelligent Sytem Singapore, [2] H. Fleyeh, "Road and Traic Sign Color Detection and Segmentation - A Fuzzy Approach," preented at Machine Viion Application (MVA2005 IAPR, Tukuba-Japan, [] H. Fleyeh, "Traic Sign Color Detection and Segmentation in Poor Light Condition" preented at Machine Viion Application (MVA2005 IAPR, Tukuba-Japan, [4] W. Shadeed, D. Abu-Al-Nadi, and M. Mimar, "Road traic ign detection in color image" preented at 10th IEEE Inter. Con. on Electronic Circuit and Sytem (ICECS 200, Sharjah, United Arab Emirate 200. [5] M. énallal and J. Meunier, "Real-time color egmentation o road ign" preented at Canadian Con. on Electrical and Computer Engineering (IEEE CCECE, Montréal, Canada, 200. [6] A. de la Ecalera, J. Armingol, and M. Mata, "Traic ign recognition and analyi or intelligent vehicle" Image and Viion Comput., vol. 21, pp , 200. [7] C. Fang, S. Chen, and C. Fuh, "Road-ign detection and tracking," IEEE Tran. on Vehicular Technology, vol. 52, pp , 200. [8] G. Jiang and T. Choi, "Robut detection o landmark in color image baed on uzzy et theory," preented at Fourth Inter. Con. on Signal Proceing, eijing, China, [9] N. Hooe, Computer Image Proceing in Traic Engineering. New York: John Wiley & on Inc., [10] S. Vitabile and F. Sorbello, "Pictogram road ign detection and undertanding in outdoor cene" preented at Con. Enhanced and Synthetic Viion, Orlando, Florida, [11] P. Parodi and G. Piccioli, "A eature-baed recognition cheme or traic cene" preented at Intelligent Vehicle '95 Sympoium, Detroit, USA, [12] J. Plane, Traic Engineering Handbook: Prentice-Hall, [1] M. Lalonde and Y. Li, "Road ign recognition. Technical report, Center de recherche inormatique de Montrèal, Survey o the tate o Art or ub-project 2.4, CRIM/IIT," [14] S. Vitabile, A. Gentile, and F. Sorbello, "A neural network baed automatic road ign recognizer," preented at The 2002 Inter. Joint Con. on Neural Network Honolulu, HI, USA, [15] J. Miura, T. Kanda, and Y. Shirai, "An active viion ytem or real-time traic ign recognition," preented at 2000 IEEE Intelligent Tranportation Sytem Dearborn, MI, USA, [16] C. Fang, C. Fuh, S. Chen, and P. Yen, "A road ign recognition ytem baed on dynamic viual model," preented at The 200 IEEE Computer Society Con. Computer Viion and Pattern Recognition, Madion, Wiconin, 200. [17] S. Vitabile, G. Pollaccia, G. Pilato, and F. Sorbello, "Road ign Recognition uing a dynamic pixel aggregation technique in the HSV color pace," preented at 11th Inter. Con. Image Analyi and Proceing, Palermo, Italy, [18] S. uluwar and. Draper, "Color recognition in outdoor image" preented at Inter. Con. Computer viion, ombay, India, [19] S. uluwar and. Draper, "Non-parametric claiication o pixel under varying outdoor illumination," preented at ARPA Image Undertanding Workhop, [20] T. Gever and A. Smeulder "Color-baed object recognition," Pattern Recog., vol. 2, pp , [21] Swedih-Road-Adminitration, " TALE II. INVARIANCE OF COLOUR MODELS TO IMAGING CONDITIONS. Y DENOTES INVARIANCE AND N DENOTES SENSITIVITY OF COLOUR MODELS TO IMAGING CONDITIONS [20]. Colour eature Viewing direction Surace orientation Highlight Illumination direction Illumination intenity I N N N N N RG N N N N N Nrgb Y Y N N N H Y Y Y Y Y S Y Y N Y Y

7 (a (b (c Figure 2. Reult. (a Original Image (b Image egmented with hadow-highlight variant algorithm [1] (c Image egmented with hadow-highlight invariant algorithm.