АЛГОРИТЪМ ЗА РАЗПОЗНАВАНЕ НА ЦВЕТОВЕ, АДАПТИРАН ЗА ПЛАТФОРМИ С ОГРАНИЧЕНИ РЕСУРСИ ALGORITHM FOR COLOR RECOGNITION, ADAPTED FOR PLATFORMS WITH LIMITED RESOURCES R. Ivaov Techical Uiversity of Gabrovo, 5300 Gabrovo, 4 H.Dimitar Str. E-mail: rs-soft@ieee.org Abstract: New techologies represet a greet opportuity for the improvemet of the life ad idepedet livig of the disabled people. This paper presets the algorithm realizatio of color recogitio that ca be used for creatio of Java applicatios for mobile termials (J2ME) with built-i photo camera. The aim is creatig of mass accessible applicatio for people with impaired visio (blid ad color-blid people) so that they ca be iformed (by Text To Speech coversio) about the ame of recogized color i ceter of the video frame. Color filterig ad statistics are also possible. Color recogitio is based o combiatio of crisp color segmetatio ad several small Artificial Neural Networks (ANN). Such applicatios ca provide a sigificat help for disability compesatio to avoid the gap icreasig betwee disabled ad o disabled people. Key words: Color recogitio, Image processig, J2ME applicatios 1. INTRODUCTION Color recogitio is a major task i artificial visio systems (patter recogitio, avigatio of robots, weapos cotrol ad may others) [4]. I particular research, color recogitio is used as part of the Java applicatio for mobile termials, which aims to ease the lives of visually impaired people (recogizig the degree of illumiatio, detectio of objects i their color, idoor ad outdoor orietatio [1]). Existig applicatios i this area are few ad usually are based o hardware-software platforms with a high price. The most frequetly [13] about te basic colors are recogized. I the preseted research most of the color s ames are formed usig oe or more adjectives to color, such as: "light sky blue", "sow white", "tomato red", "pale violet", "royal blue", ad etc. The mai problem i the developmet of such applicatios is the eed to combie mutually exclusive requiremets, such as: High probability for correct recogitio that is idepedet to source of light ad its itesity. Iformig the user through text to speech coversio. Obtaiig result i soft real time. These features are hardly achievable whe the microcotroller or systems with limited computig resources ad capacity of operative memory are used. This makes a big part of the existig algorithms for color recogitio irrelevat i such platforms. 2. PLATFORM SELECTION It is proposed to use ay mobile termial with built-i camera ad Java Virtual Machie (JVM). The reasos for this are: 1. Over 65% of mobile termials maufactured i 2008 have built photo-camera ad JVM. 2. Guarateed platform idepedece of Java applicatios - MIDP versio 2.0 ad Mobile Media API (MMAPI) [10] i order to access photo camera is required. 3. Easily applicatio upgrade. 4. Usig photo-camera istead of color sesors allows the creatio of more complex applicatios, such as image ad face recogitio. The mai problems that must be resolved whe selectig mobile platform are: Optimize the speed of implemetatio, takig ito accout the still low productivity of most JVM for mobile termials. So far, few photo-cameras for mobile termials support white balace. This leads to false detectio of colors i various lightig coditios. Whe low-resolutio is used is ot possible program realizatio of the white balace algorithm that gives satisfactory results. Number of mobile termials that allow program cotrol of the flash, the focus, exposure ad white balace are still little. It is ecessary mobile termial to support the Advaced Multi-Media Supplemet Specificatio (AMMS) API [11], which is the extesio of MMAPI.
3. ALGORITHM REALISATION Algorithm for color recogitio is realized i the followig sequece: Color space selectio, Color segmetatio, Adaptive adjustmets ad ANN classificatio. 3.1. Color space selectio Color space (also called color model) defies the umerical represetatio of color eeded for program processig. There are may color models each of which has advatages ad disadvatages to be determied by the specific requiremets ad characteristics of the developed applicatio. The models, that are used most ofte i color recogitio, are [6]: RGB (Red, Gree, Blue). HSV (Hue, Saturatio, Value). TSL (Tit, Saturatio, Lightess). YCrCb (Luma, R chromiace, B chromiace). The ideal color model i color recogitio poit of view should simulate some characteristics of huma visual system, such as: o-liear sesitivity to visible light with differet wavelegth ad weak depedece o the itesity of light. The color iformatio, obtaied directly from the photo cameras, usually is i RGB format. The mai drawback of RGB color model is the high degree of correlatio of iformatio from color chaels. The RGB model does ot allows the separatio of color from itesity. HS based models (HSV, HSB, HSL) correspod better to how people experiece colors tha the RGB color space used: Hue (H) is a color attribute that describes what a pure color is, H values vary from 0 to 1 (from red through orage, yellow, gree, cya, blue, mageta ad back to red; Saturatio (S) values varies from 0 to 1 ad correspodig colors vary from usaturated to fully saturated. There are color models that attempt to simulate more accurately huma visio system, such as: CIELAB, CIEXYZ ad CIELUV [9]. However, experimets show that their use does ot improve the characteristics of color recogitio systems. It is proposed to use modificatio of HSV color model: Parameter I Parameter I represet the o-liear formatio of light s itesity, I = 0.299R + 0.587G + 0. 114B, (1) R = R 256, G = G 256 и B = B 256. (2) Parameter s values are i iterval [0, 1]. Whe I=0 color is 100% black, ad whe I=1 100% white. Parameter H This parameter describes the domiat color. The scale, which describes all possible base colors, is defied: the values i the iterval [0, 2) describe all red colors, iterval [2, 4) describes gree colors ad iterval [4, 6) describes blue colors: H = ( G B ) ( max mi), if ( max = R ) 2 + ( B R ) ( max mi), if ( max = G ) 4 + ( R G ) ( max mi), if ( max = B ) max = MAX ( R, G, B ), MIN ( R, G, B ) (3) mi =. (4) Normalizatio of the scale is eeded, to fit H parameter values i the iterval [0, 1]: H = H 6, (5а) H = H +1, if H < 0. (5b) Parameter S This parameter describes the purity of the color: 0, if ( max = mi) ( ) ( ) ( ) max mi max+ mi, if max+ mi < 1 ( max mi) ( 2 max mi), otherwise. S =, (6) Whe S=0 color is 100% gray. I 100% pure color value of S is 1. Proposed HSI model is characterized by much more elemetary mathematical apparatus eeded to obtai the values of H ad S parameters i compariso with the color model HSV. 3.2. Color segmetatio Color segmetatio is used for coarse estimatio of pixel s color. It is implemeted o the basis of the values of parameters H, S, ad I. The segmetatio is realized usig IF-THEN rules. O Figure 1 show the color map (HS plot) ad thresholds for red, orage, yellow, gree, cya, blue, mageta ad gray colors (H r, H o, H y, H g, H c, H b, H m ad S g ). Fig. 1. Color map (HS plot)
Thresholds are obtaied takig ito accout the sesitivity of the huma eyes to visible light [8] ad experimets. Ulike the stadard algorithm for fuzzy clusterig [3], the proposed approach allows realizatio i real time. Before color segmetatio, the color oise i frames is reduced by 3x3 slidig filterig [7]. The value of cetral pixel is replaced with mea value of color of pixels i widow, c ˆN 1 / 2 = mea( c0, c1,..., cн 1), (7) where N=9. Color segmetatio starts with checkig whether the color is white or black. If I>I w, the color is white, ad if I<I b - color is black. If S<S g color is gray, otherwise color segmetatio, based o the values of H, is realized. Te color classes are recogized: red, orage, yellow, gree, cya, blue, mageta, white, gray ad black. 3.3. Adaptive correctios Whe light itesity is 25% lower or higher of average itesity, obtaied durig the system traiig a adjustmet of values of the parameters R, G, ad B is realized. To do this i traiig stage average itesity values for each color class Ic are obtaied, c {"red", "orage", mea "yellow", "gree", "cya", "blue", "mageta", "gray"}. This correctio adjusts itesity of the curret pixel to the average itesity of the class to which he has belog ad improves the results geerated by ANN classifiers: R = I R, G = I G, B = I B, (8а) 3.4. Color classificatio I = Icmea I, I = R + G + B. (8b) ANN classifiers geerate ame of color for every iput test feature vector. Nie ANN of type 3-layer perceptro are used as show i Fig. 2. R,G,B S I H Itesity Correctio Black & White Classificatio Color Segmetatio ANN Classificatio Fig. 2. Color classifier architecture black white color ANN select Color ame ANN iput vectors are formed by the values of parameters H, S ad I. Traiig is implemeted through maual labelig of pixels from video frames, obtaied from the mobile termials [2]. The umber of output euros coicides with the umber of colors for each class. To reduce the errors because of o-fuzzy boudaries betwee the color segmets, ANN traied with vectors from eighborig segmets. For this purpose, each segmet is divided ito 3 sub-segmets, for example for red color sub-segmets are: red-mageta, red ad red-orage. Red color ANN is traied with traiig vectors from left ad right sub-segmets: mageta-red ad orage-red (see Table 4). O traiig phase (EBP algorithm is used) all ANN weight vectors for each color class except "black" are obtaied. ANN parameters are saved i ASCII resource files ad loaded i operative memory whe ecessary. This decisio has the followig advatages: 1. Usig multiple ANN, istead oe ANN for all colors [5], allows the result to be obtaied i real time ad the memory to be used more efficietly. 2. Due to the small umber of euros i iput ad output layer traiig is realized much more effectively with a high probability for obtaiig a global miimum. The color recogitio process rus i the followig sequece: 1. Color model parameters (H, S, I) are obtaied. 2. The block "Black & White Classificatio" geerates 3 output sigals: black if black color is recogized (go to 6), white if pixels belogs to white color class (go to 4) ad color if color segmetatio is eeded (go to 3). 3. Coarse color segmetatio is realized by the block "Color Segmetatio". The color of pixels at ceter of frame (3x3 regio) is assiged to oe of all 9 possible color classes. 4. Check pixels itesity (block "Itesity correctio"). If itesity adjustmet is ecessary equatios (8) are used. 5. Fie color classificatio by ANN. Depedig o the results of steps 2 ad 3, a ANN for recogized color class is selected. 6. Ed. The umber of colors, that are recogized for each class, is differet ad is described i Table 1. Table 1. Number of recogized colors for each class No Class ame Number of colors 1. red 13 2. orage 15 3. yellow 11 4. gree 14 5. cya 18 6. blue 14 7. mageta 13 8. white 9 9. gray 5 10. black 1 Total: 113
Other threshold levels used have the followig values: 4. EXPERIMENTAL RESULTS Proposed algorithm for color recogitio ca be used i mobile platforms with limited resources. All experimets are realized with mobile termials Nokia 6630 ad Nokia N95. I Table 2 the performace of Nokia 6630 ad N95 JVM are preseted (SPMarkJava06 is used [12]). Table 2. Nokia 6630 ad Nokia N95 JVM bechmarks Operatio Add, Subtract Multiply Divide Performace, KOps/s Operad type Nokia 6630 71,470 1,847 55,601 2,227 1,155 957 iteger iteger iteger Nokia N95 59,931 2,679 41,888 3,417 1,629 1,423 To improve color recogitio time oly iteger operatios are used. Whe Nokia 6630 mobile termial is used the mea time itervals required for realizatio of color recogitio are described i Table 3. Table 3. Color recogitio algorithm timig Stage 1. Get sapshot 2. Mea 3x3 filterig (160x120) 3. Color segmetatio 4. ANN classificatio Total time: Time, ms 244 14 1 7 Sg=0.0116, Iw=0.95, Ib=0.05. (9) Oe hudred test frames are used: 50 frames obtaied i a room (exteral lightig 20 frames, fluorescet light 10 frames, light of the icadescet lamp 10 frames ad 10 frames from TV set) ad 50 outdoors frames (25 i sushie ad 25 i cloudy weather). The probability of correct color classificatio whe usig a mobile termial Nokia 6630 is 91% as show i Table 5. I this case the white balace ad auto focus is ot used. Table 5. Color recogitio algorithm performace Test mode Recogitio rate, % 1. White balace, auto focus ad flash cotrol are ot supported (Nokia 6630) 2. White balace, auto focus ad flash cotrol are supported (Nokia N95) 91 96 The errors are because of white balace missig ad light itesity i part of frames is too low or high. Five percet of errors are elimiated whe usig a mobile termial Nokia N95 that supports AMMS API - white balace ad auto focus are eabled. The results of testig of the applicatio are show i Figures 3 to 6. The user iterface is show i Figure 3. 266 Table 4 describes the H-itervals ad thresholds used i the formatio of the traiig vectors for each ANN. Table 4. H itervals ad thresholds ANN class Red Hr=0.0418 Orage Ho=0.1255 Yellow Hy=0.2497 Gree Hg=0.4170 Cya Hc=0.5830 Blue Hb=0.7503 Mageta Hm=0.9164 Traiig sub-segmets mageta-red, red-mageta, red, red-orage, orage-red red-orage, orage-red, orage, orageyellow, yellow-orage orage-yellow, yellow-orage, yellow, yellowgree, gree-yellow yellow-gree, gree-yellow, gree, greecya, cya-gree gree-cya, cya-gree, cya, cya-blue, blue-cya cya-blue, blue-cya, blue, blue-mageta, mageta-blue blue-mageta, mageta-blue, mageta, mageta-red, red-mageta Hue iterval 0.0-0.06275 0.87451-1.0 Fig.3. Experimetal results user iterface 0.02092 0.14611 0.10458 0.2915 0.20784 0.45882 Fig.4. Experimetal results test mode (get color, fid color, fid colors) 0.37516 0.62484 0.54118 0.79216 0.7085 0.95817 Fig.5. Experimetal results color recogitio
6. REFERENCES 1. Buluswar, S.D., B.A. Draper, Color Recogitio i Outdoor Images, 6th It. Cof. o Computer Visio, pp.171-177, 1998. 2. Ivaov, R., A applicatio for Image Database Developmet for Mobile Face Detectio ad Recogitio Systems, It. Scietific Cof. Computer Sciece 2008, Part I, pp.264-269, Greece. Fig.6. Experimetal results fid color(s) The results, obtaied i testig mode (PNG resource file is used), are show i Figure 4. I Figures 5 ad 6 are show results obtaied i Color recogitio ad Fid color(s) modes. 5. CONCLUSION A algorithm for color recogitio, which combies the advatages of fast crisp color segmetatio ad artificial eural etworks is preseted. Algorithm ca be used for platforms with limited memory ad computig power, icludig mobile termials with built-i JVM. Cotiuous time usig the applicatio depeds maily o the cosumptio of a photo-camera. To icrease this time, the camera turs off if the user does t iteract with applicatio for more tha 15 secods or if applicatio loses focus. The proposed algorithm is part of the Java applicatio for color ad huma face recogitio. It ca be istalled o ay mobile termial that supports MMAPI (AMMS API is optioal) ad has built photo-camera. Applicatio is adapted for visually impaired people (partial or complete loss of visio ad color-blid). 3. Khodja, L., et al., Fuzzy Clusterig for Color Recogitio Applicatio to Image Uderstadig, 5th IEEE It. Cof. o Fuzzy Systems, Vol.II, pp.1407-1413, 1996. 4. Stachowicz, M.S., D. Lemke, Color Recogitio, 22d It. Cof. o Iformatio Techology Iterfaces, pp. 329-334, 2000. 5. Stoksik, M., et al., A Neural Net Based Color Recogitio System, 2d It. Cof. o Artificial Neural Networks, pp.86-90, 1991. 6. Plataiotis, K.N., A.N. Veetsaoupoulos, Color image processig ad applicatios, Spriger, Germay, 2000. 7. Pratt, W.K., Digital Image Processig, 3rd editio, NY, Wiley, 2001. 8. Trussel, J., E.Saber, ad M.Vrhel, Color Image Processig, IEEE Sigal Procesiig Magazie, Vol.22, No.1, pp.14-22, 2005. 9. Wyszecki, G., W.S. Stiles, Color Sciece: Cocepts ad Methods, Quatitative Data ad Formulae, Joh Wiley ad Sos, 1982. 10. Mobile Media API (JSR-135) specificatio, Available at http://jcp.org/e/jsr/detail?id=135. 11. Advaced Multimedia Supplemet Specificatio (JSR-234) specificatio, Available at http://jcp.org/e/jsr/detail?id=234. 12. http://www.futuremark.com 13. http://seeigwithsoud.com/midlet.htm