An Image-Based Food Classification System

Transcription

1 S. Somatlake, A. N. Chalmer, An Image-Baed Food Clafcaton Sytem, Proceedng of Image and Von Computng New Zealand 2007, pp , Hamlton, New Zealand, December An Image-Baed Food Clafcaton Sytem S. Somatlake and A.N. Chalmer ECE Department, Manukau Inttute of Technology, Prvate Bag 94006, Manukau 2240, Auckland, New Zealand. Emal: Abtract A food clafcaton ytem ntroduced to grade x type of freh produce ung average CIE L*a*b* and Bayean nference technque. Regon-baed egmentaton remove gloy regon on the food urface and olate the food ample from t background. Reference food ample mage provde colour dtrbuton probablte at each rpenng tage, enablng food rpene to be clafed. The two quanttatve technque have been verfed by mean of a ubjectve aement procedure. Both technque appeared to perform contently, wth varyng level of agreement wth the ubjectve aement wth the Bayean nference technque gvng the more atfactory general performance. Overall, dgtal colour magng can be a veratle, relable and a low-cot tool for colour-baed clafcaton of freh produce, wth the potental to replace other, more cotly technque. Keyword: Food clafcaton, Dgtal colour magng, CIELAB, Bayean nference 1 Introducton The colour of food one of the frt parameter of qualty evaluated by conumer [1]. Common colour meaurng devce, uch a pectrophotometer, are expenve and have major lmtaton n term of ther ablty to meaure food colour wth uffcent peed and accuracy for the food producton envronment. Manual food qualty aement and clafcaton technque requre human npector to be traned for colour evaluaton whch ncreae producton cot and ntroduce the potental for human error. However, t ha been demontrated that dgtal colour magng provde a feable alternatve for food colour meaurement, qualty npecton and clafcaton (e.g. ee ref. [2]) and can yeld gnfcant avng n term of labour cot together wth an ncreae n product qualty. Th project wa a laboratory nvetgaton of machne von to determne t effectvene a a tool for foodqualty quantzaton. It recognzed that an ndutral ntallaton would requre the ue of utable ruggedzed component and aemble. However, for preent purpoe (prmarly proof-of-concept) the ue of a conumer dgtal colour camera together wth utable mage proceng oftware provde a coteffectve oluton. 2 Food Colour Meaurement The ntenton wa to etablh dgtal colour magng, ung a conumer-type dgtal colour camera, a a relable and effectve technque for food colour meaurement. Th ecton decrbe the applcaton of th method for the colour meaurement and clafcaton of x type of freh produce. 2.1 Image Acquton A hgh-reoluton (5-megapxel) Sony dgtal colour camera (Model DSCV3) wa utlzed. The captured mage were obtaned a uncompreed TIFF mage fle. The food ample were llumnated by four Phlp 150W Photocrecenta photographc lamp havng a colour temperature of approxmately 2900K. The lght ource were potoned to llumnate the ample at an angle of 45, at a dtance of approxmately 450mm from the ample, wth two lamp on each de, to acheve unform llumnaton of the ample. 2.2 Food Sample The hypothe that the external colour appearance of the food ample a crucal ndcator of ther qualty. The experment wa therefore degned not only to capture the appearance of the ample a accurately a poble, but alo to track the change n ther appearance a they were allowed to age naturally over a perod of nne day. The ample repreent x type of freh produce, elected from the market at ther earlet tage of rpenng pecfcally an apple, banana, broccol, green pepper, pear and tomato. In order to be able to work wth ample of manageable phycal ze, the broccol and green pepper food ample were exced from the orgnal vegetable. 260

2 Fgure 1: RGB mage of (a) Apple, (b) Banana, (c) Broccol, (d) Pear, (e) Green pepper, and (f) Tomato, at four rpene level. Number down the rght-hand de are the equence number a ued n Table 1 (Secton 6.1). The food ample were tored at room temperature (17 C 22 C) and mage were acqured every three day over a perod of nne day to how dfferent level of rpene. See Fgure Camera Calbraton Calbraton of the dgtal camera an eental tage n the procedure for colour meaurement. The frt tep wa to determne the camera gamma. Th wa done by photographng a ere of calbrated grey-cale patche under the exact lghtng condton to be ued to llumnate the food ample, and relatng the grey-cale value n the mage to the calbrated reflectance of the patche. The gamma wa found to be Next, the colour calbraton of the camera wa verfed by photographng ten Munell colour ample card that had prevouly been meaured ung pectrophotometry technque. Comparon of the reult for the colour meaured by thee two method ndcated a hgh level of agreement. The fnal tep wa to et the whte balance of the camera. Th wa done by manual adjutment of the whte balance whle photographng an A4 heet of whte paper (a an approxmate tandard of whtene) under the lghtng condton decrbed above. 3 Colour Analy It ha to be borne n mnd that food colour wll ultmately be judged by the human eye o that any ntrumental technque mut how good correlaton wth human repone. For th reaon t wa decded that the colour analy hould be baed on the CIE L*a*b* colour model (CIELAB) whch a threedmenonal colour pace provdng a good approxmaton to the human percepton of colour appearance [3] rather than the mpler (but le meanngful) approach of workng wth mage RGB value. Th ecton outlne the dervaton of the colormetrc reult for each pxel of the colour mage. 3.1 Convertng RGB to CIE L*a*b* In order to obtan meanngful colormetrc data from the colour mage, t ha to be borne n mnd that the camera normal mage detnaton a colour montor conformng to the RGB tandard [4]. The converon from mage RGB data to CIELAB value requre three major tep: In the frt tep, the raw RGB value need to be gamma-lnearzed, ung an nvere-gamma value of (0.64) -1 = The econd tep nvolve calculatng the CIE trtmulu value X,Y,Z from the lnear RGB value by aumng RGB condton, vz. ITU-R (BT 709) prmare together wth a D 65 whte pont, ung equaton (1). X = R G B Y = R G B (1) Z = R G B Th fnally enable one to calculate the CIELAB coordnate ung equaton (2 4), where X 0,Y 0,Z 0 261

3 are the whte pont trtmulu value obtaned by ettng R 0 = G 0 = B 0 = /3 Y L* = (2) Y0 X Y a* = 500 X0 Y0 1/3 1/3 Y Z b* = 200 Y0 Z0 1/3 1/3 (3) (4) Note that the chromatc coordnate, a* and b* can be combned to gve a meaure of CIELAB chroma a hown n equaton (5). 2 2 [( a*) ( *) ] 1/ 2 C * = + b 3.2 CIELAB Analy (5) The analy of CIE L*a*b* colour value demontrate the utlty of dgtal colour magng technque for the ue n food qualty analy. Fgure 2 llutrate a typcal analy where b*/a* plot repreent the colour dtrbuton of the food ample, the pear n th cae, a t rpen over tme. Here each pont n the b*/a* plot repreent one pxel from the accompanyng pear mage, and one able to gan a feel for the dtrbuton of colour over the urface of the pear. The b*/a* profle ndcate that the pxel colour value for the pear are clutered n the approxmate range -20 < a* < 0 and 15 < b* < 70, a hown by Fgure 2 (a c). However, by day 9, t evdent that the colour of the pear ha changed gnfcantly, a can be een n Fgure 2 (d) whch how movement toward the hgher value of a*. Thu, the b*/a* plot can provde valuable nformaton of how a partcular food change over tme. The more olated (apparently random) outler of the colour profle n all four cae are pobly due to textural rregularte on the food urface. The L*/C* plot of the food ample, uch a that of the pear, can alo gve an ndcaton of how lghtne L* and chroma C* vary a the pear age, a hown n Fgure 3. Agan, t notceable how there a defnte change n the value for day 9, where L* and C* both how a udden growth n pxel number toward the orgn. 4 Food Clafcaton Sytem A food clafcaton ytem wa mplemented and developed a a GUI (uer nterface) n MATLAB to perform qualty aement and clafcaton of the x food type elected for th experment. The current ytem capable of loadng the food mage acqured by the conumer-type dgtal colour camera, performng regon-baed egmentaton to olate the food ample for analy, and performng food clafcaton ung two technque. 4.1 Implementaton The two key tage of the mplementaton are mage egmentaton and pxel clafcaton. Segmentaton remove the pecular patche that are n mage acquton and alo olate the food ample from t background to mplfy the analy of the food appearance. The clafcaton can then be carred out ung two technque: average L*a*b*, and Bayean nference. Fgure 2: Colour profle (b* v. a*) of the pxel n the pear mage. 262

4 Fgure 3: Colour profle (L* v. C*) of the Pear Regon-baed Segmentaton Snce the acqured mage contaned both the food ample and background, a technque capable of removng the background from the mage a prerequte for the colour analy procedure that follow. A regon-baed egmentaton technque wa mplemented and appled to mage to olate the food ample from t background [5]. The algorthm egment the colour n an automated fahon by applyng a mak on the ample mage, a llutrated n Fgure 4 for the banana mage. The mak determned from two mall egment at elected locaton on the ample mage. The frt egment (A) correpond to a locaton on the background of the mage (th mplemented by default n the ytem); and the econd egment locaton (B) elected on a porton of the food urface by the uer. The two egment are compared to the ret of the mage, baed on ther average colour dfference n a*,b* pace. Th enable the mage to be egmented nto two regon the ample and background. The created mak appled to the orgnal mage, whch dtnguhe the food ample from t background. The background now blacked out and the food ample olated n the central part of the mage for further analy. It can be een n Fgure 4 that the technque alo replace mall urface blemhe by black pxel Average L*a*b* Technque The average L*a*b* technque ue an ntal approxmaton for clafcaton. Th technque utlze the average L*,a*,b* value of a ere of reference mage of a partcular food at dfferent rpenng tage, and compare them wth the average L*,a*,b* value of the ample mage. The average L*,a*,b* value of the ample mage are tored n a vector, Lab, a hown n equaton (6), and mlarly the average L*,a*,b* value of reference mage are tored n the vector, Lab, hown n equaton (7). [ L a b ] Lab = (6) [ L a b ] Lab = (7) The vector can now be compared to ee whch reference mage mot mlar to the ample mage. The mplet analy would be to compare ndvdual component of the vector. For example, the reference mage wth the mot mlar average L* to the ample mage can be evaluated by fndng the mnmum abolute dfference; then equaton (8) dentfe the mot mlar mage, j L. Smlar analy can be done wth the other two component, a* and b*. j L { L L } = arg mn (8) A method of comparng the colour, conderng all three component multaneouly to evaluate the length of the vector, Lab Lab. The dfference vector wth the hortet length gve the reference mage, j, whch the mage wth the cloet match n average CIELAB coordnate, a hown n equaton (9). { Lab Lab } j = arg mn (9) Bayean Inference Technque Typcally, food urface contan a range of colour rather than a ngle colour. Therefore a technque capable of utlzng a colour data et contanng a number of colour from one food mage, can be ued to fnd mlar colour data n another food mage, and provde ueful nformaton for clafcaton [6]. The Bayean nference technque mprove on the ue of average by capturng more detaled nformaton about the colour, and ther dtrbuton, n the mage. It calculate proporton of varou range of colour by dvdng the colour pace of the reference ample mage nto everal regon. By calculatng the reference regon probablte, the proce enable one to evaluate whch of the reference food mage the mot lkely match to the ample food mage. Th technque ha been mplemented n a Bayean algorthm that ha been developed n Matlab. The tartng pont to compute the a*,b* coordnate for each pxel n the elected mage. Next, thee colour value are automatcally aeed by dvdng the a*,b* plane nto nne regon (n a 3 3 grd) and countng the number of pxel repreented n each regon. 263

5 (A) (B) Fgure 4: Regon-baed Segmentaton. Fgure 5: Graphcal Uer Interface for Subjectve Aement. By applyng th treatment to the reference and ample mage, the Bayean etmate of probablty of mage mlarty computed. The mage par yeldng the hghet probablty of mlarty judged to be the bet match, and o forth. 4.2 The Matlab GUI The GUI algorthm allow the uer to pecfy the reference mage, compute the reult, and export the reult to a data fle. The reult are wrtten to AVE and IDF data fle for the average L*a*b* and Bayean nference technque repectvely. Ung th data, a ample mage can now be clafed accordng to t degree of mlarty to the gven reference mage, by applcaton of both the above technque. 5 Subjectve Aement In order to verfy the mplemented clafcaton ytem, a ubjectve aement wa conducted. A panel of 20 oberver were each hown a ere of computer-dplayed colour mage of dfferent food ample, a llutrated n Fgure 5. In each tage of the aement, four reference mage of the ame type of food, rangng from unrpe to rpe (to over-rpe, n ome cae) were dplayed. The partcpant were ntructed to match the food ample to one of the four reference mage by conderng the colour and the degree of rpene. The reult from the vual aement are ncluded under the Survey column n Table 1. It appeared that there wa a mall but notceable non-unformty of the expoure n the food mage (a can be een from a cloe npecton of Fgure 1) and th may pobly have led to ome obervatonal dffculte. 264

6 6 Reult 6.1 Comparon of Method Table 1 how the rankng of the mage n the two mage-analy method, a compared wth the order of preference determned n the ubjectve experment ( urvey ). The number n the table correpond to the mage number n each mage equence, a ndcated on the rght-hand de of Fgure 1 (e.g for the banana mage n the urvey ndcate that the oberver judged banana mage 2, taken on Day 3, a the mot preferred ; followed by number 1 and 3, and fnally mage 4, Day 9, a the leat preferred ;... etc.). Table 1: Comparon of mage rank order n the dfferent technque ued. Proceng peed can be mproved by optmzaton, whch partcularly neceary n the cae of the RGB to L*a*b* converon. For real-tme applcaton, t would probably be preferable to programme the algorthm nto a dedcated hardware devce. The technque of computng and dplayng the dtrbuton of pxel value, a llutrated n Fgure 2 and 3, thought to hold conderable prome for a number of ndutral colour-analy applcaton. In concluon, we beleve that further development of th ytem would provde a powerful, yet low-cot, colour gradng and analy tool for a range of ndutre ncludng, but not lmted to, the food ndutry. 8 Acknowledgement The author acknowledge the upport of th reearch by the ECE Department at Manukau Inttute of Technology. It een that, for fve out of the x food ample type the Bayean method agree wth the ubjectve method; wherea the average L*a*b* method agreed n only three out of x cae [7]. 6.2 Proceng Tme Table 2 ummarze the executon tme for the varou procee decrbed. Thee tme were meaured for procee runnng n Matlab veron 7.3 runnng under Wndow XP on an AMD Turon 64 ML32 (1.8GHz) proceor wth 512 MB of RAM. Table 2: Typcal proceng tme per mage. Proce Tme () RGB to L*a*b* converon 30 Regon-baed Segmentaton 10 Bayean nference clafcaton 20 Average L*a*b* clafcaton 8 7 Dcuon The ytem decrbed here a developmental prototype and, a uch, demontrate atfactory level of accuracy and performance. The dcrepance n the ubjectve evaluaton can be overcome by mprovng the unformty of expoure for the expermental mage, and by ncreang the number of oberver to obtan more robut ubjectve data. Reference [1] J.B. Hutchng, Chapter 1 n Food Color and Appearance, 2 nd ed. Gatherburg, Maryland, Apen Publcaton, 1999, pp [2] K.L. Yam & S.E. Papadak, A mple dgtal magng method for meaurng and analyng colour of food urface n Journal of Food Engneerng, vol. 61, pp , [3] M.D. Farchld, Color Appearance Model, Readng, Maachuett, Addon-Weley, 1998, pp [4] M. Stoke, M. Anderon, S. Chandraekar & R. Motta, A tandard default color pace for the Internet RGB, Veron 1.10 (1996). Downloadable at Graphc/Color/RGB.html. [5] MATLAB Claroom Edton 7.1. Color- Baed Segmentaton ung L*a*b* Color Space, Image Proceng ToolBox, The MathWork Inc., [6] E.W. Weten, Baye Theorem, MathWorld A Wolfram Web Reource, CRC Pre, Downloadable at BayeTheorem.html. [7] S. Somatlake, Food Colour Meaurement, BE(Hon) Engneerng Reearch Project Report, Manukau Inttute of Technology,