S. Battato et al.: Analyss and Characterzaton of JPEG 2 Standard for Imagng Devces Analyss and Characterzaton of JPEG 2 Standard for Imagng Devces S. Battato 1, A. R. Bruna 2, A. Buem 2 and A. Castorna 2 773 Abstract Ths paper presents a seres of possble mprovement strateges and/or mplementatons related to the JPEG 2 standard for magng devces. A comparatve study relatve to varous codng optons s presented together wth some bref consderatons about rate-ctrl technques and perceptve mprovement strateges. Index Terms Classfcaton, compresson standard, contentdependent optmzaton, EBCOT, HVS, Jpeg2. I. INTRODUCTION The wde dffuson of dgtal stll and vdeo mage capturng devces ponted out new challenges manly focused n qualty mprovement and reducton of the processng tme [6]. Increasng of dgtal pctures qualty s acheved by reducng nose ntroduced by the sensor durng the acquston of the data [2], whle ncreasng the resoluton of the sensor allows mprovng compresson performances for a more effcent manpulaton and storage of mages [6], [8]. The hgh rate of nnovaton n the mage processng technology nvolves changng n the product features. The new Jpeg2 compresson standard [1], [14], [21] s promsng a lot as far as compresson effcency and features to be supported, at the cost of a hgher complexty wth respect to Jpeg. It has been created to nclude all functonaltes needed n a compresson system. These addtonal functons make t sutable for a wde range of applcatons and markets: Internet, moble communcaton, dgtal photography, dgtal lbrary, prntng, scannng and so on. The JPEG2 standard allows to manage mages wth functons that were not present n the other formats and ths suggest a wde usage of ths format n moble magng applcatons. JPEG2 has a better effcency than JPEG for hgh compresson rates and currently perform, on more mages, better results than any other standard. Moreover usng a progressve decodng, the mage vsualzaton wll be refned at each step adaptng tself to the partcular resoluton dsplay used (.e resoluton and/or qualty scalablty). the other hand Moble Stll Image s focused on the achevement of the best trade off between qualty and low power n terms of processng, related hardware and bandwdth occupaton. A typcal moble devce for mage has a VGA (64 x 48 pxels) sensor that captures an mage of.3 mllon of pxels dong very few elaboratons. The man purpose of the paper s the detaled descrpton of ths new dfferent world both n term of complexty (hardware/software) and related capabltes wth respect to the JPEG 2 standard compresson. The rest of the paper s structured as follows. The next secton descrbes how magng devces acqure mages usng CCD/CMOS sensors. Secton III s devoted to the descrpton of an exhaustve study about the numerous codng optons of the new standard. Some consderatons about Jpeg2 rate-control technques are brefly dscussed n Secton IV. The next Secton shows a smart technque able to mprove the perceved qualty of mages heavly Jpeg2-compressed modulatng the relatve bt-rate of adjacent tles. A conclusons Secton closes the paper trackng drecton for future works and research. II. SENSORS AND DATA ACQUISITION A dgtal engne uses an electronc sensor to acqure the spatal varatons n lght ntensty and then use mage processng algorthms to reconstruct a color pcture from the data provded by the sensor (see Fgure 1). Up to now Dgtal Stll Camera (DSC) [6] market has been focused on qualty mprovement (obtaned by ncreasng the resoluton of the sensor and by usng more sophstcated mage processng algorthms) and n reducton of processng tme. On 1 Sebastano Battato works at Catana Unversty Dpartmento d Matematca ed Informatca E-mal {battato@dm.unct.t}; 2 Antono Buem, Arcangelo Raner Bruna, Alfo Castorna work for STMcrolectroncs AST Catana Lab E-mal: {antono.buem, arcangelo.bruna, alfo.castorna}@st.com. Contrbuted Paper Manuscrpt receved September 23, 23 98 363/ $1. 23 IEEE Fgure 1: Image acquston (DSC/Moble) prncple. Two technologes exst to manufacture magng sensors: CCD (Charge Coupled Devce) or CMOS (Complementary Metal Oxde Semconductor). The CMOS approach s more flexble because each pxel can be read ndvdually. CCDs use a specal manufacturng process to create the ablty to transport charge across the chp wthout dstorton. CMOS chps, on the other hand, use
774 completely normal manufacturng processes to create the chp: the same processes used to make most mcroprocessors. There are several notceable dfferences between CCD and CMOS sensors. CCD sensors create hgh-qualty, low-nose mages. CMOS sensors, tradtonally, are more susceptble to nose but are low-power: 1 tmes less than an equvalent CCD sensor. Based on these dfferences, CCDs tend to be used n cameras that focus on hgh-qualty mages wth lots of pxels and excellent lght senstvty. CMOS sensors usually have lower qualty, lower resoluton and lower senstvty. However, CMOS cameras are much less expensve and have great battery lfe. Usually each senstve element of the sensor (known as pxel) s senstve to one color component only. Ths pattern s known as Bayer pattern [7]. Pcture qualty s strctly related to the number of pxels composng the sensor: the hgher the better. But ncreasng the sensor resoluton wthout ncreasng ts sze reduces the area of ndvdual pxels, and therefore ther senstvty (see Fgure 2). IEEE Transactons on Consumer Electroncs, Vol. 49, No. 4, NOVEMBER 23 Recently we have consdered the possblty to compress mages comng from CCD/CMOS sensors drectly n Bayer format [3], [5] to reduce the amount of swtchng actvty nsde the bus-data. Classcal codng technques n ths case do not always offer satsfyng performances. In partcular, a nonstandard vector quantzaton -based [5] approach was compared wth JPEG and JPEG-LS standard algorthms. All these technques have hgh performance n terms of compresson qualty at hgh btrates wthout perceptual loss. Moreover, compresson does not affect further stages of IGP that take compressed data as nput n order to obtan a full color mage n a typcal dgtal camera archtecture. Performances were evaluated also n terms of computatonal complexty, a basc evaluaton crtera for bayer-orented compresson algorthms. The Jpeg2, as descrbed n the next secton, for ts relatve complexty, s not well suted for such knd of applcaton where a low-power consumpton soluton s needed. III. FEATURES AND COMPLEXITY ANALYSIS A. Codng Optons a) Fgure 2: A sensor color s obtaned by a mono sensor plus sutable RGB color flters. Each pxel/color flter has a small lens a) placed on top, mprovng sgnfcantly lght gatherng at pxel ste. On the rght b) t s showed the typcal layout of a sngle CMOS sensor. In wreless magng feld, the characterstcs of the sensor used for moble applcaton must take nto account prmarly dmenson and power consumpton. For a moble devce workng wth battery supply s essental to use less charge as possble and not to be bg or have an excessve weght. For all these reasons a CMOS sensor s preferred to a CCD sensor and many sensor manufacturers are orented n ths drecton. They support both stll and vdeo magng wth lower resoluton as VGA (64x48) or CIF (352x288) formats. Sensor must be very low power and n less than a cubc centmeter must comprse also optc, lens and mechancal housng. Another ssue for the system-level project regardng the sensor s that the preferred area to ntegrate the sensor n near the handset s antenna and so strong Electro-Magnetc Interference can be regstered on the sensor. For ths reason sensor packagng has a relevant role n project phase. Another very mportant pont n the wreless magng archtecture s the parttonng of the mage processng n the archtecture and the nterface between communcaton modules. For these reasons the sensor could be used only as nput devce demandng properly processng to a remote and dstrbuted envronment [19]. b) The Jpeg2 mage compresson standard has been desgned to address requrements of a large range of applcatons, so t provdes many dfferent features and functonaltes. Supportng dfferent functonaltes means, of course, realzng a more complcated desgn of the encoder system that manage these new capabltes. The nfluence of the man Jpeg2 codng parameters over the compresson qualty has been evaluated varyng the codng parameters and measurng the resultng PSNR. In partcular, the expermental results reported here has been obtaned over a database of 3 mages coded varyng the followng parameters: Bt rate (mnmum =.125 bpp, maxmum=3 bpp). Wavelet transforms. Number of transform decomposton levels (vared from 2 to 1). Tle sze (mnmum = 64x64 pxels, maxmum = 512x512). Overlapped and un-overlapped tlng. No-tlng encodng has also been consdered. All the Jpeg2 coded mages have been generated usng the Verfcaton Model software [15]. The best compresson results were obtaned wth (9,7) wavelet transform, overlapped tlng wth bg tle sze and 4 decomposton levels. B. Wavelet transforms comparson To evaluate how wavelet transforms affect the compresson qualty n Jpeg2, our database of testng mages has been encoded settng the number of decomposton levels to the default value (fve) wthout usng tlng parttons and properly varyng the output bt rate. Fnally the PSNR of the resultng
S. Battato et al.: Analyss and Characterzaton of JPEG 2 Standard for Imagng Devces 775 mages has been computed. (9,7) floatng pont wavelet applcaton yelds better compresson than the (5,3) nteger wavelet, n terms of PSNR. The gap between the wavelet transforms grows wth the ncreasng of the bt rate untl of 2 bpp, and then t begns to decrease. On the average, however, there s a PSNR dfference of about.3 db n the compresson qualty obtaned usng the two DWT dfferent codng optons, so there s no a perceptve dfference n the vsual analyss of two mages obtaned n ths way (see Fgure 3 for an example). Fgure 4: Input test mage for Jpeg/Jpeg2 comparson. C. Jpeg2 versus Jpeg Ths secton presents a comparson between Jpeg2 and Jpeg standard. To reduce the amount of data, Jpeg uses a RGB to YCbCr transform followed by subsamplng. Even f Jpeg2 could acheve the same effect usng the multresoluton nature of the wavelet transform [1], our analyss presents results obtaned over 4:2:2 Jpeg2 coded mages to make an effectve comparson wth Jpeg. Mult-components codng s enabled n Jpeg2 processng each component ndependently. As show the dagram n Fgure 5, whch presents the PSNR behavor of the mage at dfferent bt rates, n ths partcular example Jpeg2 outperforms Jpeg and presents also a reduced blockng effect at.25 bpp, but Jpeg yelds a better compresson at bt rates greater than 1 bpp. 45 4 35 3 25 2 Jpg J2k 15 1 5.25.5 1. 1.5 2. 2.5 3. bt rate (bpp) Fgure 5: PSNR comparson between Jpeg and Jpeg2 compresson for the pcture Fgure 4. Fgure 3: nput mage (top). Image compressed at.125 bpp usng W5X3 (mddle) and W9X7 (bottom) wth no tlng, and 5 decomposton levels. However such weakly dfferent results were obtaned when dfferent knds of mage are processed. For the textured mage n Fgure 6, Jpeg2 outperforms Jpeg at every bt rates (Fgure 7). Artfacts are more evdent n the Jpeg coded mage even at qute hgh bt rates, as confrmed by a perceptve analyss. The mage n Fgure 8 shows an example wth more colors and detals than the prevous, and the superorty of Jpeg2, agan, decreases as bt rate ncreases. Jpeg has
776 IEEE Transactons on Consumer Electroncs, Vol. 49, No. 4, NOVEMBER 23 better performances snce the bt rate s greater or equal to 2 bpp (see Fgure 9). PSNR (db) 45 4 35 3 25 2 15 1 5.25.5 1. 1.5 2. 2.5 3. Jpg J2k bt rate (bpp) Fgure 6: Textured test mage for Jpeg/Jpeg2 comparson. PSNR (db) 4 35 3 25 2 15 1 5.25.5 1. 1.5 2. 2.5 3. bt rate (bpp) Fgure 7: PSNR comparson between Jpeg and Jpeg2 compresson for the pcture n Fgure 6. Fgure 8: Input test mage for Jpeg/Jpeg2 comparson. Jpg J2k Fgure 9: PSNR comparson between Jpeg and Jpeg2 compresson for the pcture n Fgure 8. IV RATE CONTROL The Jpeg2 core-codng engne s based on the 2-ters paradgm of the EBCOT [2], [22] algorthm. The frst ter provdes a low level codng of block samples belongng to the varous mage sub-bands by means of a context adaptve BAC (Bnary Arthmetc Codng) referred as MQ coder [17]. Once all blocks are compressed they are re-examned to produce a full packed bt-stream, where packets can be descrbed as subparts of code-blocks bt-stream that are spread across dfferent qualty layers wth each qualty layer ncreasngly mprovng the fnal mage. Ths framework provdes a bt-stream that s both resoluton scalable (f DWT s adopted) and qualty scalable (f multple layers are provded). Of course such structured bt-stream s well suted for wreless applcatons where bandwdth constrants requre maxmum lmted data transmsson and/or a lmted dsplay resoluton s avalable. Optmal block truncaton s decded wthn Rate-Dstorton optmsaton algorthms [16] to produce layers of fxed bt rate or dstorton. RD problems can be easly solved relyng on the well-known lagrangan multpler [12] method. If, for example, a target rate R max s desred for a gven layer the followng problem must be solved: Mnmze R = N 1 = R n D = R N 1 = max D n such that, where N are the number of the blocks, n the chosen truncaton pont for the -th block and n n R, D respectvely dstorton and rate for a block at a gven truncaton pont. Possble truncatons ponts (3*K-1 wth K the bt depth) are produced by the MQ-coder n 3 dfferent codng steps. Solvng (1) equals solvng the followng: Mnmze N 1 = (1) n n D( λ ) + λr( λ) = D + λr (2)
S. Battato et al.: Analyss and Characterzaton of JPEG 2 Standard for Imagng Devces 777, for a λ> found through bsecton to acheve the target rate. The [14] provdes a basc dstorton metrc but t would be appealng to draw a specfc metrc takng nto account specfc dsplay lmtaton (sze, bt-depth, vewng angle, ). Such drawbacks are, well known to be relevant for magng moble devces. The benefts comng from the above descrbed rate-control mechansm, known n lterature as PCRD (Post Compresson Rate Dstorton Optmzaton) poses some practcal restrctons both n terms of memory allocaton and overall computaton. Usng some adaptve sngle-pass analyss smlar to that descrbed n [8], a more effectve strategy could be desgned. V. PERCEPTIVE IMPROVEMENT The algorthms used for compresson factor control have the man purpose of assurng a fxed fle sze. Ths constrant affects heavly the choce of the trade-off between compresson rato and mage qualty. In order to mprove the vsual qualty of the mage for a fxed bt rate, some basc property of the HVS could be exploted to code fewer bts to represent perceptual less mportant areas of the mage. These propertes have been exploted n several methods n order to mprove Jpeg compresson standard performances [9], [11], [18], [23]. Usng smlar prncples, we developed a method to optmze the vsual qualty of Jpeg2 compressed mages, leavng the global bt-rate unchanged [4]. Snce Jpeg2 works parttonng the nput mage nto rectangular tles that are compressed ndependently, the global compresson rato s fxed assgnng the same bt-rate to every tle. The proposed approach re-dstrbutes the avalable bts over the tles, n order to preserve mportant data. It reduces the number of bts assgned to tles n whch dstorton s less vsble and, conversely, allocates more bts to tles n whch errors are more vsble. Ths results n an mage of better vsual qualty, whle the total number of bts coded remans unchanged. Tles that carry most mportant perceptual nformaton are found usng a lght but effectve classfcaton algorthm. Such technque, based on smple energy measures, s able to dstngush between dfferent areas (texture, homogeneous, edge ), n order to assgn a dfferent number of bts to tles. To obtan a sutable classfcaton of the tle content, the nput mage s fltered usng a combnaton of drectonal edgedetecton flters. We use Sobel flters to fnd horzontal and vertcal edges and Roberts flters to detect dagonal edges [13]. To each tle s then assocated a weght based on credts. A postve number of credts s assgned to tles wth vsually mportant detals and, conversely, a negatve number of credts s assocated wth tles where dstorton s less vsble. To guarantee real tme performances, only smple energy measures are computed. The classfcaton s based on two statstcal measures over the fltered mages: the mean of the fltered coeffcents of each tle, whch measures the quantty of edges n the tle, and the varance of each coeffcent wth wth respect to the mean, whch measures the mean slope of the lumnance values of the pxels close to the edges. A low mean ndcates that the tle has a plan background. If the varance s also low, then the tle has no sgnfcant detals. Conversely, a hgh varance means that the tle has well vsble detals. Hgh values for both mean and varance suggest that the tle coeffcents exhbt hgh varablty. Applyng approprate thresholds to the mean and varance measures, we are able to classfy tles. Exactly, each tle s classfed as Plan (f the area presents both low mean and varance values), Edge (low mean and hgh varance), Texture (hgh mean and varance values) or Unknown (when the classfer s not able to understand tle features). To deal wth colour mages, the proposed algorthm takes nto account only the lumnance channel. Expermental results show that the proposed approach has better performances than classc Jpeg2 n terms of perceptual qualty. Effectveness of the algorthm depends on the classfcaton and t s strongly related wth the tle sze and the bt-rate chosen. Of course, at hgh bt-rates no perceptble mprovements could be acheved varyng the dstrbuton of avalable bts among tles and, on the other hand, when the tle sze s too large t carres too much nformaton to buld a coherent classfcaton. (a) (b)
778 Fgure 1: (a) an example of tle classfcaton. (b) Compresson obtaned by standard Jpeg2. (c) Compresson obtaned usng the proposed approach. (d) Tles hghlghted n (b) and (c) enlarged to show the dfferent performances. VI. CONCLUSION The Jpeg 2 compresson standard allows workng wth mage compresson wth new and powerful advanced features. The overall performances are sensbly hgher than classcal JPEG, especally at low bt-rates. At frst glance such behavor coupled wth natve scalablty seems devoted to wreless applcatons. Nevertheless n real world the j2k format has not wdely adopted yet. Classcal Jpeg s stll nowadays the frst choce n magng devces maybe for two man reasons: 1 Manufacturer ndustry re-converson problem (DCT to DWT based processors); 2 Is the overhead due to the numerous codng optons and related codng complexty justfed for actual devce lmts? The paper has shown some ntal studes addressng the man basc characterstcs. Of course further nvestgaton s needed to fnd optmal trade-offs. REFERENCES [1] M. Antonn, M. Barlaud, P. Matheu, I. Daubeches, Image Codng Usng Wavelet Transform, IEEE Trans. On Image Processng, Vol.1, n.2, pp. 25-22, Aprl 1992. [2] S. Battato, A.Bosco, M. Mancuso, G. Spampnato, Temporal Nose Reducton of Bayer Matrxed Data, In Proceedngs of IEEE ICME 2 Internatonal Conference on Multmeda and Expo 22, Lausanne, Swtzerland, August 22. [3] S. Battato, A.Bruna, A. Buem, F. Naccar Codng Technques for CFA Data Images To appear n Proceedngs of IEEE ICIAP 3 Internatonal Conference on Image Analyss and Processng Mantova, Italy September 23. [4] S. Battato, A. Buem, G. Impoco, M. Mancuso, Jpeg2 Coded Images Optmsaton Usng a Content-dependent Approach, IEEE Transacton on Consumer Electroncs - Vol. 48, Issue 3, pp. 4-48, August 22. (d) IEEE Transactons on Consumer Electroncs, Vol. 49, No. 4, NOVEMBER 23 [5] S. Battato, A. Buem, L. Della Torre, A. Vtal, A Fast Vector Quantzaton Engne for CFA Data Compresson, To appear n Proceedngs of IEEE-EURASIP Workshop on Nonlnear Sgnal and Image Processng NSIP 23 Grado, Italy - June 23. [6] S. S. Battato, M. Mancuso, An Introducton to the Dgtal Stll Camera Technology, ST Journal of System Research - Specal Issue on Image Processng for Dgtal Stll Camera, Vol. 2, No.2, December 21. [7] B.E Bayer, Color magng array, US Patent 3,971,65 1976. [8] A. Bruna, M. Mancuso, JPEG Compresson Factor Control: A New Algorthm, n Proceedngs of IEEE ICCE21, pp.26-27, Los Angeles, June 21. [9] C. Chou, Y. L, A Perceptually Tuned Subband Image Coder Based on the Measure of Just-Notceable-Dstorton Profle, IEEE Trans. On Crcuts and Systems for Vdeo Technology, Vol.5, n.6, pp. 467-476, December 1995. [1] C. Chrstopulos, A. Skodras, T. Ebrahm, The Jpeg2 Stll Image Codng System: an Overvew, IEEE Trans. On Consumer Electroncs, Vol. 46, n.4, pp.113-1127, November 2. [11] S. Daly, W. Zeng, J. L, S. Le, Vsual Maskng n Wavelet Compresson for Jpeg2, SPIE/IS&T Electronc Imagng: Image and Vdeo Comm. and Processng, San Jose, January 2. [12] H. Everett, Generalzed Lagrange multpler method for solvng problems of optmum allocaton of resources, Operatons Research, vol. 11, pp.399-417, 1963. [13] R.C. Gonzales, R.E. Woods Dgtal Image Processng, Addson Wesley, pp. 197-21, 1993. [14] JPEG2 Image Codng System Part I, ISO/IEC FCD 15444-1. March 2. [15] JPEG2 Verfcaton Model 9.1, ISO/IEC JTC 1/SC 29/WG1 N2165, June 21. [16] A. Ortega, K. Ramchandran, Rate-Dstorton Methods for Image and Vdeocompresson, IEEE Sgnal Processng Magazne, November 1998. [17] W. B. Pennebaker, J. L. Mtchell, JPEG: Stll Image Compresson Standard, Van Nostrand Renhold, New York, 1993. [18] M. G. Perkns, T. Lookabaugh, A Psychophyscally Justfed Bt Allocaton Algorthm for Subband Image Codng Systems, pp.1815-1818, n Proceedngs of ICASSP, 1989. [19] A. Pulafto, M. Mancuso, M. Vllar, M. Guarnera, Dgtal stll cameras and moble agents: How to create a dstrbuted servce for mage processng, In proc. of EW22; [2] D. Taubman, Hgh performance scalable mage compresson wth EBCOT, IEEE Trans. Image Processng, vol. 9, pp. 1158-117, July 2. [21] D. Taubman, M. Marcelln JPEG2 Image Compresson Fundamentals, The Kluwer Int. Seres n Eng. and Computer Scence Volume 642 Hardbound, ISBN -7923-7519-X, Nov. 21. [22] D. Taubman, E. Ordentlch, M. Wenberger, G.Serouss, Embedded block codng n JPEG2, In Proceedngs of IEEE ICIP, 21. [23] K. A. Wen, C.Y. Lu, M. C. Tsa, The Transform Image Codec Based on Fuzzy Control and Human Vsual System, IEEE Trans. on Fuzzy Systems, Vol. 3, No. 3, 1995.
S. Battato et al.: Analyss and Characterzaton of JPEG 2 Standard for Imagng Devces 779 Sebastano Battato receved hs Italan degree n Computer Scence n 1995 and PhD n Appled Mathematcs and Computer Scence n 1999 both at Catana Unversty. From November 1999 to August 23 has worked at STMcroelectroncs n AST Catana Lab n the Dgtal Stll Camera Group as Algorthm Dev. Team Leader. Actually he works as contract professor at Catana Unversty. Hs current research nterests are n the area of Image Processng, Compresson (Jpeg, Jpeg2), Watermarkng, and Analyss/Synthess of Texture. Arcangelo Bruna receved hs Italan degree n Electronc Engneer n 1998 at the Unversty of Palermo. Hs thess was on vdeo ndexng database and features extracton based on MPEG sequences and was publshed by IEEE on ICMCS-1999. Then he worked n Rome for Telesoft frm on telecommuncaton area as consultant for about 9 months. Snce July 1999 he s workng at STMcroelectroncs n AST Catana Lab n the Dgtal Stll Camera Group. Durng the frst year hs man actvty was the research of new algorthms and archtectures for the JPEG compresson factor control and hs results allowed to get 3 patents. Nowadays he s workng on MOW (Multmeda Over Wreless), a project joned wth Agrate and Edmburgh Labs. Antono Buem receved hs Italan degree n Computer Scence at the Unversty of Catana n 1998. In 1999 he won a study grant announced by Italan Natonal Research Councl and he spent t at Unversty of Catana, developng new mage processng algorthms based on fuzzy logc. Snce the end of 2 has been workng at STMcroelectroncs n AST Catana Lab n the Dgtal Stll Camera Group as System Engneer. Hs current research actvty concerns the followng areas of Image Processng: compresson of dgtal mage (Jpeg2), textures classfcaton and fuzzy algorthms for medcal applcatons. Alfo Castorna receved hs Italan degree n Computer Scence n 2 at the Unversty of Catana dong a thess about watermarkng algorthms for dgtal mages. Snce September 2 he has been workng n STMcroelectoncs n the AST Dgtal Stll Camera Group as System Engneer. Hs current actvtes nclude hgh dynamc range magng, post-processng mage enhancement technques and Jpeg 2.