FAST INVERSE TONE MAPPING WITH REINHARD S GLOBAL OPERATOR. Yuma Kinoshita, Sayaka Shiota and Hitoshi Kiya

Transcription

1 FAST IVERSE TOE MAPPIG WITH REIHARD S GLOBAL OPERATOR Yuma Kinoshita, Sayaka Shiota and Hitoshi Kiya Tokyo Mtropolitan Univrsity Dpartmnt of Information and Communication Systms Tokyo, Japan ABSTRACT This papr proposs a novl invrs TMO, which nabls to gnrat HDR imags from ons, not only without using any spcific paramtrs but also at low computing costs. Furthrmor, th invrs TMO has a nw charactristic whn an imag is mappd from an HDR on by global oprator. In th cas, th HDR imag rconstructd by th proposd mthod without paramtrs can b rmappd into th sam imag as that rmappd from an HDR on rconstructd with paramtrs. Exprimntal rsults show that th proposd invrs ton mapping TM opration can b carrid out, whil kping bttr structural similarity and lowr computing cost than convntional mthods. Indx Trms high dynamic rang, invrs ton mapping, ton mapping, HDR imaging, imag nhancmnt. ITRODUCTIO Th intrst of high dynamic rang HDR imaging has rcntly bn incrasing in various ara: photography, mdical imaging, computr graphics, on vhicl camras and astronautics. Thy hav widr dynamic rang of pixl valus than standard low dynamic rang imags. To visualiz HDR imags and vidos, HDR displays hav bn dvlopd. Th most common approach to produc HDR imags with a dtctor is to squntially captur multipl imags of th sam scn using diffrnt xposurs [,2]. Howvr, this kind of approachs is only suitabl for static scns. In addition, ths approachs ar not suitabl for rproducing ral-world apparanc imags through lgacy imags or vidos. Thrfor, a numbr of invrs ton mapping oprators TMOs for dynamic rang xpansion hav bn proposd to visualiz imags on HDR dvics. Various rsarch works on invrs TM hav so far bn don [3 7]. Huo t al. succdd in xpanding th local dynamic rang in dark and bright ara by using th dodging and burning algorithm with a S curv oprator [6]. Wang t al. proposd an invrs TM opration which conflats psudo-multipl-xposurs HDR imags gnratd from a This rsarch was partly supportd by Grant-in-Aid for Rsarch on Priority Aras, Tokyo Mtropolitan Univrsity, Rsarch on social big data. singl imag [7]. In addition, th invrs TM is usd not only to gnrat an HDR imag from an on, but also to comprssion coding an HDR imag such as JPEG- XT standard [8]. A high-prformanc invrs TM opration at low computational costs is rquird bcaus HDR imaging tchniqus ar xpctd to apply to not only imags but also vidos. Howvr, th xisting invrs TM oprations rquir a lot of calculation for gnrating high quality HDR imags. For xampl, th Bantrl t al. s mthod [3] has complx procssing such as bilatral filtring and th mdian cut algorithm. This is du to a loss of information about th dynamic rang of th ral scn. To ovrcom ths problms, w propos a novl invrs TMO basd on th xisting TMO. Many rsarch works on TM oprations which gnrat an imag from an HDR imag hav bn studing [9 6]. Th proposd invrs TMO, which is basd on a invrs transform of global oprator [7,8], allows to b carrid out not only without any paramtrs but also without any data-dpndd calculation. Furthrmor, th proposd on has anothr application i.. rmapping imags into othr ons with dsirabl quality. Whn an imag is mappd from an HDR on by global oprator [7], th HDR imags rconstructd by th proposd mthod can b rmappd into th sam imag as that rmappd from an HDR on rconstructd with paramtrs. Thrfor, th proposd invrs TMO also allows to fficintly stor HDR imags as ons by convntional imag coding mthods such as JPEG, JPEG2000 and HEVC. W valuat th ffctivnss of th proposd invrs TMO in trms of th quality of gnratd HDR imags and th xcuting tim by a numbr of simulations. Simulation rsults show that th proposd mthod is abl to rmap ons mappd by global TMO, with high quality. In addition, it is confirmd that th proposd mthod is to ffctivly prform invrs TM maintaining th structural similarity with a low computational cost. 2. PREPARATIO A TM opration gnrats an imag I L from an HDR imag I H. A typical TM opration is rviwd /7/$ IEEE 972 ICASSP 207

2 2.. Photographic Ton Rproduction Photographic Ton Rproduction [7] which is a typical TM opration is summarizd, hr. This TM opration consists of th following six stps. a Th world luminanc L w p of an HDR imag I H is calculatd from RGB pixl valus of th HDR on as, L w p = 0.27Rp Gp Bp whr Rp, Gp and Bp ar RGB pixl valus of th HDR imag with a pixl p, rspctivly. b Th gomtric man L w of th world luminanc L w p is calculatd as follows: L w = xp log L w p 2 whr is th total numbr of pixls in th input HDR imag I H. c Th scald luminanc Lp is calculatd as Lp = α L w p 3 L w whr α [0, ] is th paramtr calld ky valu, which indicats subjctivly if th scn is light, normal, or dark [7]. α corrsponds to th gomtric man of th scald luminanc. d Th display luminanc L d p is calculatd by using a TMO y as follows: L d p = ylp. 4 Th global oprator [7] which is a wllknown TMO is givn by L d p = Lp + Lp. 5 Th floating-point pixl valus C f p of th imag is calculatd as follows: C f p = L dp Cp 6 L w p whr Cp {Rp, Gp, Bp} is th floating-point RGB valu of th input HDR imag I H, and C f p {R f p, G f p, B f p}. Bsids, th gamma corrction is prformd for C f p as ndd. f Th 8-bit color RGB valus C i p of th imag I Lg is drivd from C i p = roundc f p whr roundx rounds x to its narst intgr valu, and C i p {R i p, G i p, B i p} Scnarios This papr proposs a novl invrs TMO to stimat HDR imags from ons. W hav two scnarios from th diffrnc of applications. Scnario : Invrs TM of xisting imags Th first scnario is to stimat HDR imags from xisting ons capturd in lgacy format as shown in Fig.. In this cas, thr is no information on th rlation- Existing I L Proposd Invrs TM Paramtrs ar constants. Proposd Invrs TM Fig.. st scnario a Rmapping opration global TMO b st TM Estimatd HDR I H Arbitrary Stord Fig. 2. 2nd scnario ship btwn th ons and th ral scns. Thrfor, it is difficult to dtrmin oprators or paramtrs for invrs TM. As a rsult, convntional invrs TM oprations rquir high computing cost procssing for gnrating high quality HDR imags from xisting ons. By using th proposd mthod, fast invrs TM is abl to carry out without any paramtrs vn whn imags ar not gnratd by global oprator. Scnario 2: rmapping imags Th scond scnario is to rmap imags to othr ons with diffrnt qualitis by using arbitrary TMO shown in Fig. 2a. In th scnario, th first TMO is th global oprator s Fig. 2b. Th scond is an arbitrary TMO such as a local TMO, which can provid an imag with dsirabl quality. Bfor th scond TMO, an invrs TM opration is rquird. Convntionally, two paramtrs i.. α and L w hav to b stord to conduct it. In contrast, th proposd invrs TM opration do not rquir th us of th paramtrs. In th cas, th proposd invrs TMO has a nw charactristic that an HDR imag rconstructd by th proposd mthod without paramtrs can b rmappd into th sam imag I Li as that rmappd from an HDR on rconstructd with paramtrs. 3. PROPOSED IVERSE TM OPERATIO Assuming th us of th global oprator, a nw invrs TM opration basd on photographic ton rproduction is proposd. 3.. Invrs TMO basd on th TMO Th invrs function of th global oprator is givn as, from qs.3 and 5. L w p = L w α Lp = L w L d p 8 α L d p 973

3 whr Lp = L dp L d p. 9 It is confirmd that th ky valu α and th gomtric man L w ar ncssary to calculat q.8. Thrfor, ths paramtrs or original HDR imags ar rquird to b stord. By contrast, th proposd invrs TMO which dos not us th paramtrs is shown blow. L wp = G A Lp = G L dp 0 A L d p whr A, G is constant valus. Th abov quation corrsponds to th function substitutd by α = A and L w = G in q.8 rspctivly Proposd procdur Th procdur for Scnario is summarizd as follows.. Calculat a display luminanc L d p from RGB valus of an imag as L d p = 0.27R ip G i p B i p Calculat a world luminanc L wp by qs. and Obtain an HDR imag I H with color componnts C p {R p, G p, B p} as follows: C p = L wp L d p C f p = L wp L d p Cip Driving th proposd invrs TMO W discuss th validity that th paramtrs can b arbitrary valus in th proposd invrs TMO. First, w considr rmapping an invrs mappd imag by q. 0 basd on Photographic ton rproduction. Th gomtric man L w of an invrs mappd imag is calculatd by substituting q. 0 into q. 2, L w = xp = G A xp log G L dp A L d p log Lp = G α. 3 A Thn, th scald luminanc L p for th scond TMO is rprsntd as, from qs. 3 and 0 L p = α L L wp = α G L d p w L A L w d p. 4 By substituting q. 3 into q. 4, L p = α α L d p 5 L d p whr α is a ky valu for th scond TMO. From q. 5, th display luminanc L d p of th rmappd imag is providd by α L dp = yl p = y α L d p. 6 L d p ot that thr ar no constants A and G. On th othr hand, if α and L w ar known, th gomtric man L w of th invrs mappd imag is calculatd from q. 8 L w = xp log L w L d p = L w. 7 α L d p Thn, th scald luminanc L p for th scond TMO is rprsntd as, in a similar way to L p. L p = α L wp = α L w L d p α L d p L w = α α L d p 8 L d p Thrfor, th display luminanc L d p of th rmappd imag is providd by α L dp = yl p = y α L d p. 9 L d p By comparing q. 6 with q. 9, w arriv at th rlation: L dp = L dp. 20 Hnc, th rmappd rsult by th proposd framwork ar th sam imag as th rmappd on with th tru paramtrs α and L w rgardlss of th constants A and G. In this papr, A = G = is usd in th invrs TM for simplicity. L w 3.4. Evaluating th proposd mthod HDR imags gnrally hav a much widr dynamic rang than that of ons. For this rason, convntional quality assssmnts such as PSR or SSIM ar not suitd to valuat th quality of HDR imags. Thrfor, various rsarch works on valuating HDR imags hav so far bn don [9]. In this papr, w us typical quality mtrics i.. HDR-VDP- 2.2 [20] and PU ncoding [2] + MS-SSIM [22] to valuat th quality of HDR imags. To valuat th quality of HDR imags, rfrnc HDR imags ar ndd. Gnrally, thr ar no rfrnc HDR ons in datasts. Thrby, w choos HDR imags from HDR imag datast and gnrats ons from ths HDR ons by various TMOs. 4. SIMULATIO W valuat th proposd invrs TMO in trms of th quality of a gnratd HDR imag I H and th xcuting tim by a numbr of simulations with HDR imags. 4.. Simulation conditions W usd 60 HDR imags slctd from th databass [23, 24] for th valuation s Fig. 3. Th following procdur was carrid out to valuat th ffctivnss.. Map an HDR imag I H to an imag I Li by svral TMOs T MO i. In this simulation, w usd 0 TMOs slctd from th litratur [25] 2. Carry out som invrs TMOs, rfrrd to as it MO j for I Li, to obtain I Hij. 3. Evaluat th similarity btwn imags I Hij and I H, in accordanc with th critrions i.. HDR-VDP-2.2 MOS valu [20] and PU ncoding [2]+ MS-SSIM 974

4 0 adjustr tr mmorial ] c [ s t im g t in u c x E IPTR proposd Kuo's ITMO Huo's ITMO Fig. 3. Exampls of imags I Li mappd by svral TMOs [22]. Ths ar typical HDR imag quality assssmnts for valuating HDR imags [9]. Fiv invrs TMOs, i.. th proposd mthod, th convntional invrs opration using q.8 with th tru paramtrs α and L w, [7], Kuo s mthod [5] and Huo s mthod [6], ar compard. Th proposd mthod and th convntional invrs opration ar calld invrs photographic ton rproduction IPTR in th following sction. Th simulation was run on a PC, with a 3.4GHz procssor, a main mmory of 6Gbyts and MATLAB R204b Simulation rsults Figur 4 illustrats th avrag xcuting tim whn ach invrs TMOs ar carrid out 00 tims for 60 imags. From th figur, th proposd invrs TMO has much lowr computational cost than th othrs. On th othr hand, has larg computational cost. Tabls and 2 dnot th avrag valus of similaritis btwn HDR imags wr valuatd by HDR-VDP-2.2 [0, 00] and PU ncoding + MS-SSIM [0, ], whos lagr scor indicats a highr similarity btwn two imags. Tabl shows that IPTR is abl to invrs ton map imags mappd by global TMO with th highst quality. Th proposd invrs TMO also has th corrsponding proprty and is abl to map ons with highr quality than th othrs. By contrast, has th bst HDR-VDP- 2.2 scor which is th avrag of whol imags. Bsids, IPTR with paramtrs provids bttr MS-SSIM scor avragly as shown in tabl 2. Th proposd invrs TMO provids bttr HDR imags by comparing with th othrs whras it is slightly infrior than IPTR with paramtrs Thrfor, th proposd invrs TMO is ffctiv to prform invrs TM maintaining th structural similarity compard with othr invrs TMOs. 5. COCLUSIO This papr has proposd a novl invrs TMO basd on global oprator. Th proposd invrs TMO nabls to gnrat HDR imags from ons, not only without using any spcific paramtrs but also at low computing costs. Furthrmor, th invrs TMO has a charactristic whn an imag is mappd by global oprator. That is, an HDR imag rconstructd by th proposd Th numbr of pixls [*0 6 ] Fig. 4. Excuting tim of invrs TMOs Tabl. Exprimntal rsults HDR-VDP-2.2 invrs TMO IPTR IPTR with Kuo s Huo s proposd paramtrs ITMO ITMO global local Gamma TMO Chiu s TMO Fattal s TMO Avrag 0 TMOs Tabl 2. Exprimntal rsults PU ncoding + MS-SSIM invrs TMO IPTR IPTR with Kuo s Huo s proposd paramtrs ITMO ITMO global local Gamma TMO Chiu s TMO Fattal s TMO Avrag 0 TMOs mthod without paramtrs can b rmappd into th sam imag as that rmappd form an HDR on rconstructd with paramtrs. Th simulation rsults showd that th proposd invrs TMO is abl to prform invrs TM maintaining bttr structural similarity at lowr computing costs than othr invrs TMOs. 6. REFERECES [] P. E. Dbvc and J. Malik, Rcovring high dynamic rang radianc maps from photographs, in ACM SIG- GRAPH 2008 classs. ACM, 2008, p. 3. [2] S. K. ayar and T. Mitsunaga, High dynamic rang imaging: Spatially varying pixl xposurs, in Computr Vision and Pattrn Rcognition, Procdings. IEEE Confrnc on, vol.. IEEE, 2000, pp [3] F. Bantrl, P. Ldda, K. Dbattista, and A. Chalmrs, 975

5 Invrs ton mapping, in Procdings of th 4th intrnational confrnc on Computr graphics and intractiv tchniqus in Australasia and Southast Asia. ACM, 2006, pp [4] A. G. Rmpl, M. Trntacost, H. Stzn, H. D. Young, W. Hidrich, L. Whithad, and G. Ward, Ldr2hdr: on-th-fly rvrs ton mapping of lgacy vido and photographs, ACM Transactions on Graphics TOG, vol. 26, no. 3, p. 39, [5] P.-H. Kuo, C.-S. Tang, and S.-Y. Chin, Contntadaptiv invrs ton mapping, in Visual Communications and Imag Procssing VCIP. IEEE, 202, pp. 6. [6] H. Youngquing, Y. Fan, and V. Brost, Dodging and burning inspird invrs ton mapping algorithm, Journal of Computational Information Systms, vol. 9, no. 9, pp , 203. [7] T.-H. Wang, C.-W. Chiu, W.-C. Wu, J.-W. Wang, C.- Y. Lin, C.-T. Chiu, and J.-J. Liou, Psudo-multiplxposur-basd ton fusion with local rgion adjustmnt, Multimdia, IEEE Transactions on, vol. 7, no. 4, pp , 205. [8] ISO/IEC, ISO/IEC 8477 Information tchnology - Scalabl comprssion and coding of continuous-ton still imags, 205. [9]. Banić and S. Lončarić, Puma: A high-quality rtinx-basd tonmapping oprator, in th Europan Signal Procssing Confrnc EUSIPCO, 206, pp [0] Z. Zhu, Z. Li, S. Wu, and P. Fränti, ois rducd high dynamic rang ton mapping using information contnt wights, Acoustics, Spch and Signal Procssing ICASSP, 205 IEEE Intrnational Confrnc on, pp , 205. [] J. Duan and G. Qiu, Fast ton mapping for high dynamic rang imags, in Pattrn Rcognition, ICPR Procdings of th 7th Intrnational Confrnc on, vol. 2. IEEE, 2004, pp [2] S. Thakur, M. Sivasubramanian, K. allaprumal, K. Marappan, and. Vishwanath, Fast ton mapping for high dynamic rang imags, in Computational Intllignc and Computing Rsarch ICCIC, 203 IEEE Intrnational Confrnc on. IEEE, 203, pp. 4. [3] T. Dobashi, A. Tashiro, M. Iwahashi, and H. Kiya, A fixd-point implmntation of ton mapping opration for hdr imags xprssd in floating-point format, AP- SIPA Trans. Signal and Information Procssing, vol. 3, no., pp., 204. [4] T. Murofushi, M. Iwahashi, and H. Kiya, An intgr ton mapping opration for hdr imags xprssd in floating point data, in 203 IEEE Intrnational Confrnc on Acoustics, Spch and Signal Procssing. IEEE, 203, pp [5] T. Murofushi, T. Dobashi, M. Iwahashi, and H. Kiya, An intgr ton mapping opration for hdr imags in opnxr with dnormalizd numbrs, in 204 IEEE Intrnational Confrnc on Imag Procssing ICIP. IEEE, 204, pp [6] T. Dobashi, T. Murofushi, M. Iwahashi, and H. Kiya, A fixd-point ton mapping opration for hdr imags in th rgb format, in Proc. APSIPA Annual Summit and Confrnc. APSIPA, 203, pp. 4. [7] E. Rinhard, M. Stark, P. Shirly, and J. Frwrda, Photographic ton rproduction for digital imags, ACM Transactions on Graphics TOG, vol. 2, no. 3, pp , [8] E. Rinhard, W. Hidrich, P. Dbvc, S. Pattanaik, G. Ward, and K. Myszkowski, High dynamic rang imaging: acquisition, display, and imag-basd lighting. Morgan Kaufmann, 200. [9] P. Hanhart, M. V. Brnardo, M. Prira, A. M. Pinhiro, and T. Ebrahimi, Bnchmarking of objctiv quality mtrics for hdr imag quality assssmnt, EURASIP Journal on Imag and Vido Procssing, vol. 205, no. 39, 205. [20] M. arwaria, R. K. Mantiuk, M. P. Da Silva, and P. L Callt, Hdr-vdp-2.2: a calibratd mthod for objctiv quality prdiction of high-dynamic rang and standard imags, Journal of Elctronic Imaging, vol. 24, no., 205. [2] T. O. Aydın, R. Mantiuk, and H.-P. Sidl, Extnding quality mtrics to full dynamic rang imags, in Human Vision and Elctronic Imaging XIII, sr. Procdings of SPIE, San Jos, USA, January 2008, pp [22] Z. Wang, E. P. Simonclli, and A. C. Bovik, Multiscal structural similarity for imag quality assssmnt, in Signals, Systms and Computrs, Confrnc Rcord of th Thirty-Svnth Asilomar Confrnc on, vol. 2. IEEE, 2003, pp [23] Github - opnxr. [Onlin]. Availabl: com/opnxr/ [24] High dynamic rang imag xampls. [Onlin]. Availabl: originals.html [25] F. Bantrl, A. Artusi, K. Dbattista, and A. Chalmrs, Advancd High Dynamic Rang Imaging: Thory and Practic. atick, MA, USA: AK Ptrs CRC Prss,