Revisiting Cross-channel Information Transfer for Chromatic Aberration Correction

Transcription

1 Reviiting Cro-channel Information Tranfer for Chromatic Aberration Correction Tiancheng Sun, Yifan Peng 3, Wolfgang Heidrich,3 King Abdullah Univerity of Science and Technology, Thuwal, Saudi Arabia IIIS, Tinghua Univerity, Beijing, China 3 The Univerity of Britih Columbia, Vancouver, Canada. Algorithm analyi.. Flow chart We provide a detailed flow chart of our joint algorithm. A hown in Fig., we firt make a gue of latent image uing the propoed initialization, then iterate between PSF etimation and cro-channel tranfer (CCT) tep until reaching the convergence or termination condition. During the iterating proce, we apply a imple trick to avoid boundary effect of convolution. Notice that in the main text, tep of the algorithm i written in a lightly different way a: α = argmin α J CCT B (T(I CCT G ) α). () In practice, we put the convolution operation inide T ince both of them are linear. However, operating convolution on each CCT window may introduce artifact at the border. In the contrat, we pre-compute the full ize image of B I G in the PSF etimation tep, and crop the CCT window from the pre-computed image in the CCT tep, a hown in Fig.. Initialization for each CCT window: I CCT merge I CCT = mean(icct ) mean(i CCT G ) ICCT G to full ize PSF etimation for each PSF window: B PSF = argmin B +µkb k + P a kr ab k J PSF B I PSF CCT tep CCT tep CCT ize PSF tep. PSF tep PSF ize if not converge merge B PSF I PSF G Cro channel tranfer for each CCT window: I CCT = T(I CCT G ) = argmin merge I CCT to full ize for later ue J CCT T (B I G ) CCT to full ize Figure : Working flow chart of our joint algorithm. The left two inet illutrate the different window election for CCT tep and PSF tep.

2 .. Convergence analyi Fig. how the SSIM aement of image ubject to iteration round. From the plot we ee that the image quality converge very quickly after the firt couple iteration 0 (i.e iteration), and may drop a little bit when the iterating proce continue. We acribe thi weak drop of image quality to poible over-fitting to the pixel value of green channel Oberved Initial gue Oberved Initial gue Oberved Initial gue Oberved Initial gue Oberved Initial gue Oberved Initial gue Oberved Initial gue Oberved Initial gue Figure : SSIM aement of the retored image with different iteration round. To examine the reaoning of over-fitting, we invetigate an extreme cae where the dratic hue change appear in the image, a hown in the left of Fig. 3. We ynthetically blur the image uing a blur kernel ued in the main text, accordingly we have the blurred image (mid-left). Notice that in the green channel of thi blurred image (preented in the right of Fig. 3), the pixel intenitie are very low on the region of red flower. Apparently, the cro-channel correlation doen t fit well into our model. In thi cenario, directly uing green channel a the reference channel may caue a ignificant lo of mot fine detail and the over-fitting in the plain black part of green channel. However, for other region, including the tamen and ome of the petal of red flower, the reolved image ha preerved the detail well, that in ome ene validate the robutne of our algorithm. The main reaon accounting for thi over-fitting i the implicity of our cro- channel model. We yield thi problem to future work by exploiting an improved model of cro-channel imilarity, for intance uing learning-baed trategie..3. Run time dicuion A reported in the main text, our algorithm take around ec to retore one channel of a,400,000 pixel image. It i worth noting that toring a whole image could take more than jut 4 ec, conidering the time (around min) to blindly deconvolve [4] the reference channel. For mot of the captured image, the green channel of the image i relatively harp uch that the deconvolution hall be fat and robut. Since we don t have the executable code of tate-of-the-art cro-channel deconvolution algorithm, it i tough to give an intuitive quantitative comparion. However, with the aid of the propoed compact but powerful image formation, we believe our algorithm i very competitive in term of computational efficiency.

3 (a) ground truth image (b) blurred image (c) deblurred image (d) green channel (blurred) Figure 3: Extreme example with dratic hue change (a poible failure cae of our algorithm).. Reult with an additional prior added on latent image A mentioned in the main text, we have run the experiment with a gradient-baed cro-channel prior added in the deconvolution cheme a a comparion. Specifically, we modify tep in the decribed algorithm into: α = argmin JCCT B ICCT α CCT CCT ICCT + γ ICCT G IG I, () where ICCT = T(ICCT G ) α. We found that the quality of the reulting image doen t increae (even drop a little bit), a hown in Tab.. Although we are uing l norm, the comparion reult till ugget that our model ha covered the function of a regular cro-channel prior. More pecifically, we are directly uing the pixel value of the reference channel, which fully atify the ditribution of natural image, thu, we don t need further weak tatitical prior to recover the corrupted detail. Without prior With prior Average Table : Quantitative comparion on algorithm with and without gradient prior added. 3. Additional yntheized reult In the main text, we have hown the averaged PSNR and SSIM aement on image elected from the dataet. Here we how the quantitative comparion between different algorithm on each image in Fig. 4. Notice that for mot of the image, our algorithm outperform other with repect to PSNR and SSIM value, and exhibit the cloet reult to the non-blind deconvolution verion. For the reult from Yue et al. [], ince we didn t have the ource code, only the firt 0 image are available for the comparion. The full verion of Fig. in the main text i preented here in Fig.. Particularly, compared to tate-of-the-art algorithm, our uccefully eliminate the color fringe all acro the ditorted image to the lowet level. 4. Additional real world reult 4.. Diffractive len data Full-ize reult are preented in Fig.. A hown in the image, chromatic aberration have better correction in our reolved one, including large chromatic aberration around highlight (ee econd patch of firt row, and third patch of econd row), thoe induced by harp color change (ee econd path of econd row, and firt patch of third row), and thoe due to defocu (red part in the econd path of third row). Alo, all thee image exhibit clearer information than other reult, epecially in the third patch of firt row and the firt patch of fourth row, where the decoration pattern on the gla and the text on the back can be recognized more eaily. 3

4 (a) PSNR (b) SSIM Figure 4: Quantitative comparion reult between different algorithm on teting image. For almot all the teting image, our reult out-perform other and are the cloet to the reult of non-blind algorithm. 4.. Refractive len data Full-ize reult are preented in Fig.. Similar to the previou reult of diffractive optic, the reult on refractive lene how that our algorithm can handle different kind of chromatic aberration properly without acrificing fine-grain detail (firt patch of firt row,third batch of econd row). Surpriingly, the reult of our blind algorithm are even clear than a non-blind one (ee the number in the econd patch of firt row). The reult of correcting motion blur (fourth row) have alo hown the ability of our algorithm to deal with more general deblur problem. However, there alo exit ome color fidelity problem in our reult (ee third patch of firt row, and third patch of third row), which we hope to fix it uing learning method or adaptive window ize in the future.. Dicuion on pecial cae We have teted our algorithm on the image without noticeable blur. A hown in Fig., via directly uing the ame etting in previou experiment, our algorithm work quite well on unblurred image. That i to ay, for ome region of an image that uffer from le blurry effect, we don t have to fine tune the algorithm to avoid cauing artifact a many deconvolution algorithm may need. 4

5 Reference [] F. Heide, Q. Fu, Y. Peng, and W. Heidrich. Encoded diffractive optic for full-pectrum computational imaging. Scientific Report,, 0. [] F. Heide, M. Rouf, M. B. Hullin, B. Labitzke, W. Heidrich, and A. Kolb. High-quality computational imaging through imple lene. ACM Tranaction on Graphic (TOG), 3():4, 03. [3] E. Kee, S. Pari, S. Chen, and J. Wang. Modeling and removing patially-varying optical blur. In Computational Photography (ICCP), 0 IEEE International Conference on, page. IEEE, 0. [4] D. Krihnan, T. Tay, and R. Fergu. Blind deconvolution uing a normalized parity meaure. In Computer Viion and Pattern Recognition (CVPR), 0 IEEE Conference on, page IEEE, 0. [] C. J. Schuler, M. Hirch, S. Harmeling, and B. Schölkopf. Non-tationary correction of optical aberration. In Computer Viion (ICCV), 0 IEEE International Conference on, page. IEEE, 0. [] T. Yue, J. Suo, J. Wang, X. Cao, and Q. Dai. Blind optical aberration correction by exploring geometric and viual prior. In The IEEE Conference on Computer Viion and Pattern Recognition (CVPR), June 0.

6 (b) Blurred (c) Non-blind (d) Krihnan et al. [4] (e) Yue et al. [] (f) Heide et al. [] (g) Our (b) Blurred (c) Non-blind (d) Krihnan et al. [4] (e) Yue et al. [] (f) Heide et al. [] (g) Our (b) Blurred (c) Non-blind (d) Krihnan et al. [4] (e) Yue et al. [] (f) Heide et al. [] (g) Our (a) Ground truth (a) Ground truth (a) Ground truth Figure : Full reolution verion of image retored via different algorithm. Our exhibit the leat chromatic aberration artifact compared with tate-of-the-art work.

7 (b) Heide et al.[] Figure : Full-ize retored image captured by a diffractive len. (c) Our

8 (b) Heide et al. (non-blind) [] (c) Our (b) Heide et al. (non-blind) [] (c) Our (b) Yue et al. [] (c) Our (b) Krihnan et al. [4] (c) Our Figure : Full-ize retored image captured by refractive lene from tate-of-the-art work.

9 (a) Original image (b) Retored image Figure : The reult of our algorithm on unblurred image. We apply our algorithm on the latent image with exactly the ame parameter (PSF window ize, CCT window ize). The reulting image are viually identical to the original one, uggeting that our parameter are adaptable ubject to different kind of blurry level.