New Image Restoration Method Based on Multiple Aperture Defocus Images for Microscopic Images

Size: px

Start display at page:

Download "New Image Restoration Method Based on Multiple Aperture Defocus Images for Microscopic Images"

Vernon McCormick
5 years ago
Views:

1 Sensors & Transducers, Vo. 79, Issue 9, September 204, pp Sensors & Transducers 204 by IFSA Pubishing, S. L. New Image Restoration Method Based on Mutipe Aperture Defocus Images for Microscopic Images,2 Shengi Fan, Mei Yu,,2 Yigang Wang, Gangyi Jiang Facuty of Information Science and Engineering, Ningbo University, Ningbo 35200, China 2 Dept. of Information, Ningbo Institute of Technoogy Zhejiang University, Ningbo 3500, China Te.: , fax: E-mai: yumei2@26.com Received: 2 May 204 /Accepted: 29 August 204 /Pubished: 30 September 204 Abstract: Image deconvoution is an effective image restoration technique to improve the quaity of digita microscopic images resuting from out-of-focus bur. To sove the severey i-posed probem of traditiona Richardson Lucy method, considering the point spread difference of various directions, a new microscope image restoration method based on mutipe defocused images of different aperture is proposed. The maximumikeihood estimation is used to suppress the ringing artifacts and noises sensitivity of microscope image. Experimenta resuts show that the proposed agorithm performs better than Richardson Lucy method and improve peak-signa-to-noise-rate about 4 db. Copyright 204 IFSA Pubishing, S. L. Keywords: Digita optica microscope, Image restoration, Mutipe aperture.. Introduction Recenty, digita optica microscope is becoming a vita too in industry, materias science and biomedica for its intuitive, ow cost and high degree of inteigence. It is hard to get a high quaity focus image since the depth of fied is very shaow whie the optica system deviation is arge. It is the primary probem that how to get a high quaity focus image in digita optica microscope for foow-up observing, processing and measuring easiy. Image deconvoution is an effective image restoration technique to improve image quaity. Image deconvoution can be categorized into two types, that is, bind deconvoution and non-bind deconvoution. The former is more difficut since the point spread function (PSF) is unknown. Natura image statistics together with a sophisticated variationa Bayes inference agorithm are used to estimate the PSF [], [2]. Even with a known PSF, non-bind deconvoution is sti under-constrained. In our approach, we significanty reduce the artifacts in a non-bind deconvoution by taking advantage of the out-of-focus noisy image. Image deconvoution probems are often soved by Richardson-Lucy (RL) deconvoution agorithm. Initiay it was derived from Bayes s theorem in the eary 970 s by Richardson and Lucy [3], [4]. The method is a noninear iterative method based on Bayesian anaysis which s maximum ikeihood estimation adapted to poisson statistics. Two important drawbacks of RL deconvoution, however, are ringing artifacts and noises sensitivity. To suppress this probem, reguarization constraints on the object are suppemented such as non-negative, tota variation, and normaization, etc. [5]-[7]. Further more, some prior knowedge on the data has to be appied to stabiize the soution. By incorporating an anti-refective boundary condition and a re-burring step, Donatei et a. recover a atent 62

2 Sensors & Transducers, Vo. 79, Issue 9, September 204, pp image with reduced ringing with a PDE-based mode [8]. Levin et a. use a sparse derivative prior to avoid ringing artifacts in deconvoution [9]. For the image deconvoution is i-posed, the common probem that methods above must face is how to suppress ringing and noise ampification whie preserving object edges. There are motion image restoration systems that use speciaized hardware, such as measurement sensors, to augment a camera to aid in the motion image restoration [0]. Other approaches are those that use mutipe images as input to do the job. Rav- Acha et a. combined information in two motion burred images [], whie Yuan et a. use information from two images, one burry one noisy, to restore image [2]. Rav-Acha et a. present that deconvouting use two motion-burred images are better than one [3]. But for out-of-focus image, there are ess research works. Therefore, we propose a nove defocus image restoration method based on mutipe images of different aperture, which contain bur direction weight information to suppress ringing and noise ampification effect. This paper is organized as foows: Section 2 expains imaging theory for defocus and introduces the conception of sub-psf and sub-bur-image. In section 3 an image restoration agorithm based on mutipe images of different aperture is proposed. Section 4 shows the experimenta resuts of the method proposed by this paper and the cassica Richardson-Lucy method. Finay, a concusion is drawn. 2. Mode for Defocus Consider the standard optica theory of singe ens imaging system, the object is focused when Eq.() is satisfied where f is foca ength, u is distance of the object principa, and v is distance of the focused image from ens pane [4], [5] as shown in Fig.. When an image sensor is paced at distance v ', which Eq. () is not satisfied, a the rays from a point in the scene wi and on mutipe sensor points resuting in an out-of-focus burred image. + =, () u v f The reationship between the origina true object image f ( x ) and the output image g( x ) of the system can be represented as: gx ( ) = hx ( ) f + nx ( ), (2) where * denotes the convoution operator, hx ( ) denotes PSF of the system, and nx ( ) is the random spatia distribution of noise. object u Lens f v ' v Focus point Fig.. Theory of image bur. Out-of-focus point PSF depends on not ony the distance of object, but aso the shape and size of the ens. Usuay, the ens is circuar and PSF can be we approximated with a disk function. That is, a focused point wi defocus as a burred circe. Further more, the burred circe above is composed of numbers of points which is anded by rays of different direction. To represent it, the sub-psf ( ) h x of direction is introduced. PSF is decomposed into severa sub-psfs which is given by n hx ( ) = h, (3) = Pug Eq.(3) into Eq.(2), we got gx ( ) = [ h] f = g n n, (4) = = where g ( x ) is the sub-burred-image which represents the diffusion amount of direction. 3. Image Restoration Agorithm There is an image restoration agorithm based on ML estimation in iterature [6], [7] using the ratio of the rea burred image and estimated burred image to cacuate the deviation. Actuay, burred image is convoution of sharp image and PSF. Thus the difference is used instead of the ratio to cacuate prediction deviation. Let ) ) s and gˆ () x be the estimate of f ( x ) and gx ( ) at iteration k, + s ) be the estimate of f ( x ) at iteration k +. Then the sharp image can be obtained by iterating the foowing equation: Δ s = h( x x) Δg dx, (5) ) ) i i i I 63

3 Sensors & Transducers, Vo. 79, Issue 9, September 204, pp where ) + ) ) Δ s = s s, (6) () () ˆ () ) ) Δ g x = g x g x, (7), (8) ) ) gˆ = h( x x ) s ( x ) dx i I i o o o PSF can be we approximated with a disk function in defocus bur usuay. Eq.(5) iustrates that the estimate sharp image difference Δs ) is eveny spread by the estimate bur image difference Δ g ). In fact, as to the definition of sub-psf, the diffusion of each direction is different. Thus prediction errors generated. In the iterative process, these errors wi be magnified, producing ringing effects and noise ampification probems. In order to suppress above probems, the most direct and effective way is to take the direction difference into account. The difference is we characterized by the sub-burred-images. It is expected that mutipe subburred-images is used instead of the origina singe image to get better restoration resuts. At appications, mutipe sub-burred-images can be obtained by changing the aperture. Framework of the proposed image restoration method is iustrated in Fig. 2. In the iterative process, the estimated sharp image deviation Δ s ) is d according to each sub-psf h ( x ) and the tota deviation used to update the estimated sharp image at iteration k + is the sum of above. The estimated sharp image deviation Δ s ) has rich direction information, so the proposed image restoration method takes the spread inhomogeneity of a direction into consideration fuy. The advantage of the method is that ringing effects and noise ampification probems are we soved meanwhie the effect of image restoration is guaranteed. sub-bur-image update the estiamtion estimated bur image with h estimated bur image with h 2 bur image difference bur image difference2 sharp image difference sharp image difference2 tota sharp image difference sub-bur-image2 Fig. 2. Framework of mutipe aperture image restoration method. The iterative steps of agorithm are as foows: ) ) Compute the estimated bur image gˆ () x of each direction from Eq.(8); 2) Compute the estimated bur image difference Δ g ) from Eq.(7); 3) Compute the estimated sharp image difference Δ s ) from Eq.(5); 4) The tota estimated sharp image difference is the sum of Δ s ) ; 5) Compute the tota estimated sharp image + s ) from Eq.(6); 6) Iteration times not reach, return to step ), or end the iteration. 4. Experimenta Resut Two sampe images (52 52 pixes) incuding LENA image and PCB image are seected as rea sharp images. LENA image is a very cassic sampe image with abundant band information and PCB image is a rea image captured with NOVEL OPTICS NSZ-800 type optica microscope, as shown in Fig. 3(a) and Fig. 3(c). The burred image is generated by convoving h(x) with parameters r=9, we can obtain the burred image, as shown in Fig. 3(b) and Fig. 3(d). We appied our method to the synthetic data. As shown in Fig. 4(a), the deviation vaue between estimated burred image and the actua burred image is not zero, which means that there are significant differences. However, according to Eq.(5), the estimated sharp image deviation is amost zero, as shown in Fig. 4(b). These resut ringing effects in deburred image as shown in Fig. 5(a). In order to suppress ringing effect, two sub-burimages as shown in Fig. 4(d) and Fig. 4(e) were got by convouting origina sharp image with two different sub-psfs h and h 2, where h= h+ h2, (9) 64

Sensors & Transducers, Vo. 79, Issue 9, September 204, pp. 62-67 (a) (b) (c) (d) Fig. 3. Sampe image: (a) LENA sharp image; (b) LENA bur image; (c) PCB sharp image; (d) PCB bur image.

LENA image restoration process diagram: (a) estimated burred image deviation; (b) estimated sharp image deviation; (c) PSF h and sub-psfs h,h2; (d) sub-bur-image of h; (e)

PSF h and sub-psfs h, h 2 are shown in Fig. 4(c). Fig. 4(f), Fig. 4(g) were corresponding estimated burred image deviation which is much smaer than Fig. 4(a).

4(f), Fig. 4(g), Fig. 4(h) is magnified 500 times. Levin et a. use a coded aperture instead of conventiona aperture to get a sharper image [9].

Severa sub-bur-images were got by decomposing "coded PSF" into severa sub-psfs from which a sharper image was got, as shown in Fig.

4 Sensors & Transducers, Vo. 79, Issue 9, September 204, pp (a) (b) (c) (d) Fig. 3. Sampe image: (a) LENA sharp image; (b) LENA bur image; (c) PCB sharp image; (d) PCB bur image. (a) (b) (c) (d) (e) (f) (g) (h) Fig. 4. LENA image restoration process diagram: (a) estimated burred image deviation; (b) estimated sharp image deviation; (c) PSF h and sub-psfs h,h2; (d) sub-bur-image of h; (e) sub-bur-image of h2; (f)estimated burred image deviation of h; (g)estimated burred image deviation of h2; (h) estimated sharp image deviation of mutipe aperture. PSF h and sub-psfs h, h 2 are shown in Fig. 4(c). Fig. 4(f), Fig. 4(g) were corresponding estimated burred image deviation which is much smaer than Fig. 4(a). Restored image quaity is greaty improved as shown in Fig. 6(b). A resuts are obtained in 500 iterations. In order to anayze, the pixe vaue of images in Fig. 4(a), Fig. 4(b), Fig. 4(f), Fig. 4(g), Fig. 4(h) is magnified 500 times. Levin et a. use a coded aperture instead of conventiona aperture to get a sharper image [9]. In our case, "coded PSF" is used instead of "disk PSF". Severa sub-bur-images were got by decomposing "coded PSF" into severa sub-psfs from which a sharper image was got, as shown in Fig. 5. Fig. 6 demonstrates the restored images of LENA by cassica Richardson-Lucy method (singe disk aperture image), proposed disk mutipe aperture method (mutipe disk aperture images), Levin's method (singe coded aperture image) and proposed coded mutipe aperture method (mutipe coded aperture images), as shown in Fig. 6(a) ~ Fig. 6(d). Whie our method produces a sharper image with smaer ringing artifacts, especiay in image edge area, as shown in Fig. 6(e) ~ Fig. 6(h). (a) Fig. 5. LENA image restoration process diagram: (a) LENA burred image by "coded PSF"; (b) coded PSF h and sub-psfs h, h2, h3, h4, h5. (b) 65

Sensors & Transducers, Vo. 79, Issue 9, September 204, pp. 62-67 Peak Signa to Noise Ratio (PSNR) is used to evauate the effect of image restoration. Fig. 7 and Fig.

5 Sensors & Transducers, Vo. 79, Issue 9, September 204, pp Peak Signa to Noise Ratio (PSNR) is used to evauate the effect of image restoration. Fig. 7 and Fig. 8 demonstrate the reationship of image restore quaity and iterations. In condition of same iterations, the mutipe aperture image restoration method produces better resut than the singe aperture image restoration method. And with the increase in the iterations, the recovery resuts tend to staby. At this moment, the PSNR vaue of mutipe aperture is higher than singe aperture about 4 db. PSNR(dB) Code Singe-aperture 26 Code Muti-aperture Disk Singe-aperture Disk Muti-aperture iteration number Fig. 7. Reationship of LENA image restore quaity and iterations. 42 (a) (e) PSNR(dB) (b) (f) 30 Code Singe-aperture Code Muti-aperture 28 Disk Singe-aperture Disk Muti-aperture iteration number Fig. 8. Reationship of PCB image restore quaity and iterations. (c) (g) 5. Concusions In this paper, an image restoration approach based on mutipe images of different aperture is proposed. The cause of ringing effects and noise ampification probem is anayzed and severa sub-bur-images with different direction information are used to overcome above probems. The experimenta resuts showed that ringing effects and noise ampification probem were we suppressed and the restore quaity was improved greaty. Future work may incude the optimization method of combination different aperture according to different images. Acknowedgements (d) Fig. 6. Comparison of deburring agorithms with LENA: (a) deburred LENA image by cassica Richardson-Lucy method; (b) deburred LENA image by proposed disk mutipe aperture method; (c) deburred LENA image by Levin's method; (d) deburred LENA image by proposed coded mutipe aperture method; (e-h) detais of restored images with above method. (h) This work is supported by the Natura Science Foundation of China (Grant Nos , ). References []. R. Fergus, B. Singh, A. Hertzmann, et a, Removing camera shake from a singe photograph, ACM 66

Sensors & Transducers, Vo. 79, Issue 9, September 204, pp. 62-67 Transactions on Graphics, Vo. 26, Issue 3, 2006, pp. 787-794. [2]. M. Ben-Ezra, S. K.

Lucy, An iterative technique for the rectification of observed distributions, Astronomica Journa, Vo. 79, 974, pp. 745-754. [4]. W. H.

Agarwaa, High-quaity motion deburring from a singe image, ACM Transactions on Graphics, Vo. 27, Issue 3, 2008, pp. -0. [6]. D. Nicoas, B. Laure, Z.

6 Sensors & Transducers, Vo. 79, Issue 9, September 204, pp Transactions on Graphics, Vo. 26, Issue 3, 2006, pp [2]. M. Ben-Ezra, S. K. Nayar, Motion-based motion deburring, IEEE Transactions on Pattern Anaysis and Machine Inteigence, Vo. 26, Issue 6, 2004, pp [3]. L. B. Lucy, An iterative technique for the rectification of observed distributions, Astronomica Journa, Vo. 79, 974, pp [4]. W. H. Richardson, Bayesian-based iterative method of image restoration, Journa of the Optica Society of America, Vo. 62, Issue, 972, pp [5]. Q. Shan, L. J. Jia, A. Agarwaa, High-quaity motion deburring from a singe image, ACM Transactions on Graphics, Vo. 27, Issue 3, 2008, pp. -0. [6]. D. Nicoas, B. Laure, Z. Christophe, et a, Richardson-Lucy agorithm with tota variation reguarization for 3D confoca microscope deconvoution, Microscopy Research and Technique, Vo. 69, 2006, pp [7]. Y. Wang, Q. Dai, Q. Cai, et a, Bind deconvoution subject to sparse representation for fuorescence microscopy, Optics Communications, Vo. 286, 203, pp [8]. M. Donatei, C. Estatico, A. Martinei, and S. Serracapizzano, Improved image deburring with antirefective boundary conditions and re-burring, Inverse Probems, Vo. 22, Issue 6, 2006, pp [9]. A. Levin, R. Fergus, F. Durand, et a, Image and depth from a conventiona camera with a coded aperture, ACM Transactions on Graphics, Vo. 26, Issue 3, 2007, Artice No. 70, pp. -9. [0]. N. Joshi, S. B. Kang, C. L. Zitnick, et a, Image deburring using inertia measurement sensors, ACM Transactions on Graphics, Vo. 29, Issue 4, 200, Artice No. 30, pp. -9. []. A. Rav-Acha, S. Peeg, Two motion-burred images are better than one, Pattern Recognition Letters, Vo. 26, Issue 3, 2005, pp [2]. L. Yuan, J. Sun, L. Quan, et a, Image deburring with burred noisy image pairs, ACM Transactions on Graphics, Vo. 26, Issue 3, 2007, Artice No.. [3]. J. Chen, C. Tang, Robust dua motion deburring, in Procedeengs of the IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, June 2008, pp. -8. [4]. F. Paoo, B. Martin, J. Stankey, Shape from defocus via diffusion, IEEE Transactions of Pattern Recognition and Machine Inteigence, Vo. 30, Issue 3, 2008, pp [5]. Y. Wei, C. Wu, Z. Dong, Goba shape reconstruction of nano grid with singy fixed camera, Science China Technoogica Sciences, Vo. 54, Issue 4, 20, pp [6]. S. Yuan, C. Preza, 3D fuorescence microscopy imaging accounting for depth-varying point-spread functions predicted by a strata interpoation method and a principa component anaysis method, in Proceedings SPIE 7904, Three-dimensiona and Mutidimensiona Microscopy: Image Acquisition and Processing XVIII, San Francisco, Caifornia, USA, 22 January 20, pp M-7. [7]. C. Preza, J. Concheo. Depth-variant maximumikeihood restoration for three-dimensiona fuorescence microscopy, Journa of the Optica Society of America, Vo. 2, Issue 9, 2004, pp Copyright, Internationa Frequency Sensor Association (IFSA) Pubishing, S. L. A rights reserved. ( 67

Improving the Active Power Filter Performance with a Prediction Based Reference Generation

Improving the Active Power Filter Performance with a Prediction Based Reference Generation Improving the Active Power Fiter Performance with a Prediction Based Reference Generation M. Routimo, M. Sao and H. Tuusa Abstract In this paper a current reference generation method for a votage source