SURVEILLANCE SYSTEMS WITH AUTOMATIC RESTORATION OF LINEAR MOTION AND OUT-OF-FOCUS BLURRED IMAGES. Received August 2008; accepted October 2008

Size: px

Start display at page:

Download "SURVEILLANCE SYSTEMS WITH AUTOMATIC RESTORATION OF LINEAR MOTION AND OUT-OF-FOCUS BLURRED IMAGES. Received August 2008; accepted October 2008"

Emil Boone
5 years ago
Views:

1 ICIC Express Letters ICIC International c 2008 ISSN X Volume 2, Number 4, December 2008 pp SURVEILLANCE SYSTEMS WITH AUTOMATIC RESTORATION OF LINEAR MOTION AND OUT-OF-FOCUS BLURRED IMAGES Chung-Hao Chen 1, TeYu Chien 1, Wen-Chao Yang 2 Jan-Mou Li 3 and Che-Yen Wen 2 1 The University of Tennessee Knoxville, USA { cchen10; tchien }@utk.edu 2 National Central Police University, Taiwan {una135; cwen}@mail.cpu.edu.tw 3 University of Idaho, USA jli@uidaho.edu Received August 2008; accepted October 2008 Abstract. Linear motion and out-of-focus blur often coexist in a surveillance system, which degrade the quality of acquired images and thus complicate the task of object recognition and event detection. In this work, we present a point spread function-based (PSFbased) approach considering fundamental characteristics of linear motion and out-offocus blur based on geometric optics to restore coexisting motion and out-of-focus blurred images without application-dependent parameters selection, where a sharpness measure is employed as a cost function to automatically select optimal parameter values for PSF. To verify the effectiveness of our proposed approach, we compare our approach with existing de-blur approaches. Experimental results shows our proposed method can automatically select optimal parameter values for PSF and outperform existing de-blur approaches. Keywords: Surveillance systems, Digital imaging process, Out-of-focus blurred images, Linear motion blurred images 1. Introduction. Image blur cased by object or camera motion, and out-of-focus during image acquisition can case a serious degradation in terms of the image quality. As a result, this deteriorates the effectiveness of object recognition and event detection in a surveillance system. However, most existing approaches solve this problem separately. For instance, Lee et al. [8] proposed a new hierarchical out-of-focus blur identification and then restore the blurred images. Ben-Ezra and Nayayr [1] introduced hybrid imaging for motion de-blurring. In particular, Hamamoto and Aizawa [5], and Liu and Gamal [7] designed new CMOS sensors to prevent motion blur. Even though those methods prove efficient in either camera motion or out-of-focus blurred images, they have two limits: (1) linear motion and out-of-focus blur image often coexist in real life surveillance systems. Addressing either one of those problems can make the tasks of object recognition and event detection inefficient in surveillance systems; (2) most cities had deployed surveillance cameras for the purpose of community security. It is impractical to replace them with new and advanced camera configurations in the current situation. Even though Fabian and Malah [3] proposed a method based on PSF to address the problem of simultaneously existing motion and out-of-focus blur image, PSF models they introduced are not able to efficiently and automatically select parameters so that it directly affects the efficiency of the restoration and increase the time needed to de-blur images. Favaro et al. [4], therefore, proposed an anisotropic diffusion model to solve the problem. However, their methods are time consuming. Toriu and Nakajima s work [2], which calculates the saliency map of input images, inspires us to use a PSF-based 409

2 410 C.-H.CHEN,T.Y.CHIEN,W.-C.YANG,J.-M.LIANDC.-Y.WEN approach considering fundamental characteristics of linear motion and out-of-focus blur based on geometric optics to restore coexisting motion and out-of-focus blurred images without application-dependent parameters selection, where a sharpness measure is employed as a cost function to automatically select optimal parameter values for PSF. This is because successfully recognizing objects via acquired digital images is an ultimate goal for a surveillance system. However, Chen et al. [2] points out a successful recognition requires sufficient local details to differentiate various patterns in images and those details can be translated into the sharpness of acquired images. To verify the effectiveness of our proposed approach, we compare our approach with existing de-blur approaches, laplacian filtering, unsharp masking, and Katsaggelos and Lay s maximum likelihood blind convolution [6]. Those approaches are commonly used as ground truth in this area. Thus, the contributionsofthispaperare:(1)apsf-based approach with the automated selection of parameter values is proposed. It outperforms existing de-blur approaches in terms of objective and adjective evaluations; (2) our proposed model can be used in the currently deployed camera surveillance systems without changing their hardware configurations. The paper is organized as follows. The flow chart of our proposed de-blur algorithm is give in Section 2. Section 3 introduces our proposed PSF-based approach. Section 4 illustrates the experimental results. Section 5 concludes the paper. 2. Algorithm Architecture. Figure 1 shows the flow chart of our proposed de-blur algorithm. Figure 1. Illustration of the flow chart of our proposed de-blur algorithm In Figure 1, an existing surveillance system first acquires the image g(x, y) where outof-focus blur and linear motion blur coexist. G(x, y) is generated by the Fourier Transformation of g(x, y). Since the de-blur processing needs following parameters as the input: (1) the variance of the Gaussian function, σ; (2)theangleθ, between the velocity vector of 120 the object and x coordinate in the image; (3) the travel distance, * v T,ofthe object with linear motion in the image, the sharpness measure proposed by Chen et al. [2] is employed as a cost function to automatically select optimal parameter values. Details about our proposed de-blur processing will beaddressedinsection3. Oncetheoptimal parameter values are selected, the de-blured image f(x, y) is generated accordingly.

3 ICIC EXPRESS LETTERS, VOL.2, NO.4, Main Results. A widely used camera model consisting of a convex lens and an image plane is taken to obtain fundamental characteristics of linear motion and out-of-focus blur. Different from most existing PSF-based approaches for determining the blur function, we derive fundamental characteristics of linear motion and out-of-focus blur based on geometric optics to reduce the struggle to select unknown parameters. In our work, the image blur caused by the both camera/object linear motion and out-of-focus are taken into consideration. First, we model linear motion and out-of-focus blurs separately, and then derive a combined model to address both linear motion and out-of-focus blur. We first review the camera model for geometric image formation process. Since fundamental camera system consists of a single convex lens with focal length f, the relationship between a focus position in the scene and the corresponding focused position in the image plane is given by 1 p + 1 q = 1 (1) f where p is the distance between the focus position in the scene and the convex lens, and q is the distance between the corresponding focused position in the image plane and the convex lens. If the object in the scene moves a distance d parallel to the image plane, we have d p = d0 (2) q where d 0 is the displacement of the corresponding position in the image plane. From Equation (2), if we assume the film exposure time T is small, the object in the image plane appears to have the same intensity over the displacement d 0. On the other hand, PSF of linear motion blur model can be described as a rectangular function along a line in two dimensional coordinate, which is shown by 1 w(x 0,y 0 )= * v T, T v x x 0 T vx ; 2 2 y0 = v y v x x 0 (3) 0, otherwise where * v is the velocity vector of the object in the image plane; v x and v y are the two components of this velocity vector, respectively. In conclusion, the model for motion blur is Z g m (x, y) = f(x Z*v 0,y 0 )w(x x 0,y y 0 )dx 0 dy 0 (4) where g m (x, y) is the observed image with linear motion blur. f(x 0,y 0 ) is the de-blurred image. If we consider an object is in the out-of-focus position. The out-of-focus blur comes from the corresponding virtual focus position falling out of the image plane. Thus, what falls on the image plane is a circle with higher intensity in the center. On the other hand, PSF of out-of-focus blur model can be described as a Gaussian distribution function. In conclusion, the model for out-of-focus blur is ZZ 1 g o (x, y) = f(x 0,y 0 ) 2πσ e [(x 0 x) 2 +(y 0 y) 2 ] 2 2σ 2 dx 0 dy 0 (5) where g o (x, y) is the observed image with out-of-focus blur. σ is the variance for Gaussian distribution function. Since a coexisting linear motion and out-of-focus blurred image can be translated into a linear motion blur on an out-of-focus blur image, the model considering both linear motion and out-of-focus blur, therefore, can be described as g(x, y) = Z*v Z g o (x 0,y 0 )w(x x 0,y y 0 )dx 0 dy 0 (6)

4 412 C.-H.CHEN,T.Y.CHIEN,W.-C.YANG,J.-M.LIANDC.-Y.WEN where g(x, y) is the observed image with both out-of-focus blur and linear motion blur. The Fourier transform of Equation (7) is G(k x,k y )=F(k x,k y ) Gauss (k x,k y ) W (k x,k y ) (7) WiththehelpofEquation(7),wecandivideG(k x,k y ) by the known function of the Fourier transform of Gaussian function, Gauss(k x,k y ), and rectangular function, W (k x,k y ), to obtain the Fourier transform of the de-blurred image, F (k x,k y ). The final step is to use inverse Fourier transformation to obtain the de-blurred image, f(x, y). In particular, we need to choose three parameters: (1) the variance of the Gaussian function, σ; (2)theangleθ, between the velocity vector and x coordinate in the image; (3) the travel distance, * v T, of the object with linear motion in the image. In order to select optimal parameter values, we use a sharpness measure to quantify the confidence of those selected parameter values. Furthermore, it is better to have the constraints for each parameter in order to avoid the ill numbers and avoid tedious computation time. First, we assume the limit of the Gaussian variance σ variesfrom0tothesizeofthe image. Second, the constraint for the angle θ can vary from 0 to 180 degrees. Note that the reason to choose the angle θ as a parameter here instead of choosing the ratio of v y /v x, is that the ratio can vary from negative infinity to positive infinity. The size of possible values, therefore, will make the parameter selection more time-consuming and impractical. Third, the upper bound of the travel distance * v T canbelessthanthe dimension of the image. Take the image with m by n dimension as an example, the range of the travel distance would vary from 0 to m 2 + n Experimental Results. The comparisons between our proposed method and existing approaches, which includes laplacian filtering, unsharp masking, and Katsaggelos and Lay s work [6], maximum likelihood blind convolution, are conducted by Fujitsu camera with image resolution. Figure 2 shows comparisons over 64 images with different focal lengths ranging from 40mm to 110mm in an outdoor environment. In general, our method outperforms existing methods in terms of the sharpness index. Figure 3 shows one example in the database. We can see our proposed method outperforms existing approaches in terms of objective and adjective evaluations. For the adjective evaluation, we can clearly recognize both license plates in our proposed method shown in Figure 3(e), even though we cause some artificial noises in both images. For the objective evaluation, our proposed method has the highest sharpness index. Figure 2. Comparisons between our and existing methods over different focal lengths

5 ICIC EXPRESS LETTERS, VOL.2, NO.4, Conclusions. In this work, we presented a PSF-based approach considering fundamental characteristics of linear motion and out-of-focus blur based on geometric optics to restore coexisting motion and Out-of-Focus blurred images with an automatic parameter selection approach, where a sharpness measure is employed as a cost function to guide the parameter selection process. To verify the effectiveness of our proposed approach, we compare our approach with existing de-blur approaches including laplacian filtering, unsharp masking, and maximum likelihood blind convolution. Experimental results show our proposed approach outperforms existing de-blur approaches in terms of objective and adjective evaluations. In addition, our proposedmodelcanbeusedinthecurrentlydeployed camera surveillance systems without changing their hardware configurations. For our future works, we want to extend our current algorithm to be capable of handling nonlinear motion blur. Meanwhile, we need to reduce the computational time when searching for optimal parameters. (a) (b) (c) (d) (e) Figure 3. The comparisons between our proposed method and existing methods for Example 1. (a) The acquired image from the camera. Its sharpness index is (b) Laplacian filtering method. Its sharpness index is 0.3. (c) Unsharp masking method. Its sharpness index is (d) Maximum likelihood blind convolution. Its sharpness index is (e) Our proposed method. Its sharpness index is Acknowledgment. The authors thank reviewers for their helpful comments and suggestions, which have improved the presentation. REFERENCES [1] M. Ben-Ezra and S. K. Nayar, Motion deblurring using hybrid imaging, Proc. of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, USA, [2] C.-H. Chen, Y. Yao, D. Page, B. Abidi, A. Koschan and M. Abidi, Objective image quality evaluation for JPEG, JPEG 2000, Vidware, IEEE Pacific-Rim Symposium on Image and Video Technology, [3] R. Fabian and D. Malah, Robust identification of motion and out-of-focus blur parameters from blurred and noisy images, Graphical Models and Image Processing, vol.53, no.5, pp , [4] P. Favaro, M. Burger and S. Soatto, Scene and motion reconstruction from defocused and motionblured images via anisotropic diffusion, European Conference Computer Vision, [5] T. Hamamoto and K. Aizawa, A computational image sensor with adaptive pixel-based integration time, IEEE Journal of Solid-State Circuits, 2001.

6 414 C.-H.CHEN,T.Y.CHIEN,W.-C.YANG,J.-M.LIANDC.-Y.WEN [6] K. Katsaggelos and K. T. Lay, Maximum likelihood blur identification and image restoration using the EM algorithm, IEEE Transaction on Signal Processing, vol.39, no.3, [7] X. Liu and A. E. Gamal, Simultaneous image formation and motion blur restoration via multiple capture, IEEE International Conference on Acoustics, Speech and Signal Process, [8] J.Lee,Y.Yoo,J.Shin and J.Paik,Hierarchical blur identification from severely out-of-focus Images, IEEE Pacific-Rim Symposium on Image and Video Technology, [9] T. Toriu and S. Nakajma, A method of calculating saliency of images and of optimizing efficient distribution of image windows, International Journal of Innovative Computing, Information and Control, vol.3, no.6(a), pp , 2007.

4 STUDY OF DEBLURRING TECHNIQUES FOR RESTORED MOTION BLURRED IMAGES

4 STUDY OF DEBLURRING TECHNIQUES FOR RESTORED MOTION BLURRED IMAGES Abstract: This paper attempts to undertake the study of deblurring techniques for Restored Motion Blurred Images by using: Wiener filter,