Pattern Recognition in Blur Motion Noisy Images using Methods for Response Integration in Ensemble Neural Networks M. Lopez 1, 2 P. Melin 2 O. Castillo 2 1 PhD Student of Computer Science in the Universidad Autonoma de Baja California, Tijuana, B. C., México. 2 Computer Science in the Graduate Division Tijuana, Institute of Technology Tijuana, B. C., Mexico, Email: danym23@aol.com, pmelin@tectijuana.mx, ocastillo@tectijuana.mx Abstract Linear Blur Motion is one of the most common degradation functions that corrupt images. Since 1976 many researchers have tried to estimate blur motion parameters and this problem can be solved for noise free images but in the case of noisy images this can be done when the image SNR is low. In this paper, we consider pattern recognition with ensemble neural networks for the case of fingerprints; we propose the use of fuzzy methods for Response Integration in Ensemble Neural Networks for blur motion noisy images. An ensemble neural network of three modules is used; each module is a local expert on person recognition based on a biometric measure (the fingerprints). The Response Integration method of the ensemble neural networks has the goal of combining the responses of the modules to improve the recognition rate of the individual modules when the SNR rate blur motion signal increases to a high level. Keywords Pattern Recognition, Ensemble Neural Networks, Logic, Ratio SNR. 1 Introduction Blur Motion occurs when there is relative motion between the camera and the object being captured [1]. When a changing scene is observed by a camera, all the classical algorithms assume that is possible to take pictures every t instant, which means that every picture is taken with a dt 0 exposure time. If that is not the case, then the exposure time (dt = T) is large enough that different points in the scene are moving far enough and consequently their corresponding projections on the image plane travel several pixels. Therefore, during the capture process of an image, at any single image point, a certain number of scene points is projected during the exposure time, each one contributing to the final brightness of the image point; this effect is shown in figure 1. More formally, during the exposure time T in front of the pixel P i,j we could assume that they pass k scene points with brightness (C 1.C k ) respectively, then the resulting brightness value for pixel P i,j is given in equation (1), in the case it continues movement the summation is replaced by integration. This holds in general for every pixel that is moving points in the scene. It is clear that the blurring of the image exists only across the direction of the motion; this one dimensional blur is called Blur Motion. The Result of blur motion is shown in figure 1b where an image consistent of different value pixels is shown in figure 1a and then the blurred image is shown in figure 1b. Figure 1: a) Original Image without noise, b) Blur Motion with 20 pixels distance d and 50 degrees angle. The blur motion can be described mathematically as the result of a linear filter b(x, y) = i(x, y)*h(x, y) where i is the theoreticalcal image taken with an exposure time T e =0, b is the real blurred image and h the point spread function (PSF). Given an angle = and the length d = V o x T e, which is the number of scene points that affect a specific pixel, the point spread function of blur motion is given in equation (2). In practical terms, this mean computing accurate estimates for the two parameters of the blur motion PSF, namely the length, d, and the angle,. From these quantities, the relative velocity at this point can be easily recovered knowing the exposure time. In this paper for the experimental results we use fingerprint images and adding different levels of blur motion noise from 10 to 90 pixels in distance d and the angle is zero. The measurement for noise in an image is the signal-to-noise ratio, SNR. The SNR measures the relative strength of the signal in a blurred and noisy image to the strength of the signal in a blurred image without noise. A SNR of 16 db is a low noise level, while a SNR of 4 db is a high noise level [1]. 2 Noise and Registration Error 2.1 Signal-to-noise ratio Noise is present in all digital imaging systems due to a number of sources such as photon shot noise, readout noise, dark current noise and quantization noise. While some sources can be effectively suppressed, such as the dark 809
current noise by cooling and the quantization noise by using more bits, others not. Photon shot noise, for example, is unavoidable due to the particle nature of light. Readout noise increases with a higher readout rate, which is desirable in high-speed cameras. In equation 3 the calculus of the Variance and Standard deviation of the image are shown [2]. = the Standard deviation of the Fingerprint Image Original. = the variance between the Fingerprint Image Original and The Fingerprint Image with Blur Motion Noise., is the noise to the Fingerprint Image Original. = blur motion noise 10 to 90 distance pixels., is the Fingerprint Image with Blur Motion Noise. Table I: The SNR in db of the Fingerprint Images. SNR in db Blur Motion Distance Pixels Person N Fingerprint Image 10 20 30 40 50 60 70 80 90 P1 10.9 7.57 8.08 7.65 7.43 7.28 7.11 6.99 6.89 P2 8.36 8.02 7.52 7.38 7.16 7.01 6.89 6.78 6.68 P3 8.01 7.02 6,76 6.56 6.44 6.34 6.21 6.10 5.98 P4 7.67 5.75 5.57 5.46 5.26 5.17 5.07 5.01 4.93 P5 11.6 10.62 10.07 9.77 9.47 9.28 9.08 8.88 8.71 P6 7.29 7.28 6.88 6.75 6.58 6.46 6.33 6.20 6.09 P7 8.21 6.11 6.26 6.18 5.93 5.85 5.79 5.67 5.57 P8 10.6 10.07 9.4 9.08 8.8 8.56 8.47 8.15 7.97 P9 12.7 12 11.65 11.35 11.7 10.9 10.81 10.6 10.4 P10 8.94 6.43 6 5.79 5.52 5.33 5.24 5.14 5.03 In figure 3 we show the plot of the SNR of the Person Fingerprint with the blur motion level noise from 10 to 90 distance pixels. The Signal-to-Noise Ratio (SNR) of an image I is then computed as: The combined effect of these noise sources is often modeled from two images of the same scene captured under the same degradation is shown in figure 2. Figure 3: SNR with blur motion noise. Figure 2: Image degradation by blur and additive noise. 2.2 Calculus of the SNR for Fingerprint Images In this paper we use fingerprint images from the FCV2000 database of the Biometric System Laboratory at the University of Bologna [3, 4]. The image size is 300 pixels wide and 300 pixels high with a resolution of 500 ppi, and of a gray scale representation. The fingerprints were acquired by using a low-cost optical sensor; up to four fingers were collected from each volunteer (forefinger and middle finger of both the hands). The database is 10 fingers wide (w) and 8 impressions per finger deep (d) (80 fingerprints in all); the acquired fingerprints were manually analyzed to assure that the maximum rotation is approximately in the range [-15, 15 ] and that each pair of impressions of the same finger have a non-null overlapping area. We added Blur Motion Level Noise from 10 to 90 distance pixels to all 80 fingerprint images, based on equations (3) and (4) the calculus of the SNR in decibels for the Fingerprint Image of each of the person obtained is shown in table 1. 3 Ensemble Neural Networks Architecture 3.1 ANN s Architecture The Ensemble ANN s architecture consists of three main modules [5, 6], in which each of them in turn consists of a set of neural networks trained with the same data (fingerprints image), If we now using different parameters for each module of the Ensemble Neural Networks, for module 1 the used parameters are learning rate =.001, Goal Error=.001, for the module 2 learning rate =.0001, Goal Error=.0001, for the module 3 the learning rate =.0001, and Goal Error=.00001, using 2 hidden layers, 36 neurons in the first layer and 17 neurons in the second layer for each module of the Ensemble Neural Network and is shown in figure 4. Figure 4: Ensemble Neural Networks with blur Motion Level Noise Added. 810
The procedure to perform the tests was first to train the modules of the ensemble neural networks with the databases of the fingerprints without noise, in total using 8 fingerprints of 10 people, that is 80 fingerprints in total, until being able to find which architecture of the ensemble neural network responded better to arrive to the desired error. We Added Blur Motion Level Noise from 10 to 90 distance pixels to the Fingerprint Input, to obtain a blur motion image that went into each of three modules ensemble neural networks. The output of each of the ensemble neural networks was obtained response integration with type-1 or type-2 fuzzy logic, and finally with the decision module obtained the output fingerprint. 3.2 Response Integration with Type-1 Logic We also used a Type-1 Inference System as method of response integration of the ensemble neural network output, considering as input three linguistic variables, Activation Low, Activation Medium, and Activation High, and one output linguistic variable, Winning Activation of the three modules. We show below one of the rules for the fuzzy inference system. If (Module1 is ActMod1Low) and (Module2 is ActMod2Medium) and (Module3 is ActMod3Medium) then (Winner Module is Module3) Three membership functions were used for each linguistic variable (input and output) of the triangular type, to be managed in a range from 0 to 1. We show in figure 5 the membership functions designed using the editor of the fuzzy logic toolbox of MATLAB [14]. managed in a range from 0 to 1. We show in figure 6 the membership functions designed using the editor of IT2FUZZY fuzzy logic toolbox [12, 13]. Figure 6: The Input Membership function and The Output Membership function of Type-2 Logic. 4 Simulation Results 4.1 Response Integration with Type-1 Logic Once the Ensemble Neural Network is trained, the fuzzy inference system integrates the outputs of the modules. We used the same 80 persons images to which we had applied different levels of noise with blur motion, the type-1 inference systems give an output for the stage of the final decision, and we show the result if the fingerprint input was recognized. We show in figure 7 the simulation results. Figure 5: The Input Membership function and The Output Membership function of Logic Type-1. Once the Ensemble Neural Network is trained, the fuzzy inference system integrates the outputs of the modules. We used the same 80 people s images to which we had applied noise with blur motion, and the type-1 fuzzy inference system gives an answer for the stage of the final decision. 3.3 Response Integration with Type-2 Logic We used a Type-2 Inference System as method of response integration of the ensemble neural network output, considering as input three linguistic variables, Activation Low, Activation Medium, and Activation High, and one output linguistic variable, Winning Activation of the three modules, used the same rules of the type-1 fuzzy logic. Three membership functions were used for each linguistic variable of the input and output of the Gaussian type Figure 7: Simulation results for the fingerprints using the type-1 Inference System (blur motion 50 distance pixels). 4.2 Response Integration with Type-2 Logic In the same way when the Ensemble Neural Network is trained, the fuzzy inference system integrates the outputs of the modules. We used the same 80 persons images to which we had applied different levels of noise with blur motion, the type-2 inference systems give an output for the stage of the final decision, and show the result if the fingerprint input was recognized. We show in figure 8 the simulation results. Figure 8: Simulation results for the fingerprints using the type-2 Inference System (blur motion 50 distance pixels). 811
4.3 Comparison Pattern Recognition Blur Motion Noisy Images using Methods for Response Integration in Ensemble Neural Networks The Response Integration method of the ensemble neural networks has the goal of combining the responses of the modules to improve the recognition rate of the individual modules when the blur motion signal increases to high level 10 to 90 distance pixels. In Table 2 we show a Comparison between Response Integration with Type-2 and Type-1 logic when the blur motion level noise increases and the SNR in db decrease. noise increases and the down side the SNR in db decrease for each of the Person Fingerprint Images. Table II: Comparison Response Integration Type-2 and Type-1 Logic with Blur Motion Level Noise and the SNR in db of the Fingerprint Images. # Person Fingerprint SNR P1 db SNR P2 db SNR P3 db SNR P4 db SNR P5 db SNR P6 db SNR P7 db SNR P8 db SNR P9 db SNR P10 db Blur Motion Level Noise 10 20 30 40 50 60 70 80 90 10.97 7.75 8.08 7.65 7.43 7.28 7.11 6.99 6.89 100 87.5 87.5 87.5 87.5 62,5 62.5 62.5 62.5 87.5 87.5 87.5 87.5 75 62.5 62.5 62.5 62.5 8.33 8.02 7.52 7.38 7.16 7.01 6.89 6.78 6.68 100 100 87.5 100 87.5 75 62.5 50 37.5 100 100 87.5 100 87.5 75 75 50 50 8.01 7.02 6.76 6.56 6.44 6.34 6.21 6.10 5,98 100 100 87.5 87.5 62.5 62.5 37.5 25 12.5 100 100 87.5 75 75 50 0 0 0 7.67 5.75 5.57 5.46 5.26 5.17 5.075 5.01 4.94 100 100 87.5 87.5 75 75 62.5 62.5 25 87.5 87.5 87.5 87.5 75 75 62.5 62.5 37.5 11.65 10.62 10.08 9.77 9.47 9.28 9.08 8.89 8.71 100 100 100 100 100 87.5 87.5 75 50 100 100 100 100 100 87.5 75 75 50 7.29 7.28 6.88 6.75 6.58 6.46 6.33 6.21 6.1 100 100 100 100 100 100 100 87.5 75 100 100 100 100 100 100 100 87.5 75 8.21 6.11 6.26 6.18 5.93 5.85 5.79 5.67 5.57 100 100 100 87.5 87.5 87.5 62.5 50 37.5 100 100 100 75 75 87.5 62.5 50 37.5 10.64 10.07 9.40 9.09 8.8 8.56 8.47 8.15 7.97 100 100 100 87.5 87.5 87.5 62.5 50 37.5 100 100 100 75 75 87.5 62.5 50 37.5 12.76 12 11.65 11.35 11.7 10.9 10.81 10.6 10.42 8.21 6.11 6.26 6.18 5.93 5.85 5.79 5.67 5.57 In Figure 9 we show in the up side the response integration of the type-2 and type-1 fuzzy system when the blur motion 812
813
[3] D. Maltoni, D. Maio, A.K. Jain, S. Prabhakar, The full FVC2000 and FVC2002 databases are available in the DVD included in: Handbook of Fingerprint Recognition, Springer, New York, 2003. [4] D. Maio, D. Maltoni, R. Cappelli, J. L. Wayman and A. K. Jain, "FVC2004: Third Fingerprint Verification Competition", Proc. International Conference on Biometric Authentication (ICBA), pp. 1-7, Hong Kong, July 2004. [5] M. Lopez, P. Melin: Response integration in Ensemble Neural Networks using interval type-2 logic. IJCNN 2008: 1503-1508 [6] O. Mendoza, P. Melin, G. Licea, Modular Neural Networks and Type- 2 Logic for Face Recognition, In Marek Reformat, editor, Proceedings of NAFIPS 2007, Number 1, pages CD Rom, San Diego, June 2007. [7] J. Urias, D. Solano,.M. Soto, M. Lopez, P. Melin, Type-2 Logic as a Method of Response Integration in Modular Neural Networks. IC-AI 2006: 584-590. [8] P. Melin, F. González, G. Martínez: Pattern Recognition Using Modular Neural Networks and Genetic Algorithms. IC-AI 2004: 77-83. [9] P. Melin, J. Urias, D. Solano, M. Soto, M. Lopez, O. Castillo: Voice Recognition with Neural Networks, Type-2 Logic and Genetic Algorithms. Engineering Letters 13(2): 108-116 (2006). Figure 9: Response Integration Type-2 and Type-1 logic with the blur Motion Level Noise 10 to 90 Distance Pixels and SNR in db of each 10 Person Fingerprint Image. 5 Conclusions Based on the experimental results, we can conclude that the behavior of the type-2 and type-1 fuzzy systems, as methods of response integration of ensemble neural networks for the fingerprints when added blur motion level noise increase and the SNR of the Fingerprint decrease, we obtain excellent results. The identification rates obtained are between 37.5% to 100%for response integration type-2 fuzzy logic and 0% to 100% for response integration with type-1 fuzzy logic. The difference between the type-1 fuzzy and type-2 fuzzy inference systems for response integration of ensemble neural networks is appreciated when we uses the blur motion level noise. We think that there is an advantage in using a type-2 fuzzy inference system to manage the uncertainty of the knowledge base in pattern recognition problems. Future work will include, testing with more kinds of noise, other methods of response integration, using feature extraction, and image compression, with the goal of improving the identification rate. Acknowledgment We would like to express our gratitude to CONACYT, Universidad Autonoma de Baja California and Tijuana Institute of Technology for the facilities and resources granted for the development of this research project. References [1] Felix Krahmer_, Youzuo Lin, Bonnie McAdoo, Katharine Ott, Jiakou Wang, David Widemannk, Brendt Wohlberg, Blind Image Deconvolution: Motion Blur Estimation, August 2006. [2] Tuan Q. Pham, Lucas J. van Vliet, Klamer Schutte, Influence of signal-to-noise ratio and point spread function on limits of super resolution, Image Processing, Algorithms and Systems IV, Vol. 5672, pp. 169-180, 2005. [10] P. Melin, A. Mancilla, M. Lopez, D. Solano, M. Soto, O. Castillo, Pattern Recognition for Industrial Security using the Sugeno Integral and Modular Neural Networks, WSC11 11th Online World Conference on Soft Computing in Industrial Applications September 18-October 6, 2006. [11] P. Melin and O. Castillo, 2005, Hybrid Intelligent Systems for Pattern Recognition using Soft Computing, ISBN 3-540-24121-3 Springer Berlin Heidelberg New York. [12] J. R. Castro, O. Castillo, P. Melin, L.G. Martinez, S. Escobar, I. Camacho, Building Inference Systems with Interval Type-2 Logic Toolbox Number 1 in Studies in Fuzziness and Soft Computing, 6, pages 53-62, Springer-Verlag, Germany, 1 edition, June 2007. [13] J. R. Castro, O. Castillo, P. Melin, An Interval Type-2 Logic Toolbox for Control Applications; In Proc. FUZZ-IEEE 2007. [14] MATLAB Trade Marks, 1994-2007 by The Math Works, Inc. [15] S. Farsiu, D. Robinson, M. Elad and P. Milanfar. Fast and robust multi-frame super-resolution. IEEE Trans. on Image Processing, 13(10):1327-1344, 2004. [16] R.C. Hardie, K.J. Barnard and E.E. Armstrong. Joint map registration and high-resolution image estimation using a sequence of undersampled images. IEEE Trans.on Image Processing, 6(12):1621 1633, 1997. [17] R. Neelamani, H. Choi and R. Baraniuk. Forward: Fourier-wavelet regularized deconvolution for Ill-Conditioned Systems. In IEEE Trans. Signal Processing, 52(2):418 433, 2004. [18] J. Biemond. Iterative methods for image deblurring. Proceedings of the IEEE, 78(5):856 883, 1990. [19] M.Ebrahimi Moghadam, M.Jamzad, Linear motion blur parameter estimation in noisy images using fuzzy sets and power spectrum, EUROSIP Journal on applied signal processing, Article number 68985, Volume 2007, pages 862 866.. [20] Y. Yitzhaky, I. Mor, A. Lantzman, and N. S. Kopeika. Direct method for restoration of motion blurred images. Journal of the Optical Society of America. A, Optics and image science, 15(6):1512 1519, 1998. 814