A STUDY ON THE PHOTO RESPONSE NON-UNIFORMITY NOISE PATTERN BASED IMAGE FORENSICS IN REAL-WORLD APPLICATIONS Yu Chen and Vrizlynn L. L. Thing Institute for Infocomm Research, 1 Fusionopolis Way, 138632, Singapore {ychen, vriz}@i2r.a-star.edu.sg ABSTRACT In this paper, we study Photo Response Non-Uniformity (PRNU) based image tampering detection methods and their applicability in real-world image tampering detection applications. Experiments using current PRNU based image forensic methods were conducted to evaluate the performance of existing methods in the realistic applications. The PRNU based forensic approach was tested on the images taken by cameras including current and popular DSLR/sub-DSLR models which have rarely been tested and reported. In our experiments, over 800 PRNU noise patterns are generated from 80 authentic photos and the biometric-like experiments have been conducted to study the similarities between intra-class and inter-class PRNU noise patterns. The analysis for the performance of current methods and the suggestions for the further research and development towards PRNU based image forensic approach are presented. Index Terms Photo response non-uniformity; Image forensics; Image tampering detection; Image identity 1. THE RESEARCH BACKGROUND With the progress of information technology, more and more of our activities are recorded in electronic formats, e.g. digital images, videos, and sound records. Among these formats, the digital image plays an important role as a popular information carrier. Thus, there is an increasing security concern on the integrity of images. The original photos may be tampered to serve as manipulated evidences to support an inexistent fact. Therefore, a robust image tampering detection or identity verification approach is thus in requests[1]. The expectation of an image tampering detection approach is to be able to verify the integrity of an image and detect the tampered region if the image is forged. The challenges faced when designing such an approach are 1) there are plenty of image forgery methods; and the design is expected to detect targeted images generated by different kinds of tampering processes. 2) There should be no restrictions on the formats of the targeted images, and the design is expected to be reliable for most of popularly used image formats. The PRNU based image integrity verification method can be considered as a promising candidate technique for developing such an image forgery detection system. Because the PRNU based algorithm utilizes the device/camera identity for image verification, it can work for different image tampering methods and for different image formats as well [3][4] [5] [6][7]. The fundamental design idea of the PRNU verification method is that each camera bears a unique noise pattern, like each person has unique fingerprints and iris patterns, and the PRNU noise patterns are identical in all of the photos taken by this camera. Thus, the integrity of an image can be verified if the following two conditions can be met: 1) the noise pattern can be extracted from the image/images and 2) the camera PRNU noise pattern has been obtained. We believe image identity verification may be possible by using the PRNU method because it is possible to obtain multiple photos from a target camera. Those images can be obtained from existing photos or by taking new photos with the camera. However, for further design and development regarding a practical image forensic approach, the PRNU based technique must be tested for its robustness against the lower noise levels of recent consumer digital cameras. The improved noise deduction schemes of the recent cameras may affect the performance of PRNU based methods. It would be very difficult to extract the reliable PRNU patterns from the cameras which embed more advanced noise control features. Few reported works have tested the PRNU based methods for the images/photos taken by current mid-end and high-end camera models. In this work, we conduct an investigation and evaluation which are supported by newly designed experiments to discover the feasibility and possible design directions of a practical PRNU based image forensics approach for current real world applications. In the rest of this paper, a review of the existing PRNU based methods is given in Section 2. In section 3, the conducted experiments are demonstrated. Analysis and sug-
gestions for further research and development regarding the PRNU based image tampering detection approach is presented in the last section. 2. THE REVIEW OF EXISTING PRNU NOISE PATTERN BASED IMAGE FORENSICS METHODS Some algorithms and approaches that have been proposed to extract the camera PRNU noise pattern with single or multiple images. Based on the reported results, the existing methods are robust and reliable in image integrity verification as well as tampered region detection for previously introduced lowend camera models. This section would give a brief review to the current approaches, and more details of this approach can be found in [3][8]. The PRNU pattern is one kind of Fixed Pattern Noise (FPN). The FPN is a constant pattern that identifies nonuniformity noises in an imaging device/camera. The uniqueness of FPN is due to the differences caused by the small manufactory variations or defects for each device. The FPN consists of dark signal non-uniformity (DSNU) which is the noise response pattern obtained without external illuminations and the PRNU which refers to the ratio of the illumination intensity on a pixel to the output signal. The DSNU is only present in images without external illuminations e.g. taking image with lenses covers on. It is difficult to obtain such images for some of image tampering detection cases, e.g. the camera is not accessible. Obviously, the PRNU based method is more applicable for most of realworld image forensic applications, as the pattern is present in most of images with external illuminations. An existing well recognized PRNU method is proposed by[8]. Here we give a brief introduction. The image with PRNU noise would be an output of the camera sensor to the input illumination. This output can be modeled as: I = I (0) + I (0) K + θ (1) I (0) is the noise-free response, the θ is a combination of independent random noises and the I (0) K is the PRNU term. A de-noised output Î(0) can be obtained by a de-noising filter, then, W = I Î(0) = IK + I (0) Î(0) + (I (0) I)K + θ = IK + ε (2) The noise ε is the sum of θ and two additional terms introduced by the de-noising filter. The image content is significantly deducted in the noise residual W, thus the PRNU pattern can be better computed from W than from original image I. The estimator of the PRNU factor K can be derived from N images from the camera, for k = 1, 2,...N as W k I k = K + ε k I k, W k = I k Î(0) k, Î(0) k = F (I k ) (3) Then, from N images, the likelihood of K factor can obtained as: N/2 log(2πσ 2 /(Ik)) 2 L(K) = (W k /I k K) 2 2σ 2 /(I k ) 2 (4) The ML estimate ˆK for the expected PRNU fact is obtained by taking partial derivatives of (5) as: ˆK = W k I k / (I k ) 2 (5) A matrix of K can be obtained as a PRNU noise pattern of a given picture. The filter they used to obtain the noise-less I (0) is a frequency selective filter [2]. Further research and studies have been conducted to improve the PRNU based methods and investigate the methods in various applications. The authors from [9] studied the PRNU based image forensics method on video identification, and from the reported results, the PRNU based method works well for the videos with low resolutions. The work in [11] discovered that the minimum average correlation energy (MACE) filter is better than the normalized cross correlations (NCC) filter in calculating the similarity scores for camcorder identification. In [10], the Block-matching and 3D (BM3D) filter is used to obtain de-noised image and thus to get the PRNU noise pattern. Their reported results show that better PRNU patterns can be generated by using BM3D than using the wavelet selective filter [2]. 3. EXPERIMENTS OF PRNU APPROACH The purpose of our experiments is to test the PRNU based method on the images with lower noise levels. Because the designs of imaging devices/cameras improve rapidly with only several months for a new generation of models to arrive, the inherited noise is getting less obvious and more difficult to be estimated. We need to find out if the PRNU noise patterns can be still detectable for the recent mid-end or high-end camera models, and how well the detection performance is. In our experiments, we adopt BM3D denoising filter[12] to obtain the denoised image I denoised. The BM3D filter is considered as one of most advanced image denoising filters [10] and is proven to outperform the wavelet denoising filter [2] which is applied in the most of previous works. To eliminate the artefacts given by the sensor design and the colour interpolation which are identical for each brand and
model, as suggested in [8], the noise pattern is zero-meaned by subtracting the mean value of each row from each pixel in the row and then subtracting the mean value of each column from each pixel in the column. N = M i=1 N i I ori i / M (Ii ori ) 2 (6) i=1 Multiple images from the same camera have been used to estimate a PRNU noise pattern [3]. Although there are some exiting methods to estimate the PRNU noise from a single image, using multiple images would get very reliable and accurate results. As in Equation (7), to obtain the PRNU noise pattern, the noise from each image contribute with different weights regarding the intensity of the pixel in the original image to the final estimation of PRNU pattern. We treat the three channels of images separately and add one channel as the summation of RGB channels. Thus, four PRNU patterns can be extracted for one camera. To calculate the similarity between each pair of PRNU patterns P a and P b, we use the cross-correlation corr(p a, P b ). The experiments are carried out with the relatively new generations of mid-end and high-end camera models including Canon EOS 60D, NIKON J1, NIKON D90, and SONY NEX-5N. The reason we chose these camera models for our experiments is that they are currently popular consumer digital cameras with advanced noise detection features and little information or experimental result has been analyzed or reported for the camera models like those. We also believe that if a practical and reliable image forensics approach is achievable; such an approach should be able to work stably for most of the current on the market camera models as well as the new models which can be expected in the near future. With the rapid improvement of techniques, the noise levels in the newly introduced low-end and mid-end camera models could be even better than some high-end models introduced years ago. Thus, if an image forensic algorithm lacks the capabilities required for some relatively new mid-end or high-end camera models, it cannot be expected to be reliable for upcoming camera models in practical applications for a reasonable period of time e.g. two or three years. As the definition of the high-end camera model includes those professional and rarely sold in the market ones, for a convincing evaluation, we chose those four models which are currently popular and were sold at reasonable prices. The cost for each of four models camera body with one or two lenses is less than US$1,000, as in early 12. We use four cameras of those four models in our experiments. photos with natural scenes were collected for each camera. The photos were taken by our colleagues who have confirmed that none of the collected photos has been postprocessed after being transmitted out from the cameras. Some example photos we use in our experiments are illustrated in Fig.1. We randomly chose 10 photos taken from each camera to generate the reference patterns and use the rest of the photos to generate our query patterns. To generate the regional PRNU noise patterns, we divided 35 non-overlapping regions with the same dimensions of 512x512 from the top left region with the target region size of 3584x2560 for each photo. Thus, each camera can have 35 PRNU reference patterns from 10 chosen images and there are 140 reference patterns in total in our study. To illustrate the effects of image quantity to the PRNU pattern estimation, two sets of query patterns have been generated. Each of the first set of query patterns is obtained by using a pair of photos from the same camera. Thus, for each camera, there are 175 query patterns and 700 query patterns in total. Each of the second set of query cases is generated by using 5 photos from the same camera. Thus, there are 70 query patterns for each camera and we have 280 query cases in the experiment. The experiments are designed to test each of query patterns against every reference pattern by measuring the similarity scores. For the first experiment which uses the set of 700 queries, there are 98000 (700 *140) comparisons for each of four channels, among those, 700 belong to intra-class comparisons and the rest 97300 refer to inter-class comparisons. For the second experiment which uses the set of the 280 queries, there are 390 (280 *140) comparisons in total for each of four channels, among those, 280 are intra-class comparisons and the rest 389 comparisons refer to inter-class. The distributions in percentages of the similarity scores of inter-class and intra-class for two sets of experiments are illustrated in the Fig.2. The four figures on the left side of Fig.2 illustrate the similarity distributions for the first experiment with each query pattern generated from 2 images, and the four figures on the right illustrate the second experiment with each query pattern generated from 5 images. The first three rows of figures refer to the red, green and blue channel respectively and the fourth row refer to the summation channel with each similarity score is the summation of the scores of RGB channels. The horizontal axis of each figure indicates different similarity scores calculated by cross correlation, and the vertical axis of each figure indicates the different distributions of the comparisons in percentage which is obtained by dividing the total number of comparisons with a certain score from each class by the total number of comparisons conducted in the same class. From Fig.2, it can be seen from both experiments, the similarity scores for a majority of inter-class (blue bins) occupy within the small similarity scores while the similarity scores for intra-class (red bins) expand for a wider range on the horizontal axis. The similarity distributions for the second experiment are better separated between inter-class and intra-class than the first experiment. This is because the PRNU noise pattern can be better estimated with more images, each of the
Table 1. The cameras used in the experiment, with corresponding model features Attributes Canon EOS 60D NIKON J1 SONY NEX-5N NIKON D90 Resolutions (as default) 5184 x 3456 3872 x 2592 4912 x 3264 4288 x 2848 Introduced in 10 11 11 08 Output formant Number of images Camera type DSLR Sub-DSLR Sub-DSLR DSLR Fig. 1. the example photos used in our experiments query patterns for the first experiment is generated from 2 images which is less than 5 images used for the second experiment. The ROC curves for each experiment are generated and shown in Fig.3 for a further analysis. In Fig.3, the four figures on the left side refer the first experiment and the other four figures on the right refer to the second experiment. The figures from the first to fourth row demonstrate the ROC curves for the red, green, blue and the summation channel, respectively. The red curves show the corresponding false acceptance rate (FAR) and the green curves refer to the corresponding false rejected rate (FRR). The horizontal axis from each figure in Fig.3 illustrates different thresholds on the similarity scores and the vertical axis refers to the corresponding FAR and FRR rates. It is shown clearly that with the increasing of the threshold on the similarity scores, the FAR rate drops, which means fewer comparisons from inter-class are classified as identical. Meanwhile, the FRR rate increases as more comparisons from the intra-class are classified as non-identical. To better illustrate the classification capability for the tested PRNU method, the equal error rate (EER) on each channel for each experiment is given in Table 2. In Table 2, it is clearly shown that the second experiment with better PRNU estimation yields better classification performance than the first experiment. In our experiments, the best result we got is from the second experiment which utilizes 5 images to estimate each query PRNU pattern. 4. CONCLUSION AND FUTURE WORKS The results from our experiments show that the PRNU based methods are able to provide a certain level of capability in terms of verifying the integrity of the photos, even in challenging cases when using photos taken by the current midend and high-end camera models as targets, and the number of images used to estimate the PRNU pattern is limited. The experimental results support the feasibility for further design and development of a PRNU based practical and reliable image tampering detection approach for real-world applications. The lowest EER rate we obtained is at the level of 10 2 at 0.0853 which is given by the summation channel of our second experiment. However, a reliable biometric approach such as finger print or iris recognition system usually yields an EER rate well below the level of 10 3. Increasing the quantity of images used for the PRNU pattern estimation would certainly improve the performance but would also make the approach less applicable for many realistic applications when the number of photos we can obtain from the target camera is limited. Considering that the consumer digital cameras are still improving fast and the images are expected to get more noiseless, we suggest not taking the PRNU based image forensic approach as a reliable and standalone image tampering detection system but as an integrated supporting module works with other image forensic approaches to increase the decision confidence and improve the robustness of the whole system. Regarding future work, we plan to work on further testing
Fig. 2. the distributions of similarity scores for first (left column of figures) and second experiment (right column)
Fig. 3. the ROC curves for the first experiment (left column of figures) and the second experiment (right column)
Table 2. the EER rates for the red, green, blue and summation of RGB channels from the first and second experiments Experiments Red channel Green channel Blue channel Summation RGB 1st experiment EER 0.1715 0.1980 0.1954 0.1746 1st experiment Threshold 0.0074 0.0072 0.0085 0.0219 2nd experiment EER 0.0860 0.0965 0.1013 0.0853 2nd experiment Threshold 0.00 0.0225 0.0225 0.0654 on PRNU based methods with involving more camera models from low-end to high-end and from recent to previous models for a more thorough investigation. Furthermore, we are working on improvements for the PRNU pattern estimation method to obtain better PRNU pattern without involving more images. 5. ACKNOWLEDGMENTS The authors would like to thank our I2R colleagues Mr. Wee- Yong Lim, Mr. Darell J. J. Tan and Mr. Jun-Wen Wong for sharing the photos used in our experiments. 6. REFERENCES [1] A.C., Popescuand and H., Farid: Exposing Digital Forgeries in Color Filter Array Interpolated Images. IEEE Transactions on Signal Processing, vol. 53(10), pp. 39483959, 05. [2] M. K. Mihcak, I. Kozintsev, and K. Ramchandran, Spatially adaptive statistical modelling of wavelet image coefficients and its application to denoising, in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing, Phoenix, AZ, Mar. 1999, vol. 6, pp. 32533256. [3] J. Luks, J. Fridrich, and M. Goljan, Digital camera identification from sensor pattern noise, IEEE Trans. Inf. Forensics Security, vol. 1, no. 2, pp. 5214, Jun. 06. [4] H. Gou, A. Swaminathan, and M. Wu, Robust scanner identification based on noise features, in Proc. SPIE, Electronic Imaging, Security, Steganography, and Watermarking of Multimedia Contents IX, San Jose, CA, Jan. 29Feb. 1, 07, vol. 6505, pp. 0S0T [5] N. Khanna, A. K. Mikkilineni, G. T. C. Chiu, J. P. Allebach, and E. J. Delp, III, Forensic classification of imaging sensor types, in Proc. SPIE, Electronic Imaging, Security, Steganography, Watermarking Multimedia Contents IX, San Jose, CA, Jan. 29Feb. 1, 07, vol. 6505, pp. 0U0V [6] B. Sankur, O. Celiktutan, and I. Avcibas, Blind identification of cell phone cameras, in Proc. SPIE, Electronic Imaging, Security, Steganography, Watermarking Multimedia Contents IX, San Jose, CA, Jan. 29Feb. 1, 07, vol. 6505, pp. 1H1I. [7] T. Gloe, E. Franz, and A. Winkler, Forensics for flatbed scanners, in Proc. SPIE, Electronic Imaging, Security, Steganography, Watermarking Multimedia Contents IX, San Jose, CA, Jan. 29Feb. 1, 07, vol. 6505, pp. 1I1J. [8] M. Chen, J. Fridrich, and M. Goljan, and J. Luk, Determining image originand integrity using sensor noise, IEEE Trans. Inform. Sec. Forensics, vol. 3, no.1, pp. 7490, Mar. 08 [9] V. H. Wiger and G. Zeno, Using Sensor Noise to Identify Low Resolution Compressed Videos from YouTube in Proc. Computational Forensics, Lecture Notes in Computer Science, Volume 5718. ISBN 978-3-642-035-3. Springer Berlin Heidelberg, 09, p. 104 [10] Chierchia,G.,Parrilli,S., Poggi,G., Sansone,C., Verdoliva, L., On the Influence of Denoising in PRNU based Forgery detection. Proceedings of the 2nd ACM workshop on Multimdia in Forensics,Security and Intelligence 10, pp.77-82 [11] D.K., Hyun, C.H. Choi, H.K., Lee, Camcorder Identification for Heavily Compressed Low Resolution Videos, Lecture Notes in Electrical Engineering, 12, Volume 114, Part 5, 695-701, DOI: 10.1007/978-94-007-2792-2-68 [12] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, Image denoising by sparse 3D transform-domain collaborative filtering, IEEE Trans. Image Process., vol. 16, no. 8, August 07.