Applying the Sensor Noise based Camera Identification Technique to Trace Origin of Digital Images in Forensic Science

FORENSIC SCIENCE JOURNAL SINCE 2002 Forensic Science Journal 2017;16(1):19-42 fsjournal.cpu.edu.tw DOI:10.6593/FSJ.2017.1601.03 Applying the Sensor Noise based Camera Identification Technique to Trace Origin of Digital Images in Forensic Science Wen-Chao Yang 1*, Ph.D. ; Long-Huang Tsai 1, B.S. ; Chung-Hao Chen 2, Ph.D. 1 Department of Forensic Science, National Central Police University, Taoyuan City, Taiwan 2 Electrical and Computer Engineering Department, Old Dominion University, Norfolk, VA, U.S.A. Received: August 28, 2017; Accepted: September 30, 2017. Abstract With the rapid development of digital technologies, cameras and video recorders play an important role in our life, such as cell phones, cameras, or vehicle recorders. As a result, the easier we can access to those imaging devices, the more likely a perpetrator can use it to commit a crime, such as the invasion of privacy. Therefore, in recent years digital images and videos often play an important role as an evidence for coming a crime in court. Due to the rapid development of cloud and network technologies, we can easily spread digital images and videos via email or social media. However, it is difficult for us to trace the origin of the digital image/video contents. In many crime events, such as spreading national secrecy or maliciously spreading privacy cases, the ability of determining the source of the digital image/ video contents is important. In this paper, an origin-tracing method based on sensor noise is proposed to tackle this issue. The experiment result shows that the classification accuracy is close to 100 percent in some cases. Keywords: digital image/video evidence, tracing origin, authentication, sensor noise Introduction Due to the rapid development of computer science and technology, the use of digital imaging devices has incredibly become popular. Digital imaging devices, such as cell phones, cameras, or vehicle recorders, have become commodity for everyone. It lets the offenders easily commit crimes, such as intrusion of privacy or cyber bullying using digital cameras or mobile phone cameras. In addition, due to the lack the appropriate method of tracing origin of digital images/videos, it is difficult for us to identify the original imaging device when the offenders use to spread digital images and videos using email or social network methods. As a result, it becomes a challenging problem to use those images/videos as a legitimate evidence in court. Camera model identification is a technique to determine the camera model used to acquire a digital image of unknown provenance. It is based on the * Corresponding author: Wen-Chao Yang, Department of Forensic Science, National Central Police University, Taoyuan City, Taiwan. E-mail: una135@mail.cpu.edu.tw

20 Forensic Science Journal 2017; 16(1) premise that images acquired by the same camera model share common characteristics. In this paper, we illustrate our proposed method based on sensor noise can trace the origin. More than 2000 photos acquired by six cameras and seven mobile phone cameras are used in our experiment to demonstrate the effectiveness of our proposed method. Related Works and Proposed Methods The researches of camera model identification can be divided into two categories based on whether embedded information is active or passive. For an active embedding method, it embeds the distinguishing characteristics in images (videos). Metadata analysis [1] and data hiding methods [2,3] belong to this. The Color Filter Array (CFA) interpolation artifacts [4,5], sensor dust [6], and PRNU [7-13] are considered as a passive detection method. For a metadata analysis, the JPEG or TIFF images in current digital cameras usually follow standards like TU-T T.81, JETF, EXIF, CIFF and DCF [1]. Thus, the resulting images can be used to describe imaging equipment, such as shooting conditions (Fig. 1) or GPS information (Fig. 2). However, there exists risk using metadata analysis to trace the source, because the metadata record of images is easy to be removed or modified, which has to carefully confirm the accuracy and integrity of the description. Fig. 1 A shooting condition example in EXIF file header [1].

Camera Identification Technique 21 Fig. 2 A GPS information example in EXIF file header [1]. The use of watermarking technology for imaging equipment to trace origin or verify the integrity is widely studied [2]. Blythe and Fridrich [3] proposed a lossless watermarking combined with photographer s iris information to verify the photographer and ensure the integrity of images. However, this technology is limited to patent rights and market demand, and are not widely applicable to all digital imaging devices. The work of passive detection methods, in contrast to active embedding methods, is recently developed. In 1999, Kurosawa et al. [7] first proposed that the nonuniformity or defects can be retained in the chargecoupled devices (CCDs) in the photosensitive element manufacturing process. It can be used like a fingerprint feature to identify the image of the lens in the study. This situation is as identical as bullet scratches allowing forensic analysts to match a bullet to a particular barrel or gun for tracing the gun owner. They used 9 video recorders which have four kinds of labels in the experiment. The results found that 8 of the video clips produced by the video camera can get the source of the lens. In addition, the image noise obtained by the experimental video recorder is different. In 2011, they [8] used the Bayesian framework to establish the approximate proportion of the ratio (likelihood ratio) based on the noise characteristics of the recorders. Lukas et al. conducted a series of studies on analysis of digital cameras based on PRNU in 2005 [9,10] and 2006 [11]. In [9] and [10], they used the Daubechies 8 wavelet (db8) filter to estimate noise and used the correlation coefficient for digital camera sources to make a comparison of 9 different digital cameras (see Table 1) and 2 file formats (TIFF and JPEG), the results are shown in Fig. 3 and Fig. 4.

22 Forensic Science Journal 2017; 16(1) Table 1 Cameras and properties used in experiments [9]. Camera brand Sensor Maximal resolution Image format Canon PowerShot A10 1/2.7-inch CCD 1280 960 JPEG Canon PowerShot G2 1/1.8-inch CCD 2272 1704 CRW, JPEG Canon PowerShot S40 1/1.8-inch CCD 2272 1704 CRW, JPEG Kodak DC 290 1792 1200 TIFF, JPEG Olympus Camedia C765 UZ-1 1/2.5-inch CCD 2288 1712 TIFF, JPEG Olympus Camedia C765 UZ-2 1/2.5-inch CCD 2288 1712 TIFF, JPEG Nikon D100 23.7 15.5mm Nikon DX CCD 3008 2000 NEF-RAW, TIFF, JPEG Sigma SD9 20.7 13.8mm CMOS-Foveon X3 2268 1512 X3F-RAW Olympus Camedia C3030 1/1.8-inch CCD 2048 1536 TIFF, JPEG Fig. 3 Noise correlation from Olympus C765 (left) TIFF and Olympus C3030 (right) JPEG images with 9 reference patterns [9]. Fig. 4 Noise correlation from Canon G2 (left) and Nikon D100 (right) TIFF images with 9 reference patterns [9].

Camera Identification Technique 23 In both [9] and [10], they divided the noise into two main components [11], the fixed pattern noise (FPN) caused by dark currents and the photoresponse nonuniformity noise (PRNU) caused by pixel nonuniformity primarily. FPN primarily refers to the pixel-to-pixel difference when the sensor array is not exposed to light and its relationship to the exposure time and temperature. PRNU is caused by the inhomogeneity of the response of the light-sensitive element itself. They also proposed the mathematical model of image acquisition, according to the digital camera traceability analysis, the formula is shown as follows: y ij = f ij ( x ij + η ij )+c ij + ε ij (1) where i, j denote the coordinates of the image, i = 1,, m, j = 1,, n, m n is the resolution image, y ij and x ij respectively denotes the output and input of the capture sensor, η ij denotes the shot noise, c ij denotes the dark current, ε ij denotes the additive random noise, and the factors f ij are typically close to 1. In [11], they used the wavelet filter proposed in [12] as the denoise filter, and used different Garmma correction coefficients and compression ratios to analyze the sensor pattern noise in 9 different digital cameras (shown as Table 1). The experimental result is shown in Table 2. In Table 2, t denotes the correlation coefficient threshold used to distinguish the origin of images. FAR denotes the false acceptance rate and FRR denotes the false rejection rate. Table 2 Experimental results with FAR=10-3 [11]. Processing Camera None Gamma 0.7 Gamma 1.4 t FRR t FRR t FRR Nikon 0.0449 4.68 10-3 0.0443 1.09 10-2 0.0435 6.33 10-3 C765-1 0.0170 3.79 10-4 0.0163 3.88 10-4 0.0172 3.85 10-4 C765-2 0.0080 5.75 10-11 0.0076 2.57 10-11 0.0081 2.83 10-10 G2 0.0297 2.31 10-4 0.0271 3.23 10-4 0.0313 4.78 10-5 S40 0.0322 1.42 10-4 0.0298 1.64 10-4 0.0343 1.02 10-4 Sigma 0.0063 2.73 10-4 0.0060 2.93 10-4 0.0064 2.76 10-4 Kodak 0.0097 1.14 10-11 0.0096 1.08 10-8 0.0094 3.73 10-13 C3030 0.0209 1.87 10-3 0.0216 1.58 10-3 0.0195 2.67 10-3 A10 0.0166 7.59 10-5 0.0162 4.71 10-5 0.0160 2.93 10-4 Processing Camera JPEG 90 JPEG 70 JPEG 50 t FRR t FRR t FRR Nikon 0.0225 3.71 10-3 0.0231 5.83 10-2 0.0210 1.63 10-1 C765-1 0.0122 5.36 10-6 0.0064 1.55 10-6 0.0060 1.17 10-4 C765-2 0.0061 0 0.0065 9.53 10-14 0.0065 2.14 10-6 G2 0.0097 8.99 10-11 0.0079 4.85 10-11 0.0076 5.13 10-4 S40 0.0133 3.96 10-11 0.0085 4.41 10-14 0.0083 9.48 10-5 Sigma 0.0050 3.44 10-6 0.0055 9.16 10-6 0.0059 6.57 10-5 Kodak 0.0107 2.27 10-9 0.0127 4.53 10-4 0.0131 4.65 10-3

24 Forensic Science Journal 2017; 16(1) In 2009, Khanna et al. [13] used Daubechies 8 wavelet (Db8) filter as the denoise filter to estimate the imaging sensor pattern noise in 10 different cameras (shown in Table 3). They also use different size coefficients and file format conditions (see Table 3) to analyze the imaging sensor pattern noise. The experimental results are shown in Fig. 5 and Fig. 6. Table 3 Devices with respective setups for experiment[13]. Device Brand CCD Sensor Sensor Resolution Max. Picture Size Image Format c1 Canon PowerShot SD200-1 1/2.5 inch 3.2 MP 2048 1536 JPEG c2 Canon PowerShot SD200-2 1/2.5 inch 3.2 MP 2048 1536 JPEG c3 Nikon Coolpix 7600 1/1.8 inch 7.1 MP 3072 2304 JPEG c4 Panasonic DMC-FZ20 1/2.5 inch 5 MP 2560 1920 JPEG/TIFF c5 Nikon Coolpix 4100 1/2.5 inch 4 MP 2288 1712 JPEG c6 Nokia 6630 (3G smartphone) 1280 960 JPEG c7 Olympus E-10 2/3 inch 4 MP 2240 1680 JPEG/TIFF c8 Olympus D-360L 1280 960 JPEG/TIFF c9 Panasonic Lumix DMC-FZ4-1 1/2.5 inch 4 MP 2304 1728 JPEG/TIFF c10 Panasonic Lumix DMC-FZ4-2 1/2.5 inch 4 MP 2304 1728 JPEG/TIFF Fig. 5 Identification of low resolution (1024*768) c1 Canon SD200-1 images [13]. Fig. 6 The correlation values for various denoised images from c2 Canon SD200-2 images: the original images (left), the denoise images (right) [13].

Camera Identification Technique 25 In this paper, we apply the camera identification technique based on sensor noise in forensic science and propose an origin-tracing method based on foregoing research. The mathematic model is proposed as following: Y i = X i +Φ i + C i + ε i, (2) where Y i denotes the i-th output image, X i denotes the light information inputs for sensor in the i-th image, Φ i denotes the sensor noise (includes PRNU) in the i-th image, C i denotes the dark current in the i-th image, ε i denotes the random noise in the i-th image. We extract the noise pattern template as follows: Y' i = f d ( Y i ), (3) Y ' i denotes the i-th output image after denoise processing, f d (.) denotes the denoise function. N i = Y' i - Y i = Φ i + C i + ε i, (4) where N i denotes the noise in the i-th image. Because the ε i denotes the random noise in the i-th image, we assume it as a Gaussian random noise with zero mean, E ( ε i ) =0. Therefore, we obtain the noise pattern template N. N= E ( N i )=E ( Y' i - Y i )=Φ+C. (5) Experimental Results This section presents the experimental result of the camera identification technique based on sensor noise in different imaging devices. Digital cameras and mobile phones are used to verify the performance of our proposed camera identification technique. The list of digital cameras in the forensic science laboratory in Central Police University is shown as Table 4. The list of mobile phones is shown as Table 5. The noise pattern analysis algorithms were developed using Matlab 7. The Pearson correlation measure is used to as an evaluation measure to trace origin of digital photos. The testing platform is Microsoft Windows 7, Intel Core Duo 1.66 GHz with 2 GB memory. Table 4 The list of digital cameras in the experiment. Cameras No. Label Serial Number D80-1 Nikon D80 Digital SLR Camera 6134673 D80-2 Nikon D80 Digital SLR Camera 6134956 D90-1 Nikon D90 Digital SLR Camera 8339895 D90-2 Nikon D90 Digital SLR Camera 8339516 D7000-1 Nikon D7000 Digital SLR Camera 8245720 D7000-2 Nikon D7000 Digital SLR Camera 8246338 Camera Lens Serial Number Nikon 28-85mm f/3.5-4.5 AF Nikkor 3199726 Table 5 The list of mobile phones in the experiment. Label ASUS ZenFone 2 Laser (ZE550KL) ASUS ZenFone 2 ZE551ML-1 ASUS ZenFone 2 ZE551ML-2 HTC butterfly X920D Oppo r7sf Sony Xperia Z5 Sony Xperia Z5P Serial Number GCAZCY004318KMK F9AZFG011679 F9AZFG015590 FA33CPN06653 25c0cec4 BH903M6L4Z CB5A29R5F7

26 Forensic Science Journal 2017; 16(1) Digital cameras Six DSLR cameras with 3 focal length types (28mm, 50mm, and 85mm of the same camera lens) are combined into 18 different experiment samples (shown as Table 6) used in this experiment. We use the same shooting conditions, auto-mode, ISO 400, and Large JPEG (FINE) store format, to take more than 200 pictures for each experimental sample. The themes includes portrait and landscape shots. Afterwards, those photos are randomly divided into two groups, one for training noise pattern template, and the other group is used to validate the performance of our proposed method. We use the Daubechies 8 wavelet (Db8) filter and the Wiener filter as the denoise filter to extract the sensor noises. The experimental results are given in Appendix A. Table 6 The list of samples for digital cameras. Cameras No. Focus Length Experiment Samples ID D80-1 28 mm D8028-1 D80-1 50 mm D8050-1 D80-1 85 mm D8085-1 D80-2 28 mm D8028-2 D80-2 50 mm D8050-2 D80-2 85 mm D8085-2 D90-1 28 mm D9028-1 D90-1 50 mm D9050-1 D90-1 85 mm D9085-1 D90-2 28 mm D9028-2 D90-2 50 mm D9050-2 D90-2 85 mm D9085-2 D7000-1 28 mm D700028-1 D7000-1 50 mm D700050-1 D7000-1 85 mm D700085-1 D7000-2 28 mm D700028-2 D7000-2 50 mm D700050-2 D7000-2 85 mm D700085-2 From the experimental result, we find out the Wiener filter is a better tool as a denoise filter for our proposed method to extract the sensor noise pattern. In addition, the focus length is also a factor to trace origin based on sensor noise pattern. In conclusion, according to the experimental result, the performance of the proposed is able to trace each origin based on sensor noise pattern with the threshold of 0.075 correlation coefficient value. Mobile phones As the foregoing description, 7 different mobile phones (shown as Table 5) are used in this experiment. We use the shooting conditions, auto-mode and JPEG file format, to take more than 80 pictures for each device. Those themes include portrait and landscape shots. Then, we use the Wiener filter as the denoise filter to

Camera Identification Technique 27 extract the sensor noises. The experimental results are given in Appendix B. From the experimental result, we find out the performance of our proposed is able to trace each origin in mobile phone cameras with the threshold of 0.05 correlation coefficient value. Moreover, from the experimental result (Fig. B-6 and Fig. B-7), we can see that the sensor noise patterns in Sony Xperia Z5 and Sony Xperia Z5P have the same classification characteristics. Conclusions In this paper, the noise extraction mathematic model of sensor noise based camera identification technique is proposed. We also apply the sensor noise based camera identification technique to trace origin of digital photos in forensic science. Six DSLR cameras and seven mobile phone cameras are used in the experiment. The experimental results are shown that the proposed method has a good performance to trace origin of the experimental photos. In other word, we can find out the linkage between the photo and its capture device using just photo content by the proposed method. In our future work, we will add large devices to make an experiment to verify the generality of the proposed method. In addition, we will extend the proposed method to video evidence. Acknowledgements The authors would like to thank the anonymous reviewers for their insightful comments and valuable suggestions. This work was supported by National Science Council, Taiwan (MOST 106-2218-E-015-001-). References 1. Cohen K. Digital still camera forensics. Small Scale Digital Device Forensics Journal 2007;1(1):1-8. 2. Cox I, Miller M, Bloom J, Fridrich J, Kalker T. Digital watermarking and steganography. Morgan Kaufmann Publisher, 2008. 3. Blythe P, Fridrich J. Secure digital camera. Digital Forensic Research Workshop, Baltimore, 11-13 Aug 2004. 4. Popescu AC, Farid H. Exposing digital forgeries in color filter array interpolated images. IEEE Transactions on Signal Processing 2005;53(10):3948-59. 5. Swaminathan A, Wu M, Liu KJ-R. Non-intrusive forensic analysis of visual sensors using output images. IEEE Transactions of Information Forensics and Security 2007;2(1):91-106. 6. Dirik AE, Sencar HT, Memon N. Digital single lens reflex camera identification from traces of sensor dust. IEEE Transactions on Information Forensics and Security 2008;3(3):539-52. 7. Kurosawa K, Kuroki K, Saitoh N. CCD fingerprint method-identification of a video camera from videotaped images. Proc ICIP, 24-28 October 1999. 8. Geradts ZJ, Bijhold J, Kieft M, Kurosawa K, Kuroki K, Saitoh N. Methods for identification of images acquired with digital cameras. Proc SPIE Enabling Technologies for Law Enforcement and Security, 4232:505-12, 21 February 2001. 9. Lukas J, Fridrich J, Goljan M. Determining digital image origin using sensor imperfections. Proc SPIE Electronic Imaging:249-60, San Jose, CA, 16-20 January 2005. 10. Lukas J, Fridrich J, Goljan M. Digital bullet scratches for images. Proc ICIP, Genova, Italy, 11-14 September 2005. 11. Lukas J, Fridrich J, Goljan M. Digital camera identification from sensor noise. IEEE Transactions on Information Security and Forensics 2006;1(2):205-14. 12. Mihcak MK, Kozintsev I, Ramchandran K. Spatially adaptive statistical modeling of wavelet image coefficients and its application to denoising. Proc IEEE Int. Conf. Acoustics, Speech, Signal Processing;6:3253-6, Phoenix, AZ, 15-19 March 1999. 13. Khanna N, Mikkilineni AK, Delp EJ. Forensic camera classification: verification of sensor pattern noise approach. Forensic Science Communications 2009;11(1).

28 Forensic Science Journal 2017; 16(1) Appendix A Fig. A-1 The tracing origin result of photos acquired by D8028-1 with the Db8 filter. Fig. A-2 The tracing origin result of photos acquired by D8028-1 with the Wiener filter. Fig. A-3 The tracing origin result of photos acquired by D8050-1 with the Db8 filter.

Camera Identification Technique 29 Fig. A-4 The tracing origin result of photos acquired by D8050-1 with the Wiener filter. Fig. A-5 The tracing origin result of photos acquired by D8085-1 with the Db8 filter. Fig. A-6 The tracing origin result of photos acquired by D8085-1 with the Wiener filter.

30 Forensic Science Journal 2017; 16(1) Fig. A-7 The tracing origin result of photos acquired by D8028-2 with the Db8 filter. Fig. A-8 The tracing origin result of photos acquired by D8028-2 with the Wiener filter. Fig. A-9 The tracing origin result of photos acquired by D8050-2 with the Db8 filter.

Camera Identification Technique 31 Fig. A-10 The tracing origin result of photos acquired by D8050-2 with the Wiener filter. Fig. A-11 The tracing origin result of photos acquired by D8085-2 with the Db8 filter. Fig. A-12 The tracing origin result of photos acquired by D8085-2 with the Wiener filter.

32 Forensic Science Journal 2017; 16(1) Fig. A-13 The tracing origin result of photos acquired by D9028-1 with the Db8 filter. Fig. A-14 The tracing origin result of photos acquired by D9028-1 with the Wiener filter. Fig. A-15 The tracing origin result of photos acquired by D9050-1 with the Db8 filter.

Camera Identification Technique 33 Fig. A-16 The tracing origin result of photos acquired by D9050-1 with the Wiener filter. Fig. A-17 The tracing origin result of photos acquired by D9085-1 with the Db8 filter. Fig. A-18 The tracing origin result of photos acquired by D9085-1 with the Wiener filter.

34 Forensic Science Journal 2017; 16(1) Fig. A-19 The tracing origin result of photos acquired by D9028-2 with the Db8 filter. Fig. A-20 The tracing origin result of photos acquired by D9028-2 with the Wiener filter. Fig. A-21 The tracing origin result of photos acquired by D9050-2 with the Db8 filter.

Camera Identification Technique 35 Fig. A-22 The tracing origin result of photos acquired by D9050-2 with the Wiener filter. Fig. A-23 The tracing origin result of photos acquired by D9085-2 with the Db8 filter. Fig. A-24 The tracing origin result of photos acquired by D9085-2 with the Wiener filter.

36 Forensic Science Journal 2017; 16(1) Fig. A-25 The tracing origin result of photos acquired by D700028-1 with the Db8 filter. Fig. A-26 The tracing origin result of photos acquired by D700028-1 with the Wiener filter. Fig. A-27 The tracing origin result of photos acquired by D700050-1 with the Db8 filter.

Camera Identification Technique 37 Fig. A-28 The tracing origin result of photos acquired by D700050-1 with the Wiener filter. Fig. A-29 The tracing origin result of photos acquired by D700085-1 with the Db8 filter. Fig. A-30 The tracing origin result of photos acquired by D700085-1 with the Wiener filter.

38 Forensic Science Journal 2017; 16(1) Fig. A-31 The tracing origin result of photos acquired by D700028-2 with the Db8 filter. Fig. A-32 The tracing origin result of photos acquired by D700028-2 with the Wiener filter. Fig. A-33 The tracing origin result of photos acquired by D700050-2 with the Db8 filter.

Camera Identification Technique 39 Fig. A-34 The tracing origin result of photos acquired by D700050-2 with the Wiener filter. Fig. A-35 The tracing origin result of photos acquired by D700085-2 with the Db8 filter. Fig. A-36 The tracing origin result of photos acquired by D700085-2 with the Wiener filter.

40 Forensic Science Journal 2017; 16(1) Appendix B Fig. B-1 The tracing origin result of photos acquired by ASUS Zenfone 2 ZE 550KL. Fig. B-2 The tracing origin result of photos acquired by ASUS Zenfone 2 ZE 551ML1. Fig. B-3 The tracing origin result of photos acquired by ASUS Zenfone 2 ZE 551ML2.

Camera Identification Technique 41 Fig. B-4 The tracing origin result of photos acquired by HTC Butterfly X920D. Fig. B-5 The tracing origin result of photos acquired by Oppo R7SF. Fig. B-6 The tracing origin result of photos acquired by Sony Xperia Z5.

42 Forensic Science Journal 2017; 16(1) Fig. B-7 The tracing origin result of photos acquired by Sony Xperia Z5P.