Efficient Estimation of CFA Pattern Configuration in Digital Camera Images

Transcription

1 Faculty of Computer Science Institute of Systems Architecture, Privacy and Data Security esearch roup Efficient Estimation of CFA Pattern Configuration in Digital Camera Images Electronic Imaging 2010 Media Forensics and Security II Matthias Kirchner Technische Universität Dresden San Jose, CA, 2010/01/18

2 CFA Interpolation typical digital cameras use only one CCD / CMOS sensor and a color filter array (CFA) to capture full-color images missing color information is estimated from surrounding genuine elements in the raw image B B B B AW image full-color image CFA interpolation / demosaicing Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 1 of 15

3 CFA Interpolation typical digital cameras use only one CCD / CMOS sensor and a color filter array (CFA) to capture full-color images missing color information is estimated from surrounding genuine elements in the raw image B B B B AW image full-color image demosaiced images exhibit specific inter-pixel correlation artifacts CFA interpolation / demosaicing Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 1 of 15

4 CFA Artifacts in Digital Image Forensics double compression make use of image statistics for identification of source device or detection of manipulations block artifacts copy & paste resampling artifacts image forensics sensor dust sensor noise lightning, shadows lens distortions CFA pattern color filter array (CFA) interpolation artifacts form an important class of device characteristics source identification different camera models use different interpolation procedures manipulation detection post-processing damages characteristic inter-pixel correlation pattern Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 2 of 15

5 CFA Artifacts in Digital Image Forensics double compression make use of image statistics for identification of source device or detection of manipulations block artifacts copy & paste resampling artifacts image forensics sensor dust sensor noise lightning, shadows lens distortions CFA pattern color filter array (CFA) interpolation artifacts form an important class of device characteristics source identification different camera models use different interpolation procedures manipulation detection post-processing damages characteristic inter-pixel correlation pattern applications in steganalysis or digital watermarking Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 2 of 15

6 Example Application [Popescu & Farid, 2005] periodic artifacts in the linear predictor residue (p-map) Dresden Palace, image is part of the Dresden Image Database [loe & Böhme, 2010]

7 Example Application [Popescu & Farid, 2005] periodic artifacts in the linear predictor residue (p-map) Dresden Palace, image is part of the Dresden Image Database [loe & Böhme, 2010]

8 Example Application [Popescu & Farid, 2005] periodic artifacts in the linear predictor residue (p-map) DFT(p-map) CFA peak Dresden Palace, image is part of the Dresden Image Database [loe & Böhme, 2010]

9 CFA Pattern Configuration early methods did not explicitly incorporate knowledge about the actual configuration of the CFA pattern [Popescu & Farid, 2005; Bayram et al, 2005] problem of periodic, but locally inconsistent inter-pixel correlation CFA configuration valuable both for source identification [Swaminathan et al., 2007] and manipulation detection [Dirik et al., 2009] generally a means to decrease the degrees of freedom in image forensics B B B B AW image full-color image CFA configuration B B B B CFA interpolation / demosaicing? forensic examination Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 4 of 15

10 [Swaminathan et al., 2007; Dirik et al., 2009] CFA Configuration Estimation in the Literature minimum re-interpolation error assumption x y C for CFA configuration C and the demosaicing function d d(x, C) Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 5 of 15

11 [Swaminathan et al., 2007; Dirik et al., 2009] CFA Configuration Estimation in the Literature minimum re-interpolation error assumption x y C d(x, C) for CFA configuration C and the demosaicing function d d 1 (y C, ) C 1 C2 C 3 C 4 Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 5 of 15

12 [Swaminathan et al., 2007; Dirik et al., 2009] CFA Configuration Estimation in the Literature minimum re-interpolation error assumption x y C d(x, C) for CFA configuration C and the demosaicing function d d d 1 (y C, ), C 1 C2 C 3 C 4 Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 5 of 15

13 [Swaminathan et al., 2007; Dirik et al., 2009] CFA Configuration Estimation in the Literature minimum re-interpolation error assumption x d(x, C) y C C = arg min for CFA configuration C and the demosaicing function d yc d d 1 (y C, ), C 1 C2 C 3 C 4 e Ci Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 5 of 15

14 [Swaminathan et al., 2007; Dirik et al., 2009] CFA Configuration Estimation in the Literature minimum re-interpolation error assumption x d(x, C) y C C = arg min for CFA configuration C and the demosaicing function d yc d d 1 (y C, ), C 1 C2 C 3 C 4 e Ci subsampling matrix S Ci as simple approximation of inverse demosaicing d 1 (y, ) = S Ci y Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 5 of 15

15 [Swaminathan et al., 2007; Dirik et al., 2009] CFA Configuration Estimation in the Literature minimum re-interpolation error assumption x d(x, C) y C C = arg min for CFA configuration C and the demosaicing function d yc d d 1 (y C, ), C 1 C2 C 3 C 4 e Ci subsampling matrix S Ci as simple approximation of inverse demosaicing d 1 (y, ) = S Ci y assume linear relationship between raw and interpolated pixels d(x, ) = H Ci x Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 5 of 15

16 An Alternative Approach assume that we actually know the genuine raw sensor output x d(x, C) y C C 1 C2 C 3 C 4 x C = arg min d 1 (y C, ) d 1 (y, ) = S Ci y subsampling matrix S Ci as simple approximation of inverse demosaicing re-interpolation for each possible configuration not necessary Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 6 of 15

17 CFA Pattern Synthesis [Kirchner & Böhme, 2009] basic idea find a possible sensor signal x such that following the linearity assumption this is an ordinary least squares (OLS) problem y C d( x, C) 2 min y C = H C x + ɛ x C = (H C H C) 1 H C y Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 7 of 15

18 CFA Pattern Synthesis [Kirchner & Böhme, 2009] basic idea find a possible sensor signal x such that following the linearity assumption this is an ordinary least squares (OLS) problem y C d( x, C) 2 min y C = H C x + ɛ x C = (H C H C) 1 H C y caveat for a N-pixel image, H s of dimension 3N N direct implementation of the OLS solution not tractable efficiency improvements H s typically sparse (finite filter support) and of regular structure (periodic CFA) H C 3N Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 7 of 15

19 CFA Pattern Synthesis [Kirchner & Böhme, 2009] basic idea find a possible sensor signal x such that following the linearity assumption this is an ordinary least squares (OLS) problem y C d( x, C) 2 min y C = H C x + ɛ x C = (H C H C) 1 H C y caveat for a N-pixel image, H s of dimension 3N N direct implementation of the OLS solution not tractable efficiency improvements H s typically sparse (finite filter support) and of regular structure (periodic CFA) analytical solution for the bilinear interpolation kernel H C 3N Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 7 of 15

20 Approximate Solution by considering an infinite image without border conditions, approximate solutions in terms of a channel-dependent fixed linear filter can be found [Kirchner & Böhme, 2009] 2 F () y () 3 6 x Ci S Ci (Fy) = S Ci 4F () y () 7 5 F (B) y (B) equivalent to the analytical solution for large enough filter kernels Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 8 of 15

21 Approximate Solution by considering an infinite image without border conditions, approximate solutions in terms of a channel-dependent fixed linear filter can be found [Kirchner & Böhme, 2009] 2 F () y () 3 6 x Ci S Ci (Fy) = S Ci 4F () y () 7 5 F (B) y (B) equivalent to the analytical solution for large enough filter kernels application to the CFA configuration estimation problem C = arg min x Ci S Ci y 2 process image with linear CFA synthesis filters sub-sample image and filtered image to CFA pattern calculate difference between both only one linear filtering operation Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 8 of 15

22 Approximate Solution by considering an infinite image without border conditions, approximate solutions in terms of a channel-dependent fixed linear filter can be found [Kirchner & Böhme, 2009] 2 F () y () 3 6 x Ci S Ci (Fy) = S Ci 4F () y () 7 5 F (B) y (B) equivalent to the analytical solution for large enough filter kernels application to the CFA configuration estimation problem C = arg min x Ci S Ci y 2 process image with linear CFA synthesis filters sub-sample image and filtered image to CFA pattern calculate difference between both only one linear filtering operation our assumptions: Bayer CFA pattern bilinear interpolation continuous solution Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 8 of 15

23 efinements to the Estimation Procedure CFA configuration can be best determined for the green channel elements (twice as much genuine sensor pixels) C () Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 9 of 15

24 efinements to the Estimation Procedure CFA configuration can be best determined for the green channel elements (twice as much genuine sensor pixels) two-stage approach: decision for the complete configuration conditional to the estimated green channel configuration C () C Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 9 of 15

25 efinements to the Estimation Procedure CFA configuration can be best determined for the green channel elements (twice as much genuine sensor pixels) two-stage approach: decision for the complete configuration conditional to the estimated green channel configuration locally large error terms can accumulate to overall wrong decision C () C Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 9 of 15

26 efinements to the Estimation Procedure CFA configuration can be best determined for the green channel elements (twice as much genuine sensor pixels) two-stage approach: decision for the complete configuration conditional to the estimated green channel configuration locally large error terms can accumulate to overall wrong decision block-based approach: majority voting over all non-overlapping 2 2 blocks count C () C Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 9 of 15

27 Experimental Setup test database derived from 1000 full-resolution digital camera images from the Dresden Image Database [loe & Böhme, 2010] JPE AW TIFF TIFF JPE QUAL1... JPE QUALn JPE QUAL1... JPE QUALn Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 10 of 15

28 Experimental Setup test database derived from 1000 full-resolution digital camera images from the Dresden Image Database [loe & Böhme, 2010] JPE each 1000 images AW 5 different camera models with combined JPE / AW output Nikon D200 (1) Nikon D70/s (each 1) Panasonic DMC-FZ750 (3) icoh X100 (2) TIFF TIFF JPE QUAL1... JPE QUALn JPE QUAL1... JPE QUALn Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 10 of 15

29 Experimental Setup test database derived from 1000 full-resolution digital camera images from the Dresden Image Database [loe & Böhme, 2010] JPE JPE QUAL1 TIFF... JPE QUALn each 1000 images AW TIFF JPE QUAL1... JPE QUALn 5 different camera models with combined JPE / AW output Nikon D200 (1) Nikon D70/s (each 1) Panasonic DMC-FZ750 (3) icoh X100 (2) AW images demosaiced using Adobe Lightroom and dcraw 4 interpolation procedures: bilinear, VN, AHD, L Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 10 of 15

30 Experimental Setup test database derived from 1000 full-resolution digital camera images from the Dresden Image Database [loe & Böhme, 2010] JPE JPE QUAL1 TIFF... JPE QUALn each 1000 images AW TIFF JPE QUAL1... JPE QUALn 5 different camera models with combined JPE / AW output Nikon D200 (1) Nikon D70/s (each 1) Panasonic DMC-FZ750 (3) icoh X100 (2) AW images demosaiced using Adobe Lightroom and dcraw 4 interpolation procedures: bilinear, VN, AHD, L JPE compression with varying quality factors after demosacing Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 10 of 15

31 round Truth in our Experiments experimental evaluation requires ground truth CFA configurations not explicitly known for the cameras in use EXIF data not necessarily contains this information sensor datasheets are unreliable (active vs. effective pixels) CFA pattern becomes visible in the raw data of almost blue / red scences raw data Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 11 of 15

32 round Truth in our Experiments experimental evaluation requires ground truth CFA configurations not explicitly known for the cameras in use EXIF data not necessarily contains this information sensor datasheets are unreliable (active vs. effective pixels) CFA pattern becomes visible in the raw data of almost blue / red scences raw data Lightroom images (and genuine camera images) are smaller than dcraw images synchronization by maximum cross-correlation over all possible crops of the larger image Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 11 of 15

33 Baseline esults Percentage of correctly determined configurations D200 D70 D70s FZ750 X100 overall C () C C () C C () C C () C C () C C () C bilinear interpolation e Ci (total) C (total) i C (block) i VN interpolation e Ci (total) (total) (block) AHD interpolation e Ci (total) C (total) i C (block) i Adobe Lightroom e Ci (total) (total) (block)

34 Baseline esults Percentage of correctly determined configurations D200 D70 D70s FZ750 X100 overall C () C C () C C () C C () C C () C C () C bilinear interpolation e Ci (total) C (total) i C (block) i e VN interpolation Ci (total) re-interpolation with bilinear kernel e Ci (total) [Dirik 95.1et 95.1 al., 2009] C (total) i ec d 1 (total) CFA 83.4 synthesis, global 94.2 decision i C (block) i e C d 1 (block) i CFA 98.2 synthesis, block 99.2 decision AHD interpolation e Ci (total) C (total) i C (block) i Adobe Lightroom e Ci (total) (total) (block)

35 Baseline esults Percentage of correctly determined configurations D200 D70 D70s FZ750 X100 overall C () C C () C C () C C () C C () C C () C bilinear interpolation e Ci (total) C (total) i C (block) i VN interpolation e Ci (total) (total) (block) AHD interpolation e Ci (total) C (total) i C (block) i Adobe Lightroom e Ci (total) (total) (block)

36 Baseline esults Percentage of correctly determined configurations D200 D70 D70s FZ750 X100 overall C () C C () C C () C C () C C () C C () C bilinear interpolation e Ci (total) C (total) i green channel configuration can be determined most reliably C (block) i VN interpolation e Ci (total) (total) (block) AHD interpolation e Ci (total) C (total) i C (block) i Adobe Lightroom e Ci (total) (total) (block)

37 Baseline esults Percentage of correctly determined configurations D200 D70 D70s FZ750 X100 overall C () C C () C C () C C () C C () C C () C bilinear interpolation e Ci (total) C (total) i C (block) i VN interpolation e Ci (total) (total) (block) AHD interpolation e Ci (total) C (total) i C (block) i Adobe Lightroom e Ci (total) (total) (block)

38 Baseline esults Percentage of correctly determined configurations D200 D70 D70s FZ750 X100 overall C () C C () C C () C C () C C () C C () C bilinear interpolation e Ci (total) C (total) i C (block) green 100channel configuration can 99.2be determined i VN interpolation most reliably e Ci (total) block-based CFA synthesis approach superior in C (total) i virtually all cases C (block) i AHD interpolation e Ci (total) C (total) i C (block) i Adobe Lightroom e Ci (total) (total) (block)

39 Baseline esults Percentage of correctly determined configurations D200 D70 D70s FZ750 X100 overall C () C C () C C () C C () C C () C C () C bilinear interpolation e Ci (total) C (total) i C (block) i VN interpolation e Ci (total) (total) (block) AHD interpolation e Ci (total) C (total) i C (block) i Adobe Lightroom e Ci (total) (total) (block)

40 Baseline esults Percentage of correctly determined configurations D200 D70 D70s FZ750 X100 overall C () C C () C C () C C () C C () C C () C bilinear interpolation e Ci (total) C (total) i C (block) green 100channel configuration can 99.2be determined i VN interpolation most reliably e Ci (total) block-based CFA synthesis approach superior in C (total) i virtually all cases C (block) i reliability depends 98.2 to 98.2some 99.2 extent 99.2 on99.8 the99.8 source AHD interpolation of the image e Ci (total) C (total) i C (block) i Adobe Lightroom e Ci (total) (total) (block)

41 Influence of Image Size analysis of smaller image blocks of particular interest for the manipulation detection percentage of correctly determined configurations for all blocks of all images (CFA synthesis, block based) D200 D70 D70s FZ750 X100 overall C () C C () C C () C C () C C () C C () C AHD interpolation Adobe Lightroom Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 13 of 15

42 Influence of Image Size analysis of smaller image blocks of particular interest for the manipulation detection percentage of correctly determined configurations for all blocks of all images (CFA synthesis, block based) D200 D70 D70s FZ750 X100 overall C () C C () C C () C C () C C () C C () C AHD interpolation Adobe Lightroom configuration of Adobe Lightroom images is particularly harder to determine for smaller block sizes local, signal-adaptive post-processing? Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 13 of 15

43 JPE Post-compression green channel configuration C () 100 correct configuration [%] bilinear VN AHD Lightroom * JPE quality green channel configuration can be determined relatively reliable for JPE qualities as low as 90 Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 14 of 15

44 JPE Post-compression green channel configuration C () complete configuration C correct configuration [%] bilinear VN AHD Lightroom correct configuration [%] bilinear VN AHD Lightroom * JPE quality * JPE quality green channel configuration can be determined relatively reliable for JPE qualities as low as 90 complete configuration estimation is more vulnerable to JPE Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 14 of 15

45 Concluding emarks CFA pattern configuration is valuable additional knowledge in the forensic examination of digital camera images in this study: efficient method to determine the CFA pattern approximate solution to the CFA synthesis problem two-stage, block-based approach requires only 1 linear filtering step per image promising results despite the overly simplistic assumptions Limitations strong post-processing and JPE compression hamper a reliable identifictation Future work extend the CFA synthesis method to more sophisticated demosaicing procedures separate filter coefficients for horizontal / vertical edges allow for neutral decision (CFA pattern not known) Kirchner Efficient Estimation of CFA Pattern Configuration in Digital Camera Images slide 15 of 15

46 Faculty of Computer Science Institute of Systems Architecture, Privacy and Data Security esearch roup Thanks for your attention Questions? Matthias Kirchner Technische Universität Dresden Matthias Kirchner gratefully receives a doctorate scholarship from Deutsche Telekom Stiftung, Bonn, ermany.