Multimedia Forensics

Transcription

1 Multimedia Forensics Using Mathematics and Machine Learning to Determine an Image's Source and Authenticity Matthew C. Stamm Multimedia & Information Security Lab (MISL) Department of Electrical and Computer Engineering Drexel University, Philadelphia, PA 1

2 Digital Multimedia Tampering Editing software can create perceptually realistic digital multimedia forgeries Original Image Copied Regions Highlighted 2

3 Multimedia Forensics Forensics can provide information security when the information source is not trusted Detect traces known as fingerprints left by processing and manipulation Forensic research conducted by MISL: Develop forgery detection algorithms Develop source identification approaches Create and study anti-forensic techniques 3

4 Who We Are PI - Matthew C. Stamm Ph.D. Students Belhassen Bayar Chen Chen Owen Mayer Xinwei Zhao Sponsors NSF DARPA DFBA ARO 4

5 Image Forgeries 5

6 Lateral Chromatic Aberration (LCA) 6

7 Lateral Chromatic Aberration (LCA) Refractive index of light dependent on wavelength Focal location of different colors are laterally offset 7

8 Lateral Chromatic Aberration (LCA) 8

9 Real World Example 9

10 Cut-and-Paste Forgery 10

11 Forgery Detection Obtain local LCA estimates Fit to global model Measure local deviation from model within window Detection criteria: Local estimates significantly differ from model 11

12 Image Source Attribution Origin of many digital images is unknown/unverified Camera Model 1 Metadata can be unreliable Easily altered Often missing Source Device?... Camera Model 2 Camera Model N 12

13 Multimedia Forensics Solution Image contains information about originating device Source Device Use multimedia forensic techniques to extract camera traces from image Fuse multiple forensic traces to reach decision 13

14 Camera Fingerprints Imaging device processing pipeline Common elements Camera model-specific implementation Each component leaves behind traces Use camera model-specific traces to identify image s origin 14

15 Camera Traces JPEG Header Info JPEG quantization tables Huffman code tables Image Size No error in obtaining traces Cannot uniquely identify camera JPEG Quantization Tables Demosaicing Traces Image Size Demosaicing Traces Uniquely identifies camera Noisy estimates result in identification errors Improve performance using hierarchical fusion of all camera traces 15

16 Method 1 Linear Filter Estimate Original algorithm proposed by Swaminathan, Wu, and Liu Model demosaicing algorithm as a linear filter Obtain least-squares estimate of demosaicing filter coefficients Use coefficients as features for a support vector machine classifier 16

17 Method 1 Our Improvements Problems Too computationally complex to use all pixels for estimate Only utilizes information from small image window Can t control accuracy/computational cost trade-off Mathematical analysis Stronger local pixel variance yields better estimates Best pixels for estimation lie on edges Our method: Search entire image for best N pixels to use for demosaicing filter estimation Allows accuracy-complexity trade-off to be balanced 17

18 Improved Accuracy-Complexity Trade-Off 18

19 Method 1 - Test and Validation Experimental evaluation using 71 camera models 295 images per camera model 20,945 images in total Results obtained using five-fold cross validation Experiment 1 SVM Classifier No JPEG header info Achieved 86.81% accuracy Experiment 2 Hierarchical Fusion w/ SVM Classifer Full framework (exploits all information) Achieved 99.96% accuracy 19

20 Method 2 Demosaicing Residuals CFA + - Baseline Demosaicing Algorithm Demosaicing Residual Co-Occurance Matrix 20

21 Method 2 - Full Architecture Demosaicing Trace Extraction Classification 21

22 Method 2 - Test and Validation Experiments performed using 65 camera models Examined 512 x 512 image patches Minimum 300 images per camera Minimum 2,400 patches per camera Used 90% of patches for training, 10% for testing Did not use JPEG header information / full framework Achieved 99.65% accuracy 22

23 Source Camera Model Identification Tool Software implementation of full framework Matlab C (MEX-file interface with Matlab) CUDA (GPU code) Individual modules for Feature extraction Classifier training Camera identification Graphical user interface 23

24 Contrast Enhancement Forensics Contrast enhancement is a common editing technique Used to alter lighting conditions Nonlinear pixel value mapping Identified contrast enhancement fingerprints Unaltered Contrast Enhanced Developed contrast enhancement detection technique Tested on 11 different contrast enhancement mappings Achieved P d of 99% with P fa of 3% or less 24

25 Manipulation Detection Editing fingerprints are hard to discover Many editing operations exist Can we automatically learn editing fingerprints? Our approach: Convolutional Neural Networks 25

26 Convolutional Neural Networks (CNNs) 26

27 Convolutional Neural Networks (CNNs) Existing CNNs learn image content Great for object recognition Bad for forensics Want to automatically learn manipulation traces Don t want to learn image content Our solution: Constrained Convolutional Layer 27

28 Constrained Convolutional Layer Learns prediction error filters for feature extraction How do we learn? Update weights through stochastic gradient descent Project back into feasible set 28

29 Manipulation Detection CNN Architecture Low level feature extraction Hierarchical feature learning Classification 29

30 Experimental results Trained to detect four different editing operations Achieved 99.10% detection accuracy New (unpublished) results are even better! 30

31 Multimedia Forensics Using Mathematics and Machine Learning to Determine an Image's Source and Authenticity Matthew C. Stamm misl.ece.drexel.edu Multimedia & Information Security Lab (MISL) Department of Electrical and Computer Engineering Drexel University, Philadelphia, PA 31