ity Multimedia Forensics and Security through Provenance Inference Chang-Tsun Li School of Computing and Mathematics Charles Sturt University Australia Department of Computer Science University of Warwick UK 1 1
Outline Device Fingerprints Multimedia Forensic Applications Source Device Verification Source Device Identification Common Source Inference Content Authentication Source-Oriented Image Clustering Conclusions Future Works 2 2
Why not Use Metadata - EXIF File Metadata is easily removable and replaceable. 3 3
Device Fingerprints Scene Demosaicking Post- Processing Photo Lens Color Filter Array Sensor Lens aberrations (accurate to device models) CFA + demosaicking artefacts (accurate to models) Quantisation table of JPEG (accurate to models) Sensor pattern noise (accurate to individual devices) 4 4
Sensor Pattern Noise (SPN) SPN is the invisible artifacts left in the images by the sensors of devices. SPN is mainly caused by manufacturing imperfection of silicon wafers and different sensitivity of pixels to light. Sensors made from the same silicon wafer produce unique SPN SPN can differentiate cameras of the same model. 5 5
SPN Extraction Lukáš et al s model for SPN extraction (IEEE TIFS 2006) SPN: n = I( i, j) - I' ( i, j) I ' = Weiner _ filter( I) I is the original image I is the low-pass filtered version of I SPN is the high-frequency component of the image. 6 6
Interference from Scene Details Scene details also contribute to the high-frequency components of images. natural images SPN clean SPN contaminated SPN 7 C.-T. Li, "Source Camera Identification Using Enhanced Sensor Pattern Noise," IEEE Trans. on Information Forensics and Security, 2010 7
Other Sources of Interference Periodical operation: e.g., JPEG, demosaicking Periodical artefacts before enhancement after enhancement X. Lin and C.-T. Li, "Preprocessing Reference Sensor Pattern Noise via Spectrum Equalization," IEEE Trans. on Information Forensics and Security, 2016 C.-T. Li and Y. Li, "Color-Decoupled Photo Response Non-Uniformity for Digital Image Forensics," IEEE Trans. on Circuits and Systems for Video Technology, 2012 8 8
Other Sources of Interference Filters used in SPN extraction SPN: n = I( i, j) - I' ( i, j) I ' = Weiner _ filter( I) X. Lin and C.-T. Li, "Enhancing Sensor Pattern Noise via Filtering Distortion Removal," IEEE Signal Processing Letter, 2016 9 9
Source Device Verification Task: Determine whether a given image is taken with a particular device based on device fingerprints? Similarity: Normalized Cross Correlation SPN extractor r( i, j) = ( n i n i - ni ) ( n - n i n j j - n - n j j ) =? SPN extractor 10 Sensor pattern noise Sensor pattern noise 10
Source Device Identification Task: Given an image and the reference SPNs of k cameras, identify the camera that has taken the image? 11 11
Common Source Inference Task: determine whether two images are taken by the same camera or not without possessing the camera? unavailable 12 12
Real-World Applications Sussex Police (UK) Linking child pornography to an offender s mobile phone Leading to a 9-year prison sentence using SPN in 2014 Guildford Crown Court (UK) Linking a set of voyeuristic videos in the disk of a spy camera to another video store in the defendant s mobile phone original produced by the defendant s spy camera Defendant pleading guilty for installing spy camera in June 2016 13 13
Content Authentication Reference fingerprint of Camera C Forged image Images by the same camera C fingerprint of the forged image Authentication Map True positives False positives False negatives 14 14
Source-Oriented Image Clustering Objective: each formed group contains only images taken by the same camera Significance: establishing relationship among images Delivered to INTERPOL (Lyon, France) unavailable Clustering Module Image Database 15 15
Novelty of Our Clustering Alg X. Lin and C.-T. Li, "Large-Scale Image Clustering based on Camera Fingerprint, IEEE Transactions on Information Forensics and Security, vol. 12, no. 4, pp. 793 808, April 2017 16 16
Conclusions Device fingerprint can facilitate Source Device Verification Source Device Identification Common Source Inference Content Authentication Source-Oriented Image Clustering Multimedia forensics through device fingerprint analysis is of great interest to law enforcement. The most promising fingerprint is SPN at the moments Many future works to be done 17 17
Future Works We have the following now: Lens aberrations Colour filter array (CFA) interpolation artefacts Camera Response Function (CRF) Quantisation table of JPEG compression Sensor pattern noise Any other modalities? Any way to fuse them? 18 18
Future Works Issues surrounding SPN: A compact representation of SPN is needed for fast search and clustering: SPN is as big as the host image SPN is removable: sensitive to compression, transcoding, blurring, etc. SPN is replaceable! 19 19