Sensor-level Privacy for Thermal Cameras

Transcription

1 Sensor-level Privacy for Thermal Cameras Francesco Pittaluga Aleksandar Zivkovic Sanjeev J. Koppal University of Florida

2 Imaging and Tracking People Surveillance Military Gaming IoT Mobile 2

3 Balancing Privacy and Utility Hospitals Schools Retirement homes Workplaces 3

4 Conventional Privacy Processing Camera Storage Computer Secure Data Scene Post-capture processing Lock data with cryptography Edit images computationally Boyle 2000, Sweeney 2002, Johnson et al. (IETF) 2003, Gross et al. 2009, Agrawal and Narayanan 2011, 4

5 Conventional Privacy Processing Camera Storage Computer Secure Data Scene Post-capture strategy has an inherent vulnerability 5

6 Privacy Preserving Computational Cameras Secure Data Scene Computational Camera Only capture light-field samples that are needed 6

7 Computational Camera: Three Approaches X Special Optics X New sensor X Pattern recognition in every frame Storage Computer Secure Data Scene Computational Camera Three Problems Limited Adoption Nelson et al. 2005,Winkler et al. 2014, Fernandez-Berni et al Zhang et al, 2014, Pittaluga and Koppal 2015 Chattopadhyay and Boult 2007, Winkler and Renner 2010, Narayanan and Mrityunjay

8 Our Idea: Sensor-level Privacy for Thermal Cameras Off the Shelf Optics Thermal sensor Reliable Physics Based Processing Storage Computer Secure Data Scene Computational Camera Our idea: use a thermal sensor (Wavelengths > 3µm) 8

9 Thermal Cameras are Coming Soon! FLIR One 80x60 ~$250 Melaxis 16x4 ~$75 FLIR A6751SC Thermal cameras are not exotic anymore Thermal face recognition works 9

10 Three Sensor Level Approaches Thermal Sensor Secure Data Scene Computational Camera 10

11 Three Sensor Level Approaches Thermal Sensor Digitization Noise Exposure 11

12 Three Sensor Level Approaches Thermal Sensor Digitization Noise Exposure 12

13 Humans are broadband Spectral Power Response vs Wavelength W m 2 sr μm 11 Human Spectral Response r(λ) (μm)) 13

14 Facial skin temperature reaches an equilibrium 37 Degrees (Celsius) Outside temperature known = T Celsius Facial skin temperature F (T, 37) 14

15 This mapping is known Skin Temperature C Ambient Temperature C Olesen and Parsons

16 Removing pixels in this range during digitization A-to-D 16-bit decoder Seq. Counter Random-Access Memory Pixel Readout Output Voltage Upper Voltage Lower Voltage Upper Bound Comparator Lower Bound Comparator AND NOT AND Masking measurements based on temp. range Address (18.0) Data(15.0) ASIC Modification 16

17 Digitization Result 17

18 Digitization Result 18

19 Three Sensor Level Approaches Thermal Sensor Digitization Noise Exposure 19

20 Adding noise to the bolometer Active Bolometer Cint VCC GFID Tunable Bias Voltages Vout GSK Vbus Blind Bolometer 20

21 The effect of bias voltages 0 V GSK 5 V 0 V GSK 5 V 0 V 0 V GFID GFID 5 V 5 V Exposure 5 Exposure 15 0 V GSK 5 V 0 V GSK 5 V 0 V 0 V GFID GFID 5 V Exposure 25 Exposure 75 5 V 21

22 Calibrating for noise and privacy 0 V 0 V GSK 5 V GFID 5 V Bias voltages with exposure set to 5 For a flat lambertian plane Occurrences σ = 14 graylevels Grayscale values Histogram of values for the highest standard deviation 22

23 Noise Result: Head Tracking 23

24 Three Sensor Level Approaches Thermal Sensor Digitization Noise Exposure 24

25 Temperature and radiant power Spectral Power Response vs Wavelength Pixel Number vs Radiant Power W m 2 sr μm 11 Camera sensitivity s(λ) I max ~ I max Human Spectral Response r(λ) r(λ) r(λ) ~ I min (μm) Φ cold Φ human Φ hot Φ = λ t λ hs λ r λ dλ 25

26 No capture region No Capture Spectral Power Response vs Wavelength Pixel Number vs Radiant Power W m 2 sr μm 11 Camera sensitivity s(λ) I max Human Spectral Response r(λ) Δ(λ) (μm) Φ min Φ max Φ min = λ t λ h s λ [r λ λ 2 ] dλ Φ max = λ h λ s λ [r λ + 2 ] dλ λ t 26

27 Exposures that remove no capture region No Capture Spectral Power Response vs Wavelength Pixel Number vs Radiant Power W m 2 sr μm 11 Camera sensitivity s(λ) I max ~ I max Overexposure Human Spectral Response r(λ) ~ I min 8 Underexposure 7 14 (μm) Φ min Φ max Human Overexposed Human Underexposed t g I max (g) Φ min t g I min(g) Φ min 27

28 Optimal algorithm to obtain exposures ξ(n, T) = Φ min hdes h p ωdφ + Γ min Γ max hdes h p ωdφ ω = 0 h des (Φ) < h (Φ) 1 otherwise, Φ max Error Function Binary Weights argmin n,t ξ(n, T) s. t. 1. T i > 0 2. T i T i I max Φ min T i I min Φ max Grossberg and Nayar

29 Optimal algorithm to obtain exposures Objective function score Optimal solution for 112 exposures Grid search index HDR Image Algorithm assumes a single no capture region Brute force search is therefore tractable 29

30 HDR Results Over Under Fusion 30

31 HDR Results Over Under Fusion 31

32 HDR Results Over Under Fusion 32

33 HDR Results 33

34 Method Comparison 34

35 Comparison Digitization Noise Exposure Low noise Good image quality Real-time Hardware and firmware upgrades Real-time No hardware modification Low image quality Noisy Low noise No hardware modification Good image quality Multiple Images and more capture time 35

36 Future Work Pilot deployment program of private sensors at UF Health Shands Hospital. Generate database of private face images to for privacy challenge. Generate database of private videos for activity recognition in a hospital setting. 36

37 Acknowledgements Sanjeev Koppal Andreas Enqvist DHS This material is based upon work supported by the U.S. Department of Homeland Security under Grant Award Number, 2014-DN-077-ARI The views and conclusions contained in this document are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of the U.S. Department of Homeland Security.

38 Summary: Three Sensor Level Approaches Digitization Noise Exposure 38