Depth from Diffusion

Size: px

Start display at page:

Download "Depth from Diffusion"

Bryan Kelley
6 years ago
Views:

1 Depth from Diffusion Changyin Zhou Oliver Cossairt Shree Nayar Columbia University Supported by ONR

2 Optical Diffuser

3 Optical Diffuser ~ 10 micron Micrograph of a Holographic Diffuser (RPC Photonics) [Gray, 1978] [Chang et al., 2006] [Garcia-Guerrero et al. 2007]

4 Diffusers as Accessories (B&H) Diffusers for illumination Diffuser to preview the image (B&H) Diffusers to soften the image

5 Diffusion Encodes Depth Diffuser Diffuser Camera Camera The amount of diffusion i varies with depth.

6 Geometry of Diffusion: A Pinhole Camera Miss P Q Pinhole Sensor

7 Geometry of Diffusion: A Pinhole Camera θ P Pinhole Sensor Diffuser

8 Geometry of Diffusion: A Pinhole Camera θ θ A B P Pinhole Sensor Diffuser

9 Geometry of Diffusion: A Pinhole Camera Diffusion Law: Pinhole θ θ A B P 2r Sensor O V U Z Diffuser

10 Geometry of Diffusion: A Pinhole Camera Diffusion Size and Depth: Pinhole θ θ A B P 2r Sensor O V U Z Diffuser

11 Geometry of Diffusion: A Pinhole Camera Diffusion Size and Depth: Diffuser as a proxy object Z P Pinhole 2r O Sensor V U Diffuser

12 Diffusion as Convolution: A Pinhole Camera Assume field angle and depth are constant for small image patches, we have: Captured Image Latent clear image Diffusion PSF Diffusion Size

13 Geometry of Diffusion: A Lens Camera

14 Geometry of Diffusion: A Lens Camera Diffuser as a proxy object Z P Pinhole 2r O Sensor V U Diffuser

15 Geometry of Diffusion: A Lens Camera The captured image can be further blurred due to defocus. Lens Diffuser as a proxy object Z P 2r Sensor V U Diffuser

16 Diffusion as Convolution: A Lens Camera For a lens camera with a diffuser, we have: The Final PSF Diffusion PSF Defocus PSF is the diffusion PSF if a pinhole were used. is the defocus PSF if the diffuser were removed.

17 Depth from Diffusion (DFDiff) Algorithm 1. Capture Two Images With a diffuser Without a diffuser 2. Estimate Blur Size r Same form as in DFD 3. Compute Depth Z

18 Depth from Diffusion vs. Depth from Defocus Depth from Defocus P Aperture pattern r Sensor Lens Z Focal Plane Depth from Diffusion θ P Diffusion pattern r Sensor Pinhole θ Diffuser Z [Pentland, 1987] [Subbarao, 1988] [Watanabe & Nayar, 1996] [Chaudhuri & Rajagopalan, 1999] [Favaro & Soatto, 2005] [Schechner & Kiryati, 2000]

19 Depth from Diffusion vs. Depth from Defocus Depth from Diffusion Suppose 22.5x15mm Sensor, 10 um pixel, 100 mm EFL Any lens is fine! A Diffuser of 21.8 o P Field of View distance = 1000 mm Depth precision is about 0.1 mm.

20 Depth from Diffusion vs. Depth from Defocus Depth from Defocus Suppose 22.5x15mm Sensor, 10 um pixel, 100 mm EFL Lens Aperture diameter? P Field of View distance = 1000 mm Depth precision is about 0.1 mm.

21 Depth from Diffusion vs. Depth from Defocus Depth from Defocus Suppose 22.5x15mm Sensor, 10 um pixel, 100 mm EFL Aperture diameter 800 mm P distance = 1000 mm Depth precision is about 0.1 mm.

22 Depth from Diffusion vs. Depth from Defocus Depth from Diffusion Suppose 22.5x15mm Sensor, 10 um pixel, 100 mm EFL Any lens is fine! A Diffuser of 11.2 o P distance = 5000 mm Depth precision is about 1.0 mm.

23 Depth from Diffusion vs. Depth from Defocus Depth from Defocus Suppose 22.5x15mm Sensor, 10 um pixel, 100 mm EFL Lens Aperture diameter? P distance = 5000 mm Depth precision is about 1.0 mm.

24 Depth from Diffusion vs. Depth from Defocus Depth from Defocus Suppose 22.5x15mm Sensor, 10 um pixel, 100 mm EFL Aperture diameter 2000 mm P distance = 5000 mm Depth precision is about 1.0 mm.

25 PSF Measurement: A Pinhole Camera F/22, Field Angle =0 o Captured Modeled Z = 2 mm Z = 5 mm - Canon EOS T1i; EF 50mm F/1.8 Lens; - Luminit Holographic Diffuser (10 o Gaussian); - Diffuser distance: U = 1m

26 PSF Measurement: A Pinhole Camera F/22, Field Angle =10 o Captured Modeled Z = 2 mm Z = 5 mm - Canon EOS T1i; EF 50mm F/1.8 Lens; - Luminit Holographic Diffuser (10 o Gaussian); - Diffuser distance: U = 1m

27 PSF Measurement: A Lens Camera F/1.8, Field Angle =10 o Captured Modeled - Canon EOS T1i; EF 50mm F/1.8 Lens; - Luminit Holographic Diffuser (10 o Gaussian); - Diffuser distance: U = 1m

28 Experiments Canon 20D + 50mm Lens Five playing cards, 0.29mm thick each Luminit it Diffuser (20 o )

29 Experiments Captured WITHOUT a Diffuser Captured WITH a Diffuser

30 Experiments Five playing cards, 0.29mm thick each Computed Depth Map (~ 0.1 mm precision) (mm)

31 Experiments A small sculpture of about 4mm thickness Canon G5 Compact Camera Luminit Diffuser (5 o )

32 Experiments Captured WITHOUT a Diffuser Captured WITH a Diffuser

33 Experiments A small sculpture of about 4mm thickness Computed Depth Map A 3D View of Depth Map

34 Experiments 450 mm m 650 mm Canon 20D; Gaussian Diffuser (10 o )

35 Experiments Stitched Depth Map (precision) (mm)

36 Summary Formulated the image formation with optical diffusers Proposed Depth from Diffusion - Require a diffuser on the object side + High-precision depth estimation + Distant objects + Less sensitive to lens aberrations Diffuser Camera Demonstrated high-precision depth estimation

37 Depth from Diffusion Changyin Zhou Oliver Cossairt Shree Nayar Columbia University Supported by ONR

When Does Computational Imaging Improve Performance?

When Does Computational Imaging Improve Performance? Oliver Cossairt Assistant Professor Northwestern University Collaborators: Mohit Gupta, Changyin Zhou, Daniel Miau, Shree Nayar (Columbia University)