Computational Photography: Illumination Part 2. Brown 1

Transcription

1 Computational Photography: Illumination Part 2 Brown 1

2 Lecture Topic Discuss ways to use illumination with further processing Three examples: 1. Flash/No-flash imaging for low-light photography (As well as an extension using a non-visible flash) 2. Multi-flash imaging for non-photorealistic rendering 3. Dual Photography Notes are from Marc Levoy, Stanford University Brown 2

3 Brown 3

4 Brown 4

5 Brown 5

6 Brown 6

7

8 Brown 8

9 Brown 9

10

11 Brown 11

12

13

14 Brown 14

15 Brown 15

16 Brown 16

17 Brown 17

18 Brown 18

19 Brown 19

20 Brown 20

21 Dark Flash Photography SIGGRAPH 2009 Dilip Krishnan and Rob Fergus New York University Dilip (Dilip Krishnan did his UG at NTU and Masters at NUS Mathematics) Idea is to replace the visible flash with a Ultra-Violet and Infrared Flash They call this a dark flash, since the people in the scene can t see the flash Rob Brown 21

22 Example Dark Flash (you don t see the flash) Ambient light With fast exposure. Note the noise. Their reconstructed image, combining F and A. Comparison with ambient light with long exposure. Brown 22

23 Idea is simple* Don t use a visible flash, instead use a non-visible flash Perform some type of fusion similar to the previous paper to filter the ambient image to remove noise Since overall idea is from Petschnigg s 2004 paper, this paper is written to explain details of building a dark-flash camera * Simple to state, building the device is not obvious. Brown 23

24 Light Their flash was modified to significantly attenuate the visible light to only allow IR and UV to pass through (see paper s Appendix). Brown 24

25 Hardware properties UV Visible Range IR This plot shows camera RGB response to spectrum Shows flashes ability to produce Irradiance on a while surface. Camera response to the irradiance above. Fuji IS Pro (Camera) + Nikon SB-14UV (Flash) This camera is known for its response to UV and IR light. Flash was modified to remove UV absorber, they also placed a filter on the flash to remove visible light (see previous slide) Brown 25

26 Reconstruction/Filtering This paper does not use the bi-lateral filter method by Petschnigg Instead, they fuse the images in the gradient domain Somewhat similar to Poisson Image Editing, but a bit different A simplified version of what they are solving for*: R 2 2 Find a reconstructed image R that minimizes the equation R should look like the ambient image The gradient of R should be similar to the gradient of the dark flash image D * Approach in the paper is more complicated and tunes the equations based on noise and spectral response properties. Refer to the paper. Brown 26

27 Dark Flash Summary Very cool idea and more practical than the original flash/no flash Suspect you may see something like this available in consumer cameras in the future Brown 27

28 Brown 28

29 Brown 29

30 Brown 30

31 Brown 31

32 Brown 32

33 Brown 33

34

35 Brown 35

36 Brown 36

37

38 Brown 38

39 Brown 39

40 Pradeep Sen 40

41 Helmholtz reciprocity light camera scene Note from Levoy (

42 Helmholtz reciprocity camera light scene

43 Measuring transport along a set of paths projector photocell scene

44 camera Reversing the paths point light scene

45 Forming a dual photograph dual camera projector dual photocell light scene

46 dual camera Forming a dual photograph dual light image of scene scene

47 Physical demonstration light replaced with projector camera replaced with photocell projector scanned across the scene conventional photograph, with light coming from right dual photograph, as seen from projector s position and as illuminated from photocell s position

48 Marc Levoy Related imaging methods time-of-flight scanner if they return reflectance as well as range but their light source and sensor are typically coaxial scanning electron microscope Velcro at 35x magnification, Museum of Science, Boston

49 The 4D transport matrix projector photocell camera scene

50 The 4D transport matrix projector camera P C pq x 1 mn x 1 mn x pq T scene

51 The 4D transport matrix mn x pq C = T P mn x 1 pq x 1

52 The 4D transport matrix mn x pq C = mn x 1 T pq x 1

53 The 4D transport matrix mn x pq C = mn x 1 T pq x 1

54 The 4D transport matrix mn x pq C = mn x 1 T pq x 1

55 The 4D transport matrix mn x pq C = T P mn x 1 pq x 1

56 The 4D transport matrix mn x pq C = T P mn x 1 pq x 1 applying Helmholtz reciprocity... pq x mn C = T T P pq x 1 mn x 1

57 Marc Levoy Example conventional photograph with light coming from right dual photograph as seen from projector s position

58 Marc Levoy Properties of the transport matrix little interreflection sparse matrix many interreflections dense matrix convex object diagonal matrix concave object full matrix Can we create a dual photograph entirely from diffuse reflections?

59 Marc Levoy Dual photography from diffuse reflections the camera s view

60 60

61 Computational Illumination Control the light Summary Most ideas synchronize simple light control with image Consider using non-visible light too Combined controlled light with processing This is at the heart of Computational Photography, we are modifying the hardware + additional processing to produce results beyond conventional imaging 61