NTU CSIE. Advisor: Wu Ja Ling, Ph.D.

Size: px

Start display at page:

Download "NTU CSIE. Advisor: Wu Ja Ling, Ph.D."

Megan Lawrence
5 years ago
Views:

1 An Interactive Background Blurring Mechanism and Its Applications NTU CSIE Yan Chih Yu Advisor: Wu Ja Ling, Ph.D. 1

2 2

3 Outline Introduction Related Work Method Object Segmentation Depth Map Generation Image Defocus Experimental Result Applications Conclusion and Futurework 3

4 Introduction Introduction y Depth of field Circle of confusion Photo plane Real world 4

5 Introduction Depth of field Circle of confusion Photo plane Out of focus Real light Circle of Confusion 5

6 Introduction Depth of field Focus plane Photo plane Out of focus blur range Real light Depth of field Circle of Confusion Readability range 6

7 Introduction Depth of field Object distance, focal length, aperture size Focus plane Photo plane Out of focus blur range Real light Depth of field Circle of Confusion Readability range 7

8 Introduction Introduction y Shallow focus Photo plane Real world 8

9 Introduction Shallow focus Highlight the subject by softening background diffusion Deep focus by DC NIKON E4300 (2003) Shallow focus by DSLR NIKON D90(2008) 9

10 Related work Optics camera & Digital camera Depth Blur information effect Depth Blur information effect Special Effect (3D reconstruct) Other Applications 10

11 Related work Active Refocusing of Images and Videos [siggraph07] 11

12 Related work Image and Depth from a Conventional Camera with a Coded Aperture [siggraph07] 12

13 Related Work Single Image Dehazing [siggraph08] 13

14 Method Object Segmentation Lazy Snapping Alpha Matting Face Detection Depth thmap Generation Perspective Box Pop up Card Image Defocus Camera Settings Applications Privacy Preserving Photo Browser Defocus Blur 14

15 Method Segment Lazy snapping Mean shift Alpha matting Face detection User Stroke Resize to 20% Mean shift Do hard graph cut by lazy snapping Segmentation tri Map Segmentation alpha Map Lazy Snapping. ACM Trans. On Graphics Mean Shift: A Robust Approach Toward Feature Space Analysis. PAMI A Bayesian Approach to Digital Matting. CVPR

16 Method Depth map Perspective Box Vanish point Rear wall Perspective Box Pop up card Normal vector Depth Map Tour Into the Picture: Using a spidery mesh user interface to make animation from a single image. SIGGRAPH

17 Method Depth map Perspective Box Vanish point Rear wall Perspective Box Pop up card Normal vector Depth Map Tour Into the Picture: Using a spidery mesh user interface to make animation from a single image. SIGGRAPH

18 Method Depth map Perspective Box Vanish point Rear wall Perspective Box Pop up card Normal vector Depth Map Tour Into the Picture: Using a spidery mesh user interface to make animation from a single image. SIGGRAPH

19 Method Depth map Pop up card Perspective Box Pop up card Normal vector Depth Map 19

20 Method Depth map Normal vector Perspective Box Pop up card Normal vector Depth Map 20

21 Method Image defocus Blur circle diameter Depth Map Segment Map Camera setting Aperture size Focal length of the lens Distance of focus u b v b u b v b b d b d u v u v (a) (b) Blur circle diameter 21

22 Method Defocus blur bokeh I(x 1,y 1 ) I(x 2,y 2 ) I(x 3,y 3 ) E 1 (i,j) E 2 (i,j) E 3 (i,j) E n+2 (i,j) E n (i,j) E n+1 (i,j) I(x n,y n ) I(x n+1,y n+1 ) I(x n+2,y n+2 ),,,,,, x,, 1,,, 1,,,,,,,,,, 0,, 22

23 Method Defocus blur I(x 1,y 1 ) I(x 2,y 2 ) I(x 3,y 3 ) E 1 (i,j) E 2 (i,j) E 3 (i,j) E n+2 (i,j) E n (i,j) E n+1 (i,j) I(x n,y n ) I(x n+1,y n+1 ) I(x n+2,y n+2 ) 23

24 Method Near by object case Focus on the Focus on the flower flower 24

25 Method Blur circle diameter Objects in front of Objects behind h f Near by object case the focus the focus 25

26 Method Blur circle diameter Objects in front of Objects behind h f Near by object case the focus the focus Blur shape as alpha map Defocus Blur 26

27 Method Blur circle diameter Objects in front of Objects behind h f Near by object case the focus the focus Blur shape as alpha map Defocus Blur 27

28 Method Blur circle diameter Objects in front of Objects behind h f Near by object case the focus the focus Blur shape as alpha map Texture synthesis inpainting Defocus Blur Defocus Blur 28

29 Method Blur circle diameter Objects in front of Objects behind h f Near by object case the focus the focus Blur shape as alpha map Texture synthesis inpainting Defocus Blur Defocus Blur 29

30 Method Blur circle diameter Objects in front of Objects behind h f Near by object case the focus the focus Blur shape as alpha map Texture synthesis inpainting Defocus Blur Defocus Blur Interpolation by alpha map Shallow focus image 30

31 Experimental Result 1 Defocus blur method proposed in the system comparing with other blur filter results Deep focus Shallow focus 31

32 Experimental Result 1 Defocus blur method proposed in the system comparing with other blur filter results Gaussian blur Defocus blur Defocus blur + Bokeh 32

33 Experimental Result 1 Deep focus Shallow focus 33

34 Experimental Result 1 Original photograph Defocus blur method proposed in the system comparing with other blur filter results Gaussian blur Defocus blur Defocus blur + Bokeh 34

35 Experimental Result 2 with / without depth variation in the background Real photograph taken by DSLR 35

36 Experimental Result 2 with / without depth variation in the background Depth variation Original photograph Depth fixed Result after post processing 36

37 Experimental Result 3 Near by object case Focus on the Focus on the flower flower 37

38 Experimental Result 3 Near by object case Interpolation result Without inpainting 38

39 Experimental Result 4 Comparing with Photoshop Original image Photoshop (1 Hour) Our system (3~5min) 39

40 Applications Privacy Preserving 40

41 Applications Partial Viewing 41

42 Applications Partial Viewing 42

43 Applications Partial Viewing Applications Partial Viewing Applications 43

44 Applications Image Transition at Photo Browser 44

45 Applications Image Transition at Photo Browser 45

46 Applications Image Transition at Photo Browser Movie with defocus blur effect 46

47 Applications Image Transition at Photo Browser Concatenate two unrelated images 47

48 Conclusion and Future work We proposed an interactive refocusing tool for background blurring Simple user hint Defocus blur kernel Concatenate two related picture Future work concatenate two or more unrelated pictures Color based image retrieval technique 48

49 END THANK YOU 49

Nikon D750 ISO 200 1/60 sec. f/ mm lens

Nikon D750 ISO 200 1/60 sec. f/16 20 35mm lens 10 Creative Focus Sometimes tack-sharp focus isn t what you want for an image or for an entire image to tell the story you envision. What you focus on and