Multi-perspective Panoramas. Slides from a talk by Lihi Zelnik-Manor at ICCV 07 3DRR workshop

Transcription

1 Multi-perspective Panoramas Slides from a talk by Lihi Zelnik-Manor at ICCV 07 3DRR workshop

2 Objectives 1. Better looking panoramas 2. Let the camera move: Any view Natural photographing

3 Stand on the shoulders of giants Cartographers Artists

4 Cartographic projections

5 Common panorama projections Perspective Stereographic Cylindircal φ θ

6 Global Projections Perspective Stereographic Cylindircal

7 Learn from the artists Multiple view points Sharp discontinuity perspective perspective De Chirico Mystery and Melancholy of a Street, 1914

8 Renaissance painters solution School of Athens, Raffaello Sanzio ~1510 Give a separate treatment to different parts of the scene!!

9 Personalized projections School of Athens, Raffaello Sanzio ~1510 Give a separate treatment to different parts of the scene!!

10 Multiple planes of projection Sharp discontinuities can often be well hidden

11 Single view Our multi-view result

12 Single view Our multi-view result

13 Single view Our multi-view result

14 Input images Applying personalized projections Foreground Background panorama

15 Single view Our multi-view result

16 Objectives - revisited 1. Better looking panoramas 2. Let the camera move: Any view Natural photographing Multiple views can live together

17 Multi-view compositions David Hockney, Place Furstenberg, (1985)

18 Why multi-view? Multiple viewpoints Single viewpoint David Hockney, Place Furstenberg, 1985 Melissa Slemin, Place Furstenberg, 2003

19 Long Imaging Agarwala et al. (SIGGRAPH 2006)

20 Smooth Multi-View Google maps

21 What s wrong in the picture? Google maps

22 Non-smooth Google maps

23 The Chair David Hockney (1985)

24 Joiners are popular Flickr statistics (Aug 07): 4,985 photos matching joiners. 4,007 photos matching Hockney. 41 groups about Hockney Thousands of members

25 Main goals: Automate joiners Generalize panoramas to general image collections

26 For Artists: Reduce manual labor Objectives Manual: ~40min. Fully automatic

27 For Artists: Reduce manual labor Objectives For non-artists: Generate pleasing-to-the-eye joiners

28 For Artists: Reduce manual labor Objectives For non-artists: Generate pleasing-to-the-eye joiners For data exploration: Organize images spatially

29 What s going on here?

30 A cacti garden

31 Principles

32 Principles Convey topology Correct Incorrect

33 Principles Convey topology A 2D layering of images Blending: blurry Graph-cut: cuts hood Desired joiner

34 Principles Convey topology A 2D layering of images Don t distort images translate rotate scale

35 Principles Convey topology A 2D layering of images Don t distort images Minimize inconsistencies Bad Good

36 Algorithm

37 Step 1: Feature matching Brown & Lowe, ICCV 03

38 Step 2: Align Large inconsistencies Brown & Lowe, ICCV 03

39 Step 3: Order Reduced inconsistencies

40 Ordering images Try all orders: only for small datasets

41 Ordering images Try all orders: only for small datasets complexity: (m+n)α m = # images n = # overlaps α = # acyclic orders

42 Ordering images Observations: Typically each image overlaps with only a few others Many decisions can be taken locally

43 Ordering images Approximate solution: Solve for each image independently Iterate over all images

44 Can we do better?

45 Step 4: Improve alignment

46 Iterate Align-Order-Importance

47 Iterative refinement Initial Final

48 Iterative refinement Initial Final

49 Iterative refinement Initial Final

50 What is this?

51 That s me reading

52 Anza-Borrego

53 Tractor

54 Paolo Uccello, 1436 Art reproduction

55 Art reproduction Paolo Uccello, 1436 Zelnik & Perona, 2006

56 Art reproduction Single view-point Zelnik & Perona, 2006

57 Manual by Photographer

58 Our automatic result

59 Failure?

60 GUI

61 The Impossible Bridge

62 Homage to David Hockney

63 Take home Incorrect geometries are possible and fun! Geometry is not enough, we need scene analysis A highly related work: "Scene Collages and Flexible Camera Arrays, Y. Nomura, L. Zhang and S.K. Nayar, Eurographics Symposium on Rendering, Jun, 2007.

64 Thank You

65 Class Project from /f07/proj_final/www/echuangs/