Spring 2018 CS543 / ECE549 Computer Vision. Course webpage URL:

Transcription

1 Spring 2018 CS543 / ECE549 Computer Vision Course webpage URL:

2 The goal of computer vision To extract meaning from pixels What we see What a computer sees Source: S. Narasimhan

3 The goal of computer vision To extract meaning from pixels Humans are remarkably good at this Source: 80 million tiny images by Torralba et al.

4 What kind of informa.on can be extracted from an image? roof tree tree building door sky chimney building window trashcan car car person Outdoor scene ground City European Seman,c informa.on Geometric informa.on

5 Why study computer vision? Vision is useful Vision is interesting Vision is difficult Half of primate cerebral cortex is devoted to visual processing Achieving human-level image understanding is probably AI-complete

6 Successes of computer vision to date

7 Simple patterns

8 Faces

9 Faces Beijing bets on facial recognition in a big drive for total surveillance Washington Post, 1/8/2018

10 Face movies I. Kemelmacher-Shlizerman, E. Shechtman, R. Garg and S. Seitz, Exploring Photobios, SIGGRAPH 2011 YouTube Video

11 Automatic age progression I. Kemelmacher-Shlizerman, S. Suwajanakorn, and S. Seitz, Illumination-Aware Age Progression, CVPR 2014 YouTube Video

12 Digital puppetry S. Suwajanakorn, S. Seitz, and I. Kemelmacher-Shlizerman, Synthesizing Obama: Learning Lip Sync from Audio, SIGGRAPH 2017 YouTube Video

13 Reconstruction: 3D from photo collections Q. Shan, R. Adams, B. Curless, Y. Furukawa, and S. Seitz, The Visual Turing Test for Scene Reconstruction, 3DV 2013 YouTube Video

14 Reconstruction: 4D from photo collections R. Martin-Brualla, D. Gallup, and S. Seitz, Time-Lapse Mining from Internet Photos, SIGGRAPH 2015 YouTube Video

15 Reconstruction: 4D from depth cameras R. Newcombe, D. Fox, and S. Seitz, DynamicFusion: Reconstruction and Tracking of Non-rigid Scenes in Real-Time, CVPR 2015 YouTube Video

16 Reconstruction in construction industry reconstructinc.com Source: D. Hoiem

17 Recognition Computer Eyesight Gets a Lot More Accurate, NY Times Bits blog, August 18, 2014 Building A Deeper Understanding of Images, Google Research Blog, September 5, 2014

18 Self-driving cars

19 Why is computer vision difficult?

20 Challenges: viewpoint variation

21 Challenges: illumination image credit: J. Koenderink

22 Challenges: scale slide credit: Fei-Fei, Fergus & Torralba

23 Challenges: deformation Xu, Beihong 1943 slide credit: Fei-Fei, Fergus & Torralba

24 Challenges: object intra-class variation slide credit: Fei-Fei, Fergus & Torralba

25 Challenges: occlusion, clutter Image source: National Geographic

26 Challenges: Motion

27 Challenges: ambiguity Many different 3D scenes could have given rise to a particular 2D picture

28 Challenges: ambiguity slide credit: Fei-Fei, Fergus & Torralba

29 Challenges: Semantic context

30 Challenges or opportunities? Images are confusing, but they also reveal the structure of the world through numerous cues Our job is to interpret the cues!

31 Depth cues: Linear perspective

32 Depth cues: Parallax

33 Shape cues: Texture gradient

34 Shape and lighting cues: Shading Michelangelo slide credit: Fei-Fei, Fergus & Torralba

35 Grouping cues: Similarity (color, texture, proximity)

36 Grouping cues: Common fate Image credit: Arthus-Bertrand (via F. Durand)

37 Origins of computer vision L. G. Roberts, Machine Perception of Three Dimensional Solids, Ph.D. thesis, MIT Department of Electrical Engineering, 1963.

38 Origins of computer vision Source: Fei-Fei Li

39 Connections to other disciplines Artificial Intelligence Robotics Machine Learning Computer Vision Computer Graphics Cognitive science Neuroscience Image Processing

40 Growth of the field Check out the list of CVPR 2017 corporate sponsors!

41 Course overview I. Early vision: Image formation and processing II. Mid-level vision: Grouping and fitting III. Multi-view geometry IV. Recognition V. Additional topics

42 I. Early vision Basic image formation and processing * = Cameras and sensors Light and color Linear filtering Edge detection Feature extraction, feature tracking

43 Fitting and grouping II. Mid-level vision Fitting: Least squares Hough transform RANSAC Alignment

44 III. Multi-view geometry Epipolar geometry Stereo Structure from motion 3D Photography

45 IV. Recognition Instance recognition, large-scale alignment Image classification Object detection Deep learning

46 V. Additional Topics (time permitting) Segmentation Video 3D scene understanding Images and text