Introduction. BIL719 Computer Vision Pinar Duygulu Hacettepe University

Transcription

1 Introduction BIL719 Computer Vision Pinar Duygulu Hacettepe University

2 Basic Info Textbooks (suggested): Forsyth & Ponce, Computer Vision: A Modern Approach Richard Szeliski, Computer Vision: Algorithms and Applications (available online)

3 Course requirements Homeworks 50% Quizzes 20% Project 30 %

4 Source: Svetlana Lazebnik, UIUC Why study computer vision?

5 Why study computer vision? An image is worth 1000 words Images and movies are everywhere Source: Fei Fei Li, Stanford University

6 Massive amounts of visual data For YouTube alone More than 1 billion unique users Hundreds of millions of hours are watched every day 300 hours of video are uploaded every minute 6

7 What do you see in the picture? Source: Martial Hebert, CMU

8 What do you see in the picture? Black background Two objects One teapot One toy There is a light coming from right One object is shiny the other is not Toy: Consists of 5 layers, in different colors There is a text : Fisher Price The layers are in donut shape Layers are plastic Bottom is wood Teapot: Consists of body and handle Body is metal Handle is ceramic Handle: Dark blue on white Body : golden Reflection of toy on the body Source: Martial Hebert, CMU

9 Challenge What do you see in the picture? Source: Octavia Camps, Penn State

10 Challenge What do you see in the picture? A hand holding a man A hand holding a shiny sphere An Escher drawing Source: Octavia Camps, Penn State

11 Perception and grouping Subjective contours

12 Source: Michael Black, Brown University Occlusion

13 Source: Michael Black, Brown University Shading

14 Source: Michael Black, Brown University Shading

15 Source: Michael Black, Brown University Parts and relations

16 How good are our models? Source: Michael Black, Brown University

17 How good are our models? Source: Michael Black, Brown University

18 Is it only about matching? Source: Michael Black, Brown University

19 Is it only about matching? Source: Michael Black, Brown University

20 Context

21 Context a person?

22 Context a person? the blob is identical to the one on the previous slide after a 90deg rotation

23 Source: Michael Black, Brown University Prior Expectations

24 The goal of computer vision To extract meaning from pixels Source: 80 million tiny images by Torralba et al. Humans are remarkably good at this

25 The goal of computer vision To extract meaning from pixels Source: S. Narasimhan What What we see we see What a computer sees

26 Template matching Slide credit: Fei-fei Li Pinar Duygulu, ENLG

27 Scene categorization outdoor/indoor city/forest/factory/etc. Slide credit: Fei-fei Li and Sevetlana Lazebnik Pinar Duygulu, ENLG

28 Image annotation/tagging street people building mountain Slide credit: Fei-fei Li and Sevetlana Lazebnik Pinar Duygulu, ENLG

29 Object Detection find pedestrians Slide credit: Fei-fei Li and Sevetlana Lazebnik Pinar Duygulu, ENLG

30 Activity Recognition walking shopping rolling a cart sitting talking Slide credit: Fei-fei Li and Sevetlana Lazebnik Pinar Duygulu, ENLG

31 Object recognition sky mountain building tree banner people building street lamp market Slide credit: Fei-fei Li and Sevetlana Lazebnik Pinar Duygulu, ENLG

32 Applications

33 Optical character recognition (OCR) Digit recognition yann.lecun.com License plate readers Sudoku grabber Source: S. Seitz, N. Snavely Automatic check processing

34 Biometrics Fingerprint scanners on many new laptops, other devices Face recognition systems now beginning to appear more widely Source: S. Seitz

35 Face detection Many consumer digital cameras now detect faces Source: S. Seitz

36 Smile detection Source: S. Seitz Sony Cyber-shot T70 Digital Still Camera

37 Face recognition: Apple iphoto software Source: S. Lazebnik, UIUC

38 Mobile visual search: Google Goggles Source: S. Lazebnik, UIUC

39 Automotive safety Mobileye: Vision systems in high-end BMW, GM, Volvo models Pedestrian collision warning Forward collision warning Lane departure warning Headway monitoring and warning Source: A. Shashua, S. Seitz

40 Source: S. Lazebnik, UIUC Self-driving cars

41 Vision-based interaction: Xbox Kinect Source: S. Lazebnik, UIUC

42 3D Reconstruction: Kinect Fusion YouTube Video Source: S. Lazebnik, UIUC

43 3D Reconstruction: Multi-View Stereo Source: S. Lazebnik, UIUC YouTube Video

44 Photosynth Based on Photo Tourism technology developed by Noah Snavely, Steve Seitz, and Rick Szeliski Source: Szeliski, Seitz. Chen

45 Google Maps Photo Tours

46 Earth viewers (3D modeling) Image from Microsoft s Virtual Earth (see also: Google Earth) Source: Szeliski, Seitz. Chen

47 Object recognition (in supermarkets) LaneHawk by EvolutionRobotics A smart camera is flush-mounted in the checkout lane, continuously watching for items. When an item is detected and recognized, the cashier verifies the quantity of items that were found under the basket, and continues to close the transaction. The item can remain under the basket, and with LaneHawk,you are assured to get paid for it Source: Szeliski, Seitz. Chen

48 Object recognition (in mobile phones) This is becoming real: Microsoft Research Point & Find, Nokia Source: Szeliski, Seitz. Chen

49 Special effects: shape and motion capture Source: Szeliski, Seitz. Chen

50 Vision in space NASA'S Mars Exploration Rover Spirit captured this westward view from atop a low plateau where Spirit spent the closing months of Vision systems (JPL) used for several tasks Source: Szeliski, Seitz. Chen Panorama stitching 3D terrain modeling Obstacle detection, position tracking For more, read Computer Vision on Mars by Matthies et al.

51 Robotics NASA s Mars Spirit Rover Source: Szeliski, Seitz. Chen

52 Medical imaging 3D imaging MRI, CT Image guided surgery Grimson et al., MIT Source: Szeliski, Seitz. Chen

53 Why is computer vision difficult?

54 Challenges: viewpoint variation Michelangelo Source: Fei-Fei, Fergus & Torralba

55 Challenges: illumination Source: J. Koenderink

56 Challenges: scale Source: Fei-Fei, Fergus & Torralba

57 Challenges: deformation Xu, Beihong 1943 Source: Fei-Fei, Fergus & Torralba

58 Challenges: occlusion Source: Fei Fei, Fergus, Torralba

59 Challenges: Background Clutter Source: Svetlana Lazebnik, UIUC

60 Challenges: Motion Source: Svetlana Lazebnik, UIUC

61 Challenges: object intra-class variation Source: Fei-Fei, Fergus & Torralba

62 Source: Fei-Fei, Fergus & Torralba Challenges: local ambiguity

63 Challenges: local ambiguity Source: Rob Fergus and Antonio

64 Challenges: local ambiguity Source: Rob Fergus and Antonio

65 Context Slide credit: Fei-fei Li Pinar Duygulu, ENLG

66 Challenges: Inherent ambiguity Many different 3D scenes could have given rise to a particular 2D picture Image source: Svetlana Lazebnik, UIUC

67 Challenges or opportunities? Images are confusing, but they also reveal the structure of the world through numerous cues Our job is to interpret the cues! Image source: J. Koenderink

68 Depth cues: Linear perspective Image source: Svetlana Lazebnik, UIUC

69 Depth cues: Aerial perspective Image source: Svetlana Lazebnik, UIUC

70 Source: J. Koenderink Depth ordering cues: Occlusion

71 Image source: Svetlana Lazebnik, UIUC Shape cues: Texture gradient

72 Shape and lighting cues: Shading Image source: Svetlana Lazebnik, UIUC

73 Position and lighting cues: Cast shadows Source: J. Koenderink

74 Grouping cues: Similarity (color, texture, proximity) Source: Svetlana Lazebnik, UIUC

75 Grouping cues: Common fate Source: Fei Fei Li, Stanford University

76 Origins of computer vision L. G. Roberts, Machine Perception of Three Dimensional Solids, Ph.D. thesis, MIT Department of Electrical Engineering, Source: Svetlana Lazebnik, UIUC

77 Connections to other disciplines Artificial Intelligence Robotics Machine Learning Computer Vision Computer Graphics Cognitive science Neuroscience Image Processing Source: Svetlana Lazebnik, UIUC

78