Lecture 1 Introduction to Computer Vision. Lin ZHANG, PhD School of Software Engineering, Tongji University Spring 2015

Transcription

1 Lecture 1 Introduction to Computer Vision Lin ZHANG, PhD School of Software Engineering, Tongji University Spring 2015

2 Course Info Contact Information Room 314, Jishi Building Tel: TA: Lida LI, QQ: Course information can be found at

3 Materials Major materials My slides References Some papers Milan Sonka, Vaclav Hlavac, and Roger Boyle, Image Processing, Analysis, and Machine Vision, Thomson, 2008 D.A. Forsyth and J. Ponce, Computer Vision: A Modern Approach, Pearson Education, Inc., 2003

4 Examination Homework 45%: 3 times, and each time 15%. Project 50%: 2 or 3 people for one group Attendance 5% (being absent >=5 times, you will fail this course) Bonus 5%: being active in class and answering my questions correctly

5 Today What is computer vision? Why is computer vision difficult? Why do we need to study CV? Course overview

6 What is vision? The plain man s answer (and Aristotle s too) would be, to know what is where by looking. In other words, vision is the process of discovering from images what is present in the world, and where it is David Marr, Vision 1982 David Marr ( ), was a British neuroscientist and psychologist. The Marr Prize, one of the most prestigious awards in computer vision, is named in his honor.

7 What is computer vision? To bridge the gap between pixels and meaning What we see What a computer sees

8 What is computer vision? Computer vision is the science and technology of machines that see Concerned with the theory for building artificial systems that obtain information from images The image data can take many forms, such as a video sequence, depth images, views from multiple cameras, or multi dimensional data from a medical scanner

9 What is computer vision? Source: Feifei Li

10 Human vision sclera choroid blind spot

11 Human vision

12 What is it related to? Source: Feifei Li

13 Vision as a measurement device Real time stereo Structure from motion Reconstruction from Internet photo collections NASA Mars Rover Pollefeys et al. Goesele et al.

14 Vision as a source of semantic information slide credit: Fei Fei, Fergus & Torralba

15 Object categorization sky building flag banner bus face street lamp bus wall cars slide credit: Fei Fei, Fergus & Torralba

16 Scene and context categorization outdoor city traffic slide credit: Fei Fei, Fergus & Torralba

17 A little story about computer vision In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to spend the summer linking a camera to a computer and getting the computer to describe what it saw

18 Today What is computer vision? Why is computer vision difficult? Why do we need to study CV? Course overview

19 Why computer vision is difficult? Challenges: viewpoint variation Michelangelo

20 Why computer vision is difficult? Challenges: illumination

21 Why computer vision is difficult? Challenges: scale slide credit: Fei Fei, Fergus & Torralba

22 Why computer vision is difficult? Challenges: deformation Xu, Beihong 1943 Source: Feifei Li

23 Why computer vision is difficult? Challenges: occlusion Magritte, 1957

24 Why computer vision is difficult? Challenges: background clutter

25 Why computer vision is difficult? Challenges: Motion

26 Why computer vision is difficult? Challenges: object intra class variation Source: Feifei Li

27 Today What is computer vision? Why is computer vision difficult? Why do we need to study CV? Course overview

28 Why study computer vision?

29 Why study computer vision? Vision is useful: Images and video are everywhere! Personal photo albums Movies, news, sports Surveillance and security Medical and scientific images

30 Visual search Google Query image Output

31 Visual search Google Where is it?

32 Earth Viewers (3D modeling) Image from Baidu 3D Map

33 Photosynth Project products of students from 2009 Media&Arts

34 Structure from motion Bundler: Structure from Motion (SfM) for Unordered Image Collections (

35 Autonomous vehicles

36 Optical character recognition (OCR) Technology to convert scanned docs to text If you have a scanner, it probably came with OCR software Digit recognition, AT&T labs License plate readers Source: S. Seitz

37 Videos based applications

38 Face detection Many new digital cameras now detect faces Canon, Sony, Fuji, Source: S. Seitz

39 Smile detection Source: S. Seitz

40 Vision based biometrics How the Afghan Girl was Identified by Her Iris Patterns

41 Login without a password Palmprint system Fingerprint scanners on many new laptops, other devices Finger Knuckle Print system FKP Video Demo

42 Face verification National Stadium, Beijing Olympic Games, 2008

43 Special effects: motion capture Source: S. Seitz Pirates of the Carribean, Industrial Light and Magic

44 Vision in space NASA'S Mars Exploration Rover Spirit captured this westward view from atop a low plateau where Spirit spent the closing months of Vision systems (JPL) used for several tasks Panorama stitching 3D terrain modeling Obstacle detection, position tracking For more, read Computer Vision on Mars by Matthies et al.

45 Robotics NASA s Mars Spirit Rover Video Demo of Itti s Robot

46 Household surveillance robot Video Demo of Household Robot

47 Medical imaging 3D imaging MRI, CT Video demo for image guided surgery

48 You can find a good job! Many first class companies now are developing CV related applications, to name a few Google Microsoft HP Facebook Tencent Baidu iqiyi DJI Huawei

49 Today What is computer vision? Why is computer vision difficult? Why do we need to study CV? Course overview

50 Course content (just a plan) Introduction Image filtering Local interest point detectors Local feature descriptors and matching Biometrics: Theories and applications Face detection and face recognition Introduction to numerical geometry Deep learning and its applications

51 Some tips Prerequisites Linear algebra Calculus Matlab Programming C++ Programming Knowledge sources IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) IEEE Transactions on Image Processing (TIP) International Journal of Computer Vision (IJCV) IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) IEEE International Conference on Computer Vision (ICCV) European Conference on Computer Vision (ECCV)

52 Thanks for your attention