CPSC 425: Computer Vision

Transcription

1 1 / 55 CPSC 425: Computer Vision Instructor: Fred Tung ftung@cs.ubc.ca Department of Computer Science University of British Columbia Lecture Notes 2015/2016 Term 2

2 2 / 55 Menu January 7, 2016 Topics: Image Formation Cameras and Lenses Reading: Today: Forsyth & Ponce (2nd ed.) Next: Forsyth & Ponce (2nd ed.) 4.1, 4.5 Reminders: Complete Assignment 1 by Tuesday, January 12 www: piazza:

3 3 / 55 Today s Fun Example: Eye in Sink Illusion Photo credit: reddit user Liammm

4 4 / 55 Today s Fun Example: Eye in Sink Illusion Tried taking a picture of a sink draining, wound up with a picture of an eye instead Photo credit: reddit user Liammm

5 5 / 55 Lecture 1: Re-cap What we see depends on: object shape surface material illumination viewpoint Visual perception also is influenced by: familiarity context expectation

6 6 / 55 Lecture 1: Re-cap Computer vision technologies have moved from research labs into commercial products and services. Examples cited include: broadcast television sports electronic games (Microsoft Kinect) real-time language translation image search smart infrastructure

7 7 / 55 Framework for Today s Topic Problem: Obtain information about the 3D world Key Idea(s): Treat a camera as a scientific instrument for obtaining measurements of the 3D world Alternatives: Treat images as 2D entities only Treat images as just another kind of big data Theory: Optics (geometry and radiometry) Practical Detail(s): Cameras and lenses Gotchas: interpretation of 3D world can be ambiguous role of human perception

8 8 / 55 Overview: Image Formation, Cameras and Lenses Goal: to understand how images are formed Camera obscura dates from 16th century (and earlier) Basic abstraction is the pinhole camera Cameras and lenses maintain the abstraction The human eye functions very much like a camera

9 9 / 55 Camera Obscura (Latin for dark chamber ) Reinerus Gemma-Frisius observed an eclipse of the sun at Louvain on January 24, He used this illustration in his book, De Radio Astronomica et Geometrica, It is thought to be the first published illustration of a camera obscura. Credit: John H., Hammond, The Camera Obscura, A Chronicle

10 10 / 55 First Photograph on Record La table servie Credit: Nicéphore Niepce, 1822

11 11 / 55 Pinhole Camera A pinhole camera is a box with a small hole in it

12 12 / 55 Image Formation Forsyth & Ponce (2nd ed.) Figure 1.1 Credit: US Navy, Basic Optics and Optical Instruments. Dover, 1969

13 13 / 55 Pinhole Camera (Simplified) x x f z image plane pinhole object

14 14 / 55 Pinhole Camera (Simplified) (cont d) x x x f f z image plane pinhole image plane object

15 15 / 55 Perspective Effects Far objects appear smaller than close ones Size is inversely proportional to distance

16 16 / 55 Perspective Effects (cont d) Parallel lines meet

17 Vanishing Points 17 / 55

18 18 / 55 Vanishing Points Slide credit: David Jacobs

19 19 / 55 Vanishing Points (cont d) Each set of parallel lines meets at a different point the vanishing point for this direction Sets of parallel lines on the same plane lead to collinear vanishing points the line is called the horizon for that plane Good ways to spot faked images scale and perspective don t work vanishing points behave badly

20 20 / 55

21 Perspective Projection 21 / 55

22 22 / 55 Perspective Projection Forsyth & Ponce (1st ed.) Figure 1.4 3D object point, P[x, y, z], projects to 2D image point P [x, y ] where x = f x z y = f y z

23 Weak Perspective 23 / 55

24 Weak Perspective Forsyth & Ponce Figure 1.5 (1st ed.) 3D object point, P[x, y, z] in Π 0, projects to 2D image point P [x, y ] where and m = f x = m x y = m y z 0 24 / 55

25 25 / 55 Orthographic Projection Forsyth & Ponce (1st ed.) Figure 1.6 3D object point, P[x, y, z], projects to 2D image point P [x, y ] where x = x y = y

26 26 / 55 Summary of Projection Equations 3D world point, P[x, y, z], projects to 2D image point P [x, y ] where Perspective x = f x z y = f y z Weak Perspective x = m x y = m y m = f z 0 Orthographic x = x y = y

27 27 / 55 Projection Models: Pros and Cons Weak perspective (including orthographic) has simpler mathematics Accurate when object is small and/or distant Useful for recognition Perspective more accurate for real scenes Useful in structure from motion When maximum accuracy required, it is necessary to model additional details of the particular camera Use perspective projection with other calibration parameters (e.g., radial lens distortion)

28 28 / 55 Why Not a Pinhole Camera? Credit: E. Hecht. Optics, Addison-Wesley, 1987

29 29 / 55 Why Not a Pinhole Camera (cont d)? If pinhole is too big then many directions are averaged, blurring the image If pinhole is too small then diffraction becomes a factor, also blurring the image Generally, pinhole cameras are dark, because only a very small set of rays from a particular scene point hits the image plane Equivalently, pinhole cameras are slow, because only a very small amount of light from a particular scene point hits the image plane per unit time

30 30 / 55 Snell s Law n 1 sin α 1 = n 2 sin α 2

31 The Reason for Lenses 31 / 55

32 32 / 55 Pinhole Model (Simplified) with Lens x x z z image plane lens object

33 Thin Lens Equation 33 / 55

34 34 / 55 Thin Lens Equation Forsyth & Ponce (1st ed.) Figure z 1 z = 1 f

35 35 / 55 Aside: Depth From Focus Figure credit: H. Jin and P. Favaro, 2002

36 36 / 55 Focal Length f image plane lens Another way of looking at the focal length of a lens. The incoming rays, parallel to the optical axis, converge to a single point a distance f behind the lens. This is where we want to place the image plane.

37 37 / 55 Out Of Focus f image plane lens The image plane is in the wrong place, either slightly closer than the required focal length, f, or slightly further than the required focal length, f.

38 38 / 55 Spherical Aberration Forsyth & Ponce (1st ed.) Figure 1.12a

39 39 / 55 Compound Lens Systems A modern camera lens may contain multiple components, including aspherical elements

40 Vignetting Vignetting in a two-lens system Forsyth & Ponce (2nd ed.) Figure 1.12 The shaded part of the beam never reaches the second lens 40 / 55

41 41 / 55 Vignetting Image credit: Cambridge in Colour

42 42 / 55 Other (Possibly Significant) Lens Effects Chromatic aberration Index of refraction depends on wavelength, λ, of light Light of different colours follows different paths Therefore, not all colours can be in equal focus

43 43 / 55 Chromatic Aberration Image credit: Trevor Darrell

44 44 / 55 Other (Possibly Significant) Lens Effects Chromatic aberration Index of refraction depends on wavelength, λ, of light Light of different colours follows different paths Therefore, not all colours can be in equal focus Scattering at the lens surface Some light is reflected at each lens surface There are other geometric phenomena/distortions pincushion distortion barrel distortion etc

45 45 / 55 Lens Distortion Image credit: Fig in Szeliski

46 46 / 55 Human Eye Image credit:

47 47 / 55 Human Eye The eye has an iris (like a camera) Focusing is done by changing shape of lens When the eye is properly focused, light from an object outside the eye is imaged on the retina. The retina contains light receptors called rods and cones

48 48 / 55 Human Eye Rods 75 to 150 million Not involved in colour vision Sensitive to low levels of illumination Capable of responding to a single photon, but yield relatively poor spatial detail Cones 6 to 7 million Highly sensitive to colour Active only at higher levels of illumination but yield higher resolution

49 49 / 55 Human Eye Density of rods and cones Image credit: Gonzalez and Woods (3rd ed.) Fig. 2.2

50 Human Eye: Illumination Sensitivity A classic experiment to study the sensitivity of the human vision system to different illumination levels: A subject looks at a uniformly illuminated field typically a diffuser such as opaque glass that is illuminated from behind by a light source whose brightness can be varied An increment of illumination is added in the form of a short-duration flash The subject states whether or not there is a perceivable change Image credit: Gonzalez and Woods (3rd ed.) Fig / 55

51 51 / 55 Human Eye: Illumination Sensitivity Image credit: Gonzalez and Woods (3rd ed.) Fig. 2.5 The ratio I c /I, where I c is the increment of illumination that is enough to be perceivable 50% of the time, is known as the Weber ratio A small value for I c /I means a small change in illumination is discernable - high illumination sensitivity.

52 52 / 55 Human Eye: Illumination Sensitivity A typical plot of the Weber ratio as a function of brightness: Image credit: Gonzalez and Woods (3rd ed.) Fig. 2.6 Shows that illumination sensitivity is poor at low levels of illumination and improves significantly as the background illumination increases Why two branches?

53 53 / 55 Human Eye: Simultaneous Contrast Image credit: Gonzalez and Woods (3rd ed.) Fig. 2.8 Finally, it s worth noting that human-perceived brightness is not a simple function of the intensity All the center squares have the same intensity, but appear to the eye to become darker as the background becomes lighter

54 54 / 55 Summary We discussed a physics-based approach to image formation. Basic abstraction is the pinhole camera. Lenses overcome limitations of the pinhole model while trying to preserve it as a useful abstraction Projection equations: perspective, weak perspective, orthographic Thin lens equation Some aberrations and distortions persist e.g. spherical aberration, vignetting The human eye functions much like a camera

55 Reminders: Complete Assignment 1 by Tuesday, January 12 www: piazza: 55 / 55