CSE 527: Introduction to Computer Vision

Transcription

1 CSE 527: Introduction to Computer Vision Week 2 - Class 2: Vision, Physics, Cameras September 7th, 2017

2 Today Physics Human Vision Eye Brain Perspective Projection Camera Models Image Formation Digital Cameras

3 Very Briefly: Physics of Light

4 (Visible) Light EM Radiation

5 Why this exact band of the EM spectrum?

6 Photons A particle? A wave? Both? Have a frequency of oscillation Travel as the speed of light (very fast) Usually in a straight line Interact with matter (atoms) in different ways, which also depends on their frequency Oscill8 bro!

7 Light-Matter Interaction Light ( photons ) have a few choices when hitting a material: Transmission: Reflection: Absorption: go through it go back get stuck in it Other things may happen Refraction: bend through it Diffusion/Scatter: go in many ways Subsurface: hit things inside and more We re mostly interested in absorbed, reflected and refracted light.

8 What is color? Reflected light Some light is absorbed by the material, and some reflected. We see the reflected light. Materials have different properties: which frequencies are absorbed which frequencies are reflected which frequencies are transmitted...

9 The Bidirectional Reflectance Distribution Function An approximate numerical representation of the material light interaction properties Light direction wavelength Diffuse: Specular: Phong model : (introduce Ambient Lighting)

10 Lambertian Surfaces Ideal surfaces: Diffuse reflection occurs in a known pattern (Lambert s cosine rule) We often make this assumption in Vision applications. Reflect!

11 Human Vision And Visual Processing

12 Why do we care about human vision? Bionics, Bio-inspired Engineering, Biomimicry,... The human visual sense is very central (large swaths of the brain devoted to visual processing) We know that we can extract so much information from flat (stereo) images -> Inspired the creation of cameras -> Inspires (even today) computational methods for visual processing And there is still much to learn... Da Vinci s flying machine, inspired by bird wings

13 Human Eye Parts Is very much like a camera: - Light shines through a hole (pupil, iris) and focused (cornea, lens) on a sensitive substrate (retina) But much more complicated Aperture Lens Film / CCD

14 Retina Layers, Rods and Cones

15 Rods and Cones Humans may be considered to be Tetrachromats: able to see 4 colors. However mostly cones contribute to color vision, making us Trichromats.

16 Color Perception

17 This Tuesday (9/5/17). Human vision is affected by proteins too!

18 Retina Cone Mosaic Digital Sensor Grid

19 Retina Cell Arrangement, Fovea

20 Lateral Inhibition Receptor AND Gates

21 Lateral Inhibition

22 Visual Processing System Hierarchy

23 Brightness Constancy Principle

24 Brightness Constancy

25 Size Constancy

26 Pinhole Camera And Perspective Projection Perspectograph ca. 1600

27 Pinhole Camera C est moi! Kepler, Descartes, ca. 1600

28 Camera Obscura Earliest evidence: ~400 BCE Aristotle wrote about this! Later advancement (17th century): - Lenses - Drawing aid - Projectors (magic lantern) Yep.

29 Pinhole Camera Model

30 Normalized Camera

31 Focal Length Matters We ll get back to this shortly...

32 Perspective in Early Art Gentile di Fabriano (1423) Masolino (1425)

33 It s All a Matter of Perspective

34 Vanishing Points Parallel lines in the scene converge in a vanishing point in the image. Vanishing points and lines are cues to the visual system (e.g. in size constancy) to determine size and distance.

35 More Than a Single Vanishing Point One Point Two Points Three Points

36 Why Vanishing Points? Can be used to reconstruct the geometry in the scene, from a single image. [Wilczkowiak 99]

37 Back to Projection Intrinsics Matrix Our goal: Go from world coordinates (in mm) to image coordinates (in pixels). We need to introduce a scaling factor from mm to pixel. This depends on the physical focal length and the size of a single pixel on the sensor in mm. W is sensor size (in mm) and O is field of view. Also, let s shift everything so the image is centered about the camera optical axis:

38 Homogeneous Coordinates Our projection equation is nonlinear: (It has division in it) Let s take a different representation for the cartesian 2D and 3D points: Now our projection equation is linear: To go back to cartesian we divide by the added component.

39 Homogeneous Coordinates Why are they so useful? - Adding translation is trivial and turns our transformation to a linear matrix operation. - In fact, some transformations can only work with homogeneous coordinates. - Can be used to represent points at infinity, like vanishing points (by setting ) or directions. - They can do other tricks like find line intersections easily, or fetch a ball.

40 Camera Extrinsic Parameters So far we assumed the camera is at the origin But that s often not the case. rotation translation A more common notation:

41 Beyond Pinhole Radial Distortions Wholly nonlinear, but can be simple enough: Can be negated by de-warping the image.

42 Image Formation: Lenses

43 Do We Need a Lens? The eye has a lens, so... that s a good yes indicator. Lenses focus light, and allow us to image objects that are far away (or very close). The pinhole model still holds!

44 Thin Lens Circle of confusion (blur), relates to Depth of Field

45 Focal Length Field Of View

46 Lenses Add Distortions Chromatic Aberration Spherical Aberration Vignetting

47 Wrap Up