Image Formation: Camera Model

Transcription

1 Image Formation: Camera Model Ruigang Yang COMP 684 Fall 2005, CS684-IBMR

2 Outline Camera Models Pinhole Perspective Projection Affine Projection Camera with Lenses Digital Image Formation The Human Eye Adapted from slides prepared by Marc Pollefeys (UNC-CH) Fall 2005, CS684-IBMR 2

3 What is an Image? A Two-dimensional pattern of brightness values, formed by the projection of 3D objects. Figure from US Navy Manual of Basic Optics and Optical Instruments, prepared by Bureau of Naval Personnel. Reprinted by Dover Publications, Inc., 969. Fall 2005, CS684-IBMR 3

4 Animal eye: a looonnng time ago. Photographic camera: Niepce, 86. Pinhole perspective projection: Mo-Ti, 5 th Century BC Camera obscura: 7 th Century. Fall 2005, CS684-IBMR 4

5 Distant objects appear smaller Fall 2005, CS684-IBMR 5

6 Parallel lines meet vanishing point Fall 2005, CS684-IBMR 6

7 Vanishing points H VPL VPR VP VP 2 Different directions correspond different vanishing points VP 3 Fall 2005, CS684-IBMR 7

8 Geometric properties of projection Points go to points Lines go to lines Planes go to whole image or half-plane Polygons go to polygons Degenerate cases: line through focal point yields point plane through focal point yields line Fall 2005, CS684-IBMR 8

9 Pinhole Perspective Equation x y ' ' = = f f ' ' x z y z Fall 2005, CS684-IBMR 9

10 Affine Projection Model I: Weak perspective projection x' mx f ' where m = y' = = my is the magnification. z 0 When the scene relief is small compared its distance from the Camera, m = constant weak perspective projection. Fall 2005, CS684-IBMR 0

11 Affine Projection Model II: Orthographic projection x' = y' = x y When the camera is at a (roughly constant) distance from the scene, take m=. Fall 2005, CS684-IBMR

12 Planar pinhole perspective Orthographic projection Spherical pinhole perspective Fall 2005, CS684-IBMR 2

13 Limits for pinhole cameras Pinhole Size Fall 2005, CS684-IBMR 3

14 Outline Camera Models Pinhole Perspective Projection Affine Projection Camera with Lenses Digital Image Formation The Human Eye Adapted from slides prepared by Marc Pollefeys (UNC-CH) Fall 2005, CS684-IBMR 4

15 Why Lenses? Snell s law n = sinα n2 sin α Descartes law 2 Fall 2005, CS684-IBMR 5

16 Fall 2005, CS684-IBMR 6 Paraxial (or first-order) optics R γ β α h d h + = + = R β γ α d h h = = = R R d h h n h d h n Snell s law Small angles: 2 2 α α n n = R n n d n d n = sin sin α α n n =

17 Fall 2005, CS684-IBMR 7 Thin Lenses spherical lens surfaces; incoming light ± parallel to axis; thickness << radii; same refractive index on both sides R n Z n Z * = + R n Z Z n = + ' * Z R n Z n * = ' * Z R n Z n = R n n d n d n = + ) 2( and ' = = n R f f z z ' Z Z R n R n =

18 Thin Lenses x' = y' = z' z' x z y z where z' z = f and R 2( n ) Fall 2005, CS684-IBMR 8 f =

19 Thick Lens Fall 2005, CS684-IBMR 9

20 The depth-of-field Fall 2005, CS684-IBMR 20

21 Fall 2005, CS684-IBMR 2 Depth of Field f b d f Z f Z Z Z Z Z + = = / ) ( decreases with d, increases with Z 0 strike a balance between incoming light and sharp depth range

22 Deviations from the lens model Ideal lens model makes three assumptions :. all rays from a point are focused onto image point 2. all image points in a single plane 3. magnification is constant deviations from this ideal are aberrations Fall 2005, CS684-IBMR 22

23 Aberration Two Types Geometrical: Imperfections in lens Chromatic: refractive index function of wavelength From Fall 2005, CS684-IBMR 23

24 Geometrical aberrations Spherical aberration Astigmatism Distortion Coma aberrations are reduced by combining lenses Fall 2005, CS684-IBMR 24

25 Spherical aberration rays parallel to the axis do not converge outer portions of the lens yield smaller focal lengths Fall 2005, CS684-IBMR 25

26 Astigmatism Different focal length for inclined rays Fall 2005, CS684-IBMR 26

27 Distortion magnification/focal length different for different angles of inclination Pincushion Distortion Pinch and stretch the image of a square at the corners Barrel Distortion Push the corners of an image of a square in towards the center Fall 2005, CS684-IBMR 27

28 Distortion Can be corrected if parameters are know Fall 2005, CS684-IBMR 28

29 Coma point off the axis depicted as comet shaped blob Fall 2005, CS684-IBMR 29

30 Chromatic aberration Rays of different wavelengths focused in different planes Fall 2005, CS684-IBMR 30

31 Photographs (Niepce, La Table Servie, 822) Milestones: Collection Harlingue-Viollet. Daguerreotypes (839) Photographic Film (Eastman,889) Cinema (Lumière Brothers,895) Color Photography (Lumière Brothers, 908) Television (Baird, Farnsworth, Zworykin, 920s) CCD Devices (970) more recently CMOS Fall 2005, CS684-IBMR 3

32 Outline Camera Models Pinhole Perspective Projection Affine Projection Camera with Lenses Digital Image Formation The Human Eye Fall 2005, CS684-IBMR 32

33 Image Sensors Fall 2005, CS684-IBMR 33

34 An Example of Digital Image Acquisition Process Fall 2005, CS684-IBMR 34

35 Digital Image Generation Fall 2005, CS684-IBMR 35

36 Image Sampling and Quantization Fall 2005, CS684-IBMR 36

37 Cameras Sensors Two Types CCD (Charge-Coupled Devices) CMOS (Complementary Metal-Oxide Semiconductor) Fall 2005, CS684-IBMR 37

38 CCD Separate photo sensor at regular positions Most-widely used Progressive vs. Interlaced Progressive: read out every line in order. Interlaced: first read out the odd lines, then read out the even lines. They are reintegrated through image processing. Fall 2005, CS684-IBMR 38

39 CMOS Same sensor elements as CCD Each photo sensor has its own amplifier More noise (reduced by subtracting black image) Lower sensitivity (lower fill rate) Uses standard CMOS technology Allows to put other components on chip Smart pixels Foveon 4k x 4k sensor 0.8µ process 70M transistors Fall 2005, CS684-IBMR 39

40 CCD vs. CMOS Mature technology Specific technology High production cost High power consumption Higher fill rate Blooming Sequential readout Recent technology Standard IC technology Cheap Low power Less sensitive Per pixel amplification Random pixel access Smart pixels On chip integration with other components Fall 2005, CS684-IBMR 40

41 Color cameras We consider 3 concepts:. Prism (with 3 sensors) 2. Filter mosaic 3. Filter wheel and X3 Fall 2005, CS684-IBMR 4

42 Prism colour camera Separate light in 3 beams using dichroic prism Requires 3 sensors & precise alignment Good color separation Fall 2005, CS684-IBMR 42

43 Filter mosaic Coat filter directly on sensor Fall 2005, CS684-IBMR 43

44 Filter mosaic De-mosaicing (obtain full color & full resolution image) Problem Fall 2005, CS684-IBMR 44

45 Filter wheel Rotate multiple filters in front of lens Allows more than 3 colour bands Only suitable for static scenes Fall 2005, CS684-IBMR 45

46 Prism vs. mosaic vs. wheel approach Prism Mosaic Wheel # sensors 3 Separation High Average Good Cost High Low Average Frame rate High High Low Artifacts Low Aliasing Motion Bands or more High-end cameras Low-end cameras Scientific applications Fall 2005, CS684-IBMR 46

47 New color CMOS sensor Foveon sx3 better image quality smarter pixels Fall 2005, CS684-IBMR 47

48 Outline Camera Models Pinhole Perspective Projection Affine Projection Camera with Lenses Digital Image Formation The Human Eye Fall 2005, CS684-IBMR 48

49 The Human Eye Helmoltz s Schematic Eye Fall 2005, CS684-IBMR 49

50 The distribution of rods and cones across the retina Reprinted from Foundations of Vision, by B. Wandell, Sinauer Associates, Inc., (995). 995 Sinauer Associates, Inc. Cones in the fovea Rods and cones in the periphery Reprinted from Foundations of Vision, by B. Wandell, Sinauer Associates, Inc., (995). 995 Sinauer Associates, Inc. Fall 2005, CS684-IBMR 50

51 Interesting Facts about Human Visual System Simultaneous Contrast Fall 2005, CS684-IBMR 5

52 Mach Band Effect Perceived Brightness changes around strong edges. Fall 2005, CS684-IBMR 52

53 Visual Masking Threshold intensity increases at background with large non-uniform spatial, temporal changes. Fall 2005, CS684-IBMR 53

54 Optical Illusions Fall 2005, CS684-IBMR 54

55 Next Class Multi-view Geometry and Stereo Fall 2005, CS684-IBMR 55