Image Formation: Camera Model Ruigang Yang COMP 684 Fall 2005, CS684-IBMR
Outline Camera Models Pinhole Perspective Projection Affine Projection Camera with Lenses Digital Image Formation The Human Eye Adapted from slides prepared by Marc Pollefeys (UNC-CH) Fall 2005, CS684-IBMR 2
What is an Image? A Two-dimensional pattern of brightness values, formed by the projection of 3D objects. Figure from US Navy Manual of Basic Optics and Optical Instruments, prepared by Bureau of Naval Personnel. Reprinted by Dover Publications, Inc., 969. Fall 2005, CS684-IBMR 3
Animal eye: a looonnng time ago. Photographic camera: Niepce, 86. Pinhole perspective projection: Mo-Ti, 5 th Century BC Camera obscura: 7 th Century. Fall 2005, CS684-IBMR 4
Distant objects appear smaller Fall 2005, CS684-IBMR 5
Parallel lines meet vanishing point Fall 2005, CS684-IBMR 6
Vanishing points H VPL VPR VP VP 2 Different directions correspond different vanishing points VP 3 Fall 2005, CS684-IBMR 7
Geometric properties of projection Points go to points Lines go to lines Planes go to whole image or half-plane Polygons go to polygons Degenerate cases: line through focal point yields point plane through focal point yields line Fall 2005, CS684-IBMR 8
Pinhole Perspective Equation x y ' ' = = f f ' ' x z y z Fall 2005, CS684-IBMR 9
Affine Projection Model I: Weak perspective projection x' mx f ' where m = y' = = my is the magnification. z 0 When the scene relief is small compared its distance from the Camera, m = constant weak perspective projection. Fall 2005, CS684-IBMR 0
Affine Projection Model II: Orthographic projection x' = y' = x y When the camera is at a (roughly constant) distance from the scene, take m=. Fall 2005, CS684-IBMR
Planar pinhole perspective Orthographic projection Spherical pinhole perspective Fall 2005, CS684-IBMR 2
Limits for pinhole cameras Pinhole Size Fall 2005, CS684-IBMR 3
Outline Camera Models Pinhole Perspective Projection Affine Projection Camera with Lenses Digital Image Formation The Human Eye Adapted from slides prepared by Marc Pollefeys (UNC-CH) Fall 2005, CS684-IBMR 4
Why Lenses? Snell s law n = sinα n2 sin α Descartes law 2 Fall 2005, CS684-IBMR 5
Fall 2005, CS684-IBMR 6 Paraxial (or first-order) optics R γ β α h d h + = + = 2 2 2 R β γ α d h h = = = + 2 2 R R d h h n h d h n Snell s law Small angles: 2 2 α α n n = R n n d n d n 2 2 2 = + 2 2 sin sin α α n n =
Fall 2005, CS684-IBMR 7 Thin Lenses spherical lens surfaces; incoming light ± parallel to axis; thickness << radii; same refractive index on both sides R n Z n Z * = + R n Z Z n = + ' * Z R n Z n * = ' * Z R n Z n = R n n d n d n 2 2 2 = + ) 2( and ' = = n R f f z z ' Z Z R n R n =
Thin Lenses x' = y' = z' z' x z y z where z' z = f and R 2( n ) Fall 2005, CS684-IBMR 8 http://www.phy.ntnu.edu.tw/java/lens/lens_e.html f =
Thick Lens Fall 2005, CS684-IBMR 9
The depth-of-field http://www.illuminatelabs.com/illuminatelabs_sep2/illuminatelabs_new/gallery_pic2_static.htm Fall 2005, CS684-IBMR 20
Fall 2005, CS684-IBMR 2 Depth of Field f b d f Z f Z Z Z Z Z + = = / ) ( 0 0 0 0 0 0 decreases with d, increases with Z 0 strike a balance between incoming light and sharp depth range
Deviations from the lens model Ideal lens model makes three assumptions :. all rays from a point are focused onto image point 2. all image points in a single plane 3. magnification is constant deviations from this ideal are aberrations Fall 2005, CS684-IBMR 22
Aberration Two Types Geometrical: Imperfections in lens http://www.opticsforteens.org/what/geo-pg7.asp Chromatic: refractive index function of wavelength From http://www.mellesgriot.com/products/optics/fo_3_2_6.htm Fall 2005, CS684-IBMR 23
Geometrical aberrations Spherical aberration Astigmatism Distortion Coma aberrations are reduced by combining lenses Fall 2005, CS684-IBMR 24
Spherical aberration rays parallel to the axis do not converge outer portions of the lens yield smaller focal lengths Fall 2005, CS684-IBMR 25
Astigmatism Different focal length for inclined rays Fall 2005, CS684-IBMR 26
Distortion magnification/focal length different for different angles of inclination Pincushion Distortion Pinch and stretch the image of a square at the corners Barrel Distortion Push the corners of an image of a square in towards the center Fall 2005, CS684-IBMR 27
Distortion Can be corrected if parameters are know Fall 2005, CS684-IBMR 28
Coma point off the axis depicted as comet shaped blob Fall 2005, CS684-IBMR 29
Chromatic aberration Rays of different wavelengths focused in different planes http://www.dpreview.com/learn/glossary/optical/chromatic_aberrations_0.htm Fall 2005, CS684-IBMR 30
Photographs (Niepce, La Table Servie, 822) Milestones: Collection Harlingue-Viollet. Daguerreotypes (839) Photographic Film (Eastman,889) Cinema (Lumière Brothers,895) Color Photography (Lumière Brothers, 908) Television (Baird, Farnsworth, Zworykin, 920s) CCD Devices (970) more recently CMOS Fall 2005, CS684-IBMR 3
Outline Camera Models Pinhole Perspective Projection Affine Projection Camera with Lenses Digital Image Formation The Human Eye Fall 2005, CS684-IBMR 32
Image Sensors Fall 2005, CS684-IBMR 33
An Example of Digital Image Acquisition Process Fall 2005, CS684-IBMR 34
Digital Image Generation Fall 2005, CS684-IBMR 35
Image Sampling and Quantization Fall 2005, CS684-IBMR 36
Cameras Sensors Two Types CCD (Charge-Coupled Devices) CMOS (Complementary Metal-Oxide Semiconductor) Fall 2005, CS684-IBMR 37
CCD Separate photo sensor at regular positions Most-widely used Progressive vs. Interlaced Progressive: read out every line in order. Interlaced: first read out the odd lines, then read out the even lines. They are reintegrated through image processing. Fall 2005, CS684-IBMR 38
CMOS Same sensor elements as CCD Each photo sensor has its own amplifier More noise (reduced by subtracting black image) Lower sensitivity (lower fill rate) Uses standard CMOS technology Allows to put other components on chip Smart pixels Foveon 4k x 4k sensor 0.8µ process 70M transistors Fall 2005, CS684-IBMR 39
CCD vs. CMOS Mature technology Specific technology High production cost High power consumption Higher fill rate Blooming Sequential readout Recent technology Standard IC technology Cheap Low power Less sensitive Per pixel amplification Random pixel access Smart pixels On chip integration with other components Fall 2005, CS684-IBMR 40
Color cameras We consider 3 concepts:. Prism (with 3 sensors) 2. Filter mosaic 3. Filter wheel and X3 Fall 2005, CS684-IBMR 4
Prism colour camera Separate light in 3 beams using dichroic prism Requires 3 sensors & precise alignment Good color separation http://www.duncantech.com/images/image58.jpg Fall 2005, CS684-IBMR 42
Filter mosaic Coat filter directly on sensor http://www.siliconimaging.com/images/a_bayer_pattern_diagram.gif Fall 2005, CS684-IBMR 43
Filter mosaic De-mosaicing (obtain full color & full resolution image) Problem Fall 2005, CS684-IBMR 44
Filter wheel Rotate multiple filters in front of lens Allows more than 3 colour bands Only suitable for static scenes Fall 2005, CS684-IBMR 45
Prism vs. mosaic vs. wheel approach Prism Mosaic Wheel # sensors 3 Separation High Average Good Cost High Low Average Frame rate High High Low Artifacts Low Aliasing Motion Bands 3 3 3 or more High-end cameras Low-end cameras Scientific applications Fall 2005, CS684-IBMR 46
New color CMOS sensor Foveon sx3 better image quality smarter pixels Fall 2005, CS684-IBMR 47
Outline Camera Models Pinhole Perspective Projection Affine Projection Camera with Lenses Digital Image Formation The Human Eye Fall 2005, CS684-IBMR 48
The Human Eye Helmoltz s Schematic Eye http://psych.athabascau.ca/html/psych402/biotutorials/22/intro.shtml Fall 2005, CS684-IBMR 49
The distribution of rods and cones across the retina Reprinted from Foundations of Vision, by B. Wandell, Sinauer Associates, Inc., (995). 995 Sinauer Associates, Inc. Cones in the fovea Rods and cones in the periphery Reprinted from Foundations of Vision, by B. Wandell, Sinauer Associates, Inc., (995). 995 Sinauer Associates, Inc. Fall 2005, CS684-IBMR 50
Interesting Facts about Human Visual System Simultaneous Contrast Fall 2005, CS684-IBMR 5
Mach Band Effect Perceived Brightness changes around strong edges. Fall 2005, CS684-IBMR 52
Visual Masking Threshold intensity increases at background with large non-uniform spatial, temporal changes. Fall 2005, CS684-IBMR 53
Optical Illusions Fall 2005, CS684-IBMR 54
Next Class Multi-view Geometry and Stereo Fall 2005, CS684-IBMR 55