Lecture 2 Camera Models

Lecture 2 Camera Models Professor Silvio Savarese Computational Vision and Geometr Lab Silvio Savarese Lecture 2-4-Jan-4

Announcements Prerequisites: an questions? This course requires knowledge of linear algebra, probabilit, statistics, machine learning and computer vision, as well as decent programming skills. Though not an absolute requirement, it is encouraged and preferred that ou have at least taken either CS22 or CS229 or CS3A or have equivalent knowledge. Topics such as linear filters, feature detectors and descriptors, low level segmentation, tracking, optical flow, clustering and PCA/LDA techniques for recognition won t be covered in CS23 We will provide links to background material related to CS3A (or discuss during TA sessions) so students can refresh or stud those topics if needed We will leverage concepts from machine learning (CS229) (e.g., SVM, basic Baesian inference, clustering, etc ) which we won t cover in this class either. Again, we will suppl links to related material for background reading. Silvio Savarese Lecture 2-4-Jan-4

Announcements Net TA session: Fridas from 2:5-3:5pm Silvio Savarese Lecture 2-4-Jan-4

Lecture 2 Camera Models Pinhole cameras Cameras & lenses The geometr of pinhole cameras Other camera models Reading: Silvio Savarese [FP] Chapter Cameras [FP] Chapter 2 Geometric Camera Models [HZ] Chapter 6 Camera Models Some slides in this lecture are courtes to Profs. J. Ponce, S. Seitz, F-F Li Lecture 2-4-Jan-4

How do we see the world? Let s design a camera Idea : put a piece of film in front of an object Do we get a reasonable image?

Pinhole camera Add a barrier to block off most of the ras This reduces blurring The opening known as the aperture

Some histor Milestones: Leonardo da Vinci (452-59): first record of camera obscura

Some histor Milestones: Leonardo da Vinci (452-59): first record of camera obscura Johann Zahn (685): first portable camera

Some histor Milestones: Leonardo da Vinci (452-59): first record of camera obscura Johann Zahn (685): first portable camera Joseph Nicephore Niepce (822): first photo - birth of photograph Photograph (Niepce, La Table Servie, 822)

Some histor Milestones: Leonardo da Vinci (452-59): first record of camera obscura Johann Zahn (685): first portable camera Joseph Nicephore Niepce (822): first photo - birth of photograph Daguerréotpes (839) Photographic Film (Eastman, 889) Cinema (Lumière Brothers, 895) Color Photograph (Lumière Brothers, 98) Photograph (Niepce, La Table Servie, 822)

Let s also not forget Motzu (468-376 BC) Oldest eistent book on geometr in China Aristotle (384-322 BC) Also: Plato, Euclid Al-Kindi (c. 8 873) Ibn al-haitham (965-4)

Pinhole camera Pinhole perspective projection f o f = focal length o = aperture = pinhole = center of the camera

z f ' z f ' P z P Pinhole camera Derived using similar triangles

Pinhole camera k f i P = [, z] P =[, f ] O f z

Pinhole camera f f Common to draw image plane in front of the focal point. What s the transformation between these 2 planes? ' ' f f z z

Pinhole camera Is the size of the aperture important? Kate lazuka

Shrinking aperture size - Ras are mied up -Wh the aperture cannot be too small? -Less light passes through -Diffraction effect Adding lenses!

Cameras & Lenses A lens focuses light onto the film

Cameras & Lenses circle of confusion A lens focuses light onto the film There is a specific distance at which objects are in focus Related to the concept of depth of field

Cameras & Lenses A lens focuses light onto the film There is a specific distance at which objects are in focus Related to the concept of depth of field

Cameras & Lenses focal point f A lens focuses light onto the film All parallel ras converge to one point on a plane located at the focal length f Ras passing through the center are not deviated

Thin Lenses For details see lecture on cameras in CS3A z o z' f z o f R 2(n ) Snell s law: Focal length n sin = n 2 sin 2 Small angles: n n 2 2 n = n (lens) n = (air) ' ' z' z' z z

Issues with lenses: Radial Distortion Pin cushion Barrel (fishee lens)

Issues with lenses: Radial Distortion Deviations are most noticeable for ras that pass through the edge of the lens No distortion Pin cushion Barrel (fishee lens) Image magnification decreases with distance from the optical ais

Lecture 2 Camera Models Pinhole cameras Cameras & lenses The geometr of pinhole cameras Intrinsic Etrinsic Other camera models Silvio Savarese Lecture 2-4-Jan-4

Pinhole camera Pinhole perspective projection f o f = focal length o = center of the camera (,, z) 3 E 2 (f z,f ) z

From retina plane to images Piels, bottom-left coordinate sstems

Coordinate sstems c c

Converting to piels c. Off set C=[c, c ] c (,, z) (f c, f c ) z z

Converting to piels c c. Off set 2. From metric to piels (,, z) (f k c, f l c ) z z C=[c, c ] Units: k,l : piel/m f : m Non-square piels, : piel

Converting to piels c (,, z) ( c, c z z ) C=[c, c ] c Matri form? A related question: Is this a linear transformation?

(,, z) (f,f ) z z Is this a linear transformation? No division b z is nonlinear How to make it linear?

Homogeneous coordinates For details see lecture on transformations in CS3A homogeneous image coordinates homogeneous scene coordinates Converting from homogeneous coordinates

Camera Matri ) c z, c z ( z),, ( z c c z z c z c X c c C=[c, c ]

Perspective Projection Transformation z f f z f f X z f z f X i X M X M 3 H 4

X M X X K I Camera Matri z c c X Camera matri K

Finite projective cameras z c c s X Skew parameter c c C=[c, c ] K has 5 degrees of freedom!

Lecture 2 Camera Models Pinhole cameras Cameras & lenses The geometr of pinhole cameras Intrinsic Etrinsic Other camera models Silvio Savarese Lecture 2-4-Jan-4

World reference sstem R,T j w k w O w i w The mapping so far is defined within the camera reference sstem What if an object is represented in the world reference sstem

2D Translation For details see lecture on transformations in CS3A See also TA session on Frida P' P t

2D Translation Equation t P t P P (, ) t ( t, t ) t P ' P t ( t, t )

2D Translation using Homogeneous Coordinates ' t t t t P,), ( ), (,), ( ), ( t t t t t P t P I t P T P P t t P t

Scaling P' P

Scaling Equation P s P s ' s s s s P,), ( ), ( ',), ( ), ( s s s s P P S ' S P S P ) s, (s ' ), ( P P

Scaling & Translating P'' P P ' P '' S P T P ' P '' T P ' T ( S P) T S P A P

Scaling & Translating t s P '' T S P t s s t s t s t S t s t Α

Rotation P' P

Rotation Equations Counter-clockwise rotation b an angle θ P P ' ' ' cos sin ' cos sin cos sin sin cos P' R P

Degrees of Freedom ' cos sin ' sin cos R is 22 4 elements Note: R is an orthogonal matri and satisfies man interesting properties: R R T R T R I det( R)

Rotation + Scale + Translation P ' ( T R S) P t cos sin s P ' T R S P t sin cos s cos sin t s sin cos t s R t S R S t If s = s, this is a similarit transformation!

3D Rotation of Points Rotation around the coordinate aes, counter-clockwise: cos sin sin cos ) ( cos sin sin cos ) ( cos sin sin cos ) ( z R R R p Y p z

World reference sstem R,T j w X k w O w X i w In 4D homogeneous coordinates: X R T 4 4 X w I X X K Internal parameters R T ' K I X w 44 K Eternal parameters R T X w M

Projective cameras R,T j w X k w O w X i w 3 w 4 X 3 M 34 X w K 3 R T 34 X K s c c How man degrees of freedom? 5 + 3 + 3 =!

Projective cameras O w i w k w j w R,T w 3 X X M w 4 4 3 3 3 X T R K 3 2 m m m M W W W W m m m m m m X X X X 3 2 3 2 ), ( 3 2 3 w w w w X X X X m m m m E X X

Theorem (Faugeras, 993) ] [ b A K T K R T R K M 3 2 a a a A c c s K l f k; f

Properties of Projection Points project to points Lines project to lines Distant objects look smaller

Properties of Projection Angles are not preserved Parallel lines meet! Vanishing point

One-point perspective Masaccio, Trinit, Santa Maria Novella, Florence, 425-28 Credit slide S. Lazebnik

Net lecture How to calibrate a camera?