Lecture 2 Camera Models

Lecture 2 Camera Models Professor Silvio Savarese Computational Vision and Geometr Lab Silvio Savarese Lecture 2 - -Jan-8

Lecture 2 Camera Models Pinhole cameras Cameras lenses The geometr of pinhole cameras Reading: [FP] Chapter, Geometric Camera Models [HZ] Chapter 6 Camera Models Silvio Savarese Some slides in this lecture are courtes to Profs. J. Ponce, S. Seit, F-F Li Lecture 2 - -Jan-8

How do we see the world? Let s design a camera Idea : put a piece of film in front of an object Do we get a reasonable image?

Pinhole camera Apertur e Add a barrier to block off most of the ras This reduces blurring The opening known as the aperture

Some histor Milestones: Leonardo da Vinci (452-59): first record of camera obscura (52)

Some histor Milestones: Leonardo da Vinci (452-59): first record of camera obscura Johann Zahn (685): first portable camera

Some histor Milestones: Leonardo da Vinci (452-59): first record of camera obscura Johann Zahn (685): first portable camera Joseph Nicéphore Niépce (822): first photo - birth of photograph Photograph (Niépce, La Table Servie, 822)

Some histor Milestones: Leonardo da Vinci (452-59): first record of camera obscura Johann Zahn (685): first portable camera Joseph Nicéphore Niépce (822): first photo - birth of photograph Daguerréotpes (839) Photographic Film (Eastman, 889) Cinema (Lumière Brothers, 895) Color Photograph (Lumière Brothers, 98) Photograph (Niépce, La Table Servie, 822)

Let s also not forget Motu (468-376 BC) Oldest eistent book on geometr in Aristotle (384-322 BC) Also: Plato, Euclid Al-Kindi (c. 8 873) Ibn al-haitham (965-4)

Pinhole camera Pinhole perspective projection f o f = focal length o = aperture = pinhole = center of the camera

ï ï î ï í ì = = f f û ù ë é = P û ù ë é = P Pinhole camera Derived using similar triangles [Eq. ] f

Pinhole camera f k f i P = [, ] [Eq. 2] = P =[, f ] O f

Pinhole camera Is the sie of the aperture important? Kate lauka

Shrinking aperture sie -What happens if the aperture is too small? -Less light passes through Adding lenses!

Cameras Lenses P image P A lens focuses light onto the film

Cameras Lenses P image P Out of focus A lens focuses light onto the film There is a specific distance at which objects are in focus Related to the concept of depth of field

Cameras Lenses A lens focuses light onto the film There is a specific distance at which objects are in focus Related to the concept of depth of field

Cameras Lenses focal point f A lens focuses light onto the film All ras parallel to the optical (or principal) ais converge to one point (the focal point) on a plane located at the focal length f from the center of the lens. Ras passing through the center are not deviated

Paraial refraction model Z p = [, ] From Snell s law: - [Eq. 3] f ì ï = = f + o í ï R = f = 2(n -) î o [Eq. 4]

Issues with lenses: Radial Distortion Deviations are most noticeable for ras that pass through the edge of the lens No distortion Pin cushion Barrel (fishee lens) Image magnification decreases with distance from the optical ais

Lecture 2 Camera Models Pinhole cameras Cameras lenses The geometr of pinhole cameras Intrinsic Etrinsic Silvio Savarese Lecture 2 - -Jan-8

Pinhole perspective projection Pinhole camera f = focal length o = center of the camera 2 3 Â Â E ï ï î ï í ì = = f f û ù ë é = P û ù ë é = P [Eq. ] f

From retina plane to images Retina plane f Digital image Piels, bottom-left coordinate sstems

Coordinate sstems f c. Off set C =[c, c ] c (,, ) (f + c, f + c ) [Eq. 5]

Converting to piels f c C=[c, c ] c. Off set 2. From metric to piels (,, ) (f k + c, f l + c ) a b [Eq. 6] Units: k,l : piel/m f : m Non-square piels a, b : piel

Is this projective transformation linear? f c P = (,, ) P = (α + c, β + c ) [Eq. 7] C=[c, c ] c Is this a linear transformation? No division b is nonlinear Can we epress it in a matri form?

Homogeneous coordinates EàH homogeneous image coordinates homogeneous scene coordinates Converting back from homogeneous coordinates HàE

Projective transformation in the homogenous coordinate sstem! P h = α + c β + c! = α c β c! P h [Eq.8] Homogenous Euclidian P P = (α + c, β + c h )! M = α c β c

The Camera Matri f Camera matri K [Eq.9] P = M P = K! I P! P = α c β c!

Camera Skewness f c c P! = α α cotθ c β sinθ c ( ( ( ( ( ( ( ( ( θ C=[c, c ] How man degree does K have? 5 degrees of freedom!

Canonical Projective Transformation P =! =!!! P i = P = M P M 3 H 4 Â Â [Eq.]

Lecture 2 Camera Models Pinhole cameras Cameras lenses The geometr of pinhole cameras Intrinsic Etrinsic Other camera models Silvio Savarese Lecture 2 - -Jan-8

World reference sstem f R,T j w k w O w i w The mapping so far is defined within the camera reference sstem What if an object is represented in the world reference sstem? Need to introduce an additional mapping from world ref sstem to camera ref sstem

3D Rotation of Points Rotation around the coordinate aes, counter-clockwise: û ù ë é - = û ù ë é - = û ù ë é - = cos sin sin cos ) ( cos sin sin cos ) ( cos sin sin cos ) ( g g g g g b b b b b a a a a a R R R p Y p g P R 4 4 A rotation matri in 3D has 3 degree of freedom

2D Translation Please refer to CA session on transformations for more details P P t

2D Translation Equation t P t P P = (, ) t = ( t, t ) t P = P + t = ( + t, + t )

2D Translation using Homogeneous Coordinates û ù ë é û ù ë é = û ù ë é + + t t t t P P = (, ) (,,) = I t!! = T! P t t P t

Scaling P P

Scaling Equation P s P s û ù ë é û ù ë é = û ù ë é s s s s P P = (, ) (,,) S = S!! = S! ) s, (s ), ( = = P P

Rotation P P

Rotation Equations Counter-clockwise rotation b an angle θ = cosq -sinq P q P é ù ë û = = cosq + sinq écosq - sin q ësin q cosq ù û é ë ù û P = R P How man degrees of freedom? P cosθ sinθ sinθ cosθ ( ( ( ( ( (

Scale + Rotation + Translation P t t cosθ sinθ sinθ cosθ s s écosq -sinq tùés ùéù = sinq cosq t s ë ûë ûëû! = R t! S!! = R S t! If s = s, this is a similarit transformation

3D Rotation of Points Rotation around the coordinate aes, counter-clockwise: û ù ë é - = û ù ë é - = û ù ë é - = cos sin sin cos ) ( cos sin sin cos ) ( cos sin sin cos ) ( g g g g g b b b b b a a a a a R R R P P P R 4 4 A rotation matri in 3D has 3 degrees of freedom R = R (α) R (β) R (γ)

3D Translation of Points T P T P T =! T T T T P I T 4 4 A translation vector in 3D has 3 degrees of freedom

3D Translation and Rotation R = R (α) R (β) R (γ) T =! T T T P R T 4 4

World reference sstem f R,T P j w k w O w P In 4D homogeneous coordinates: P = K! I P = K! I Internal parameters!! P = R T R T 4 4 P w i w 4 4 P w Eternal parameters = K! R T M! w w w P w [Eq.]

The projective transformation f R,T P j w k w O w P i w P 3 = M 3 4 P w = K 3 3 R T 3 4 P w4 How man degrees of freedom? 5 + 3 + 3 =!

The projective transformation f R,T P j w k w O w P i w P 3 = M P w = K 3 3 R T! = m m 2 m 3! P W = m P W m 2 P W m 3 P W E ém ù P w4 M = 3 4 m2 ëm3 û ( m P w m 3 P w, m 2 P w m 3 P w ) [Eq.2]

Theorem (Faugeras, 993) [ T] = [ K R KT ] [ A b] M = K R = [Eq.3] A = éa a ë a 2 3 ù û

Properties of projective transformations Points project to points Lines project to lines Distant objects look smaller

Properties of Projection Angles are not preserved Parallel lines meet! Parallel lines in the world intersect in the image at a vanishing point

Horion line (vanishing line)

One-point perspective Masaccio, Trinit, Santa Maria Novella, Florence, 425-28 Credit slide S. Laebnik

Net lecture How to calibrate a camera?

Supplemental material

Thin Lenses [FP] sec., page 8. o = f + o f = R 2(n -) Snell s law: Focal length n sin a = n 2 sin a 2 Small angles: n a» n 2 a 2 n = n (lens) n = (air) ì ï í ï ïî = =

Horion line (vanishing line) l horion