Panoramic Image Mosaics

Save this PDF as:

Size: px
Start display at page:

Transcription

1 Panoramic Image Mosaics Image Stitching Computer Vision CSE 576, Spring 2008 Richard Szeliski Microsoft Research Full screen panoramas (cubic): Mars: New Years Eve: Richard Szeliski Image Stitching 2 Gigapixel panoramas & images Image Mosaics = Mapping / Tourism / WWT Medical Imaging Richard Szeliski Image Stitching 3 Goal: Stitch together several images into a seamless composite Richard Szeliski Image Stitching 4

2 Today s lecture Image alignment and stitching motion models image warping point-based alignment complete mosaics (global alignment) compositing and blending ghost and parallax removal Readings Szeliski, CVAA: Chapter 3.5: Image warping Chapter 5.1: Feature-based alignment (in preparation) Chapter 8.1: Motion models Chapter 8.2: Global alignment Chapter 8.3: Compositing Recognizing Panoramas, Brown & Lowe, ICCV 2003 Szeliski & Shum, SIGGRAPH'97 Richard Szeliski Image Stitching 5 Richard Szeliski Image Stitching 6 Motion models Motion models What happens when we take two images with a camera and try to align them? translation? rotation? scale? affine? perspective? see interactive demo (VideoMosaic) Richard Szeliski Image Stitching 8

3 Image Warping Image Warping image filtering: change range of image g(x) = h(f(x)) f x image warping: change domain of image g(x) = f(h(x)) f h h f f x x Richard Szeliski Image Stitching 10 x Image Warping image filtering: change range of image g(x) = h(f(x)) f g h image warping: change domain of image g(x) = f(h(x)) f h Richard Szeliski Image Stitching 11 g Parametric (global) warping Examples of parametric warps: translation rotation aspect affine perspective cylindrical Richard Szeliski Image Stitching 12

4 2D coordinate transformations translation: x = x + t x = (x,y) rotation: x = R x + t similarity: x = s R x + t affine: x = A x + t perspective: x H x x = (x,y,1) (x is a homogeneous coordinate) These all form a nested group (closed w/ inv.) Image Warping Given a coordinate transform x = h(x) and a source image f(x), how do we compute a transformed image g(x ) =f(h(x))? h(x) x x f(x) g(x ) Richard Szeliski Image Stitching 13 Richard Szeliski Image Stitching 14 Forward Warping Send each pixel f(x) to its corresponding location x = h(x) in g(x ) What if pixel lands between two pixels? Forward Warping Send each pixel f(x) to its corresponding location x = h(x) in g(x ) What if pixel lands between two pixels? Answer: add contribution to several pixels, normalize later (splatting) h(x) h(x) x f(x) x g(x ) x f(x) x g(x ) Richard Szeliski Image Stitching 15 Richard Szeliski Image Stitching 16

5 Inverse Warping Get each pixel g(x ) from its corresponding location x = h(x) in f(x) What if pixel comes from between two pixels? Inverse Warping Get each pixel g(x ) from its corresponding location x = h(x) in f(x) What if pixel comes from between two pixels? Answer: resample color value from interpolated (prefiltered) source image h(x) x x f(x) g(x ) x x f(x) g(x ) Richard Szeliski Image Stitching 17 Richard Szeliski Image Stitching 18 Interpolation Possible interpolation filters: nearest neighbor bilinear bicubic (interpolating) sinc / FIR Needed to prevent jaggies and texture crawl (see demo) Prefiltering Essential for downsampling (decimation) to prevent aliasing MIP-mapping [Williams 83]: 1. build pyramid (but what decimation filter?): block averaging Burt & Adelson (5-tap binomial) 7-tap wavelet-based filter (better) 2. trilinear interpolation bilinear within each 2 adjacent levels linear blend between levels (determined by pixel size) Richard Szeliski Image Stitching 19 Richard Szeliski Image Stitching 20

6 Prefiltering Essential for downsampling (decimation) to prevent aliasing Other possibilities: summed area tables elliptically weighted Gaussians (EWA) [Heckbert 86] Motion models (reprise) Richard Szeliski Image Stitching 21 Motion models Plane perspective mosaics 8-parameter generalization of affine motion Translation Affine Perspective 3D rotation works for pure rotation or planar surfaces Limitations: local minima slow convergence difficult to control interactively 2 unknowns 6 unknowns 8 unknowns 3 unknowns Richard Szeliski Image Stitching 23 Richard Szeliski Image Stitching 24

7 Image warping with homographies Rotational mosaics Directly optimize rotation and focal length Advantages: ability to build full-view panoramas easier to control interactively more stable and accurate estimates image plane in front image plane below Richard Szeliski Image Stitching 25 Richard Szeliski Image Stitching 26 3D 2D Perspective Projection Rotational mosaic f u c (X c,y c,z c ) Projection equations 1. Project from image to 3D ray (x 0,y 0,z 0 ) = (u 0 -u c,v 0 -v c,f) u 2. Rotate the ray by camera motion (x 1,y 1,z 1 ) = R 01 (x 0,y 0,z 0 ) 3. Project back into new (source) image (u 1,v 1 ) = (fx 1 /z 1 +u c,fy 1 /z 1 +v c ) Richard Szeliski Image Stitching 27 Richard Szeliski Image Stitching 28

8 Image reprojection mosaic PP The mosaic has a natural interpretation in 3D The images are reprojected onto a common plane The mosaic is formed on this plane Richard Szeliski Image Stitching 29 Rotations and quaternions How do we represent rotation matrices? 1. Axis / angle (n,θ) R = I + sinθ [n] + (1- cosθ) [n] 2 (Rodriguez Formula), with [n] = cross product matrix (see paper) 2. Unit quaternions [Shoemake SIGG 85] q = (n sinθ/2, cosθ/2) = (w,s) quaternion multiplication (division is easy) q 0 q 1 = (s 1 w 0 + s 0 w 1, s 0 s 1 -w 0 w 1 ) Richard Szeliski Image Stitching 30 Incremental rotation update 1. Small angle approximation ΔR = I + sinθ [n] + (1- cosθ) [n] 2 θ [n] = [ω] linear in ω 2. Update original R matrix R R ΔR Perspective & rotational motion Solve 8x8 or 3x3 system (see papers for details), and iterate (non-linear) Patch-based approximation: 1. break up image into patches (say 16x16) 2. accumulate 2x2 linear system in each (local translational assumption) 3. compose larger system from smaller 2x2 results [Shum & Szeliski, ICCV 98] Richard Szeliski Image Stitching 31 Richard Szeliski Image Stitching 32

9 Image Mosaics (stitching) Image Mosaics (Stitching) [Szeliski & Shum, SIGGRAPH 97] [Szeliski, FnT CVCG, 2006] Blend together several overlapping images into one seamless mosaic (composite) = Richard Szeliski Image Stitching 34 Mosaics for Video Coding Convert masked images into a background sprite for content-based coding = Establishing correspondences 1. Direct method: Use generalization of affine motion model [Szeliski & Shum 97] 2. Feature-based method Extract features, match, find consisten inliers [Lowe ICCV 99; Schmid ICCV 98, Brown&Lowe ICCV 2003] Compute R from correspondences (absolute orientation) Richard Szeliski Image Stitching 35 Richard Szeliski Image Stitching 36

10 Absolute orientation Stitching demo [Arun et al., PAMI 1987] [Horn et al., JOSA A 1988] Procrustes Algorithm [Golub & VanLoan] Given two sets of matching points, compute R p i = R p i 3D rays A = Σ i p i p i T = Σ i p i p it R T = U S V T = (U S U T ) R T V T = U T R T R = VU T Richard Szeliski Image Stitching 37 Richard Szeliski Image Stitching 38 Panoramas Cylindrical panoramas What if you want a 360 field of view? mosaic Projection Cylinder Richard Szeliski Image Stitching 39 Steps Reproject each image onto a cylinder Blend Output the resulting mosaic Richard Szeliski Image Stitching 40

11 Cylindrical Panoramas Map image to cylindrical or spherical coordinates need known focal length Determining the focal length 1. Initialize from homography H (see text or [SzSh 97]) 2. Use camera s EXIF tags (approx.) 3. Use a tape measure 4m 1m Image 384x300 f = 180 (pixels) f = 280 f = Ask your instructor Richard Szeliski Image Stitching 41 Richard Szeliski Image Stitching 42 3D 2D Perspective Projection (X c,y c,z c ) Cylindrical projection Map 3D point (X,Y,Z) onto cylinder f u c u Y Z X unit cylinder Convert to cylindrical coordinates Convert to cylindrical image coordinates s defines size of the final image unwrapped cylinder Richard Szeliski Image Stitching 43 cylindrical image Richard Szeliski Image Stitching 44

12 Cylindrical warping Spherical warping Given focal length f and image center (x c,y c ) (X,Y,Z) Given focal length f and image center (x c,y c ) (x,y,z) φ cos φ Y Z X (sinθ,h,cosθ) Y Z X (sinθcosφ,cosθcosφ,sinφ) sin φ cos θ cos φ Richard Szeliski Image Stitching 45 Richard Szeliski Image Stitching 46 3D rotation Radial distortion Rotate image before placing on unrolled sphere (x,y,z) φ cos φ Correct for bending in wide field of view lenses (sinθcosφ,cosθcosφ,sinφ) Z sin φ Y X p = R p cos θ cos φ Richard Szeliski Image Stitching 47 Richard Szeliski Image Stitching 48

13 Fisheye lens Extreme bending in ultra-wide fields of view Image Stitching 1. Align the images over each other camera pan translation on cylinder 2. Blend the images together (demo) Richard Szeliski Image Stitching 49 Richard Szeliski Image Stitching 50 Project 2 image stitching 1. Take pictures on a tripod (or handheld) 2. Warp images to spherical coordinates 3. Extract features 4. Align neighboring pairs using RANSAC 5. Write out list of neighboring translations 6. Correct for drift 7. Read in warped images and blend them 8. Crop the result and import into a viewer Matching features What do we do about the bad matches? Richard Szeliski Image Stitching 51 Richard Szeliski Image Stitching 52

14 RAndom SAmple Consensus RAndom SAmple Consensus Select one match, count inliers Select one match, count inliers Richard Szeliski Image Stitching 53 Richard Szeliski Image Stitching 54 Least squares fit Assembling the panorama Stitch pairs together, blend, then crop Find average translation vector Richard Szeliski Image Stitching 55 Richard Szeliski Image Stitching 56

15 Problem: Drift (x 1,y 1 ) Problem: Drift (x n,y n ) Error accumulation small (vertical) errors accumulate over time apply correction so that sum = 0 (for 360 pan.) Richard Szeliski Image Stitching 57 copy of first image Solution add another copy of first image at the end this gives a constraint: y n = y 1 there are a bunch of ways to solve this problem add displacement of (y 1 y n )/(n -1) to each image after the first compute a global warp: y = y + ax run a big optimization problem, incorporating this t i t Richard Szeliski Image Stitching 58 Full-view Panorama Full-view (360 spherical) panoramas Richard Szeliski Image Stitching 60

16 Texture Mapped Model Global alignment Register all pairwise overlapping images Use a 3D rotation model (one R per image) Use direct alignment (patch centers) or feature based Infer overlaps based on previous matches (incremental) Optionally discover which images overlap other images using feature selection (RANSAC) Richard Szeliski Image Stitching 61 Richard Szeliski Image Stitching 62 Recognizing Panoramas Recognizing Panoramas Matthew Brown & David Lowe ICCV 2003 [Brown & Lowe, ICCV 03] Richard Szeliski Image Stitching 64

17 Finding the panoramas Finding the panoramas Richard Szeliski Image Stitching 65 Richard Szeliski Image Stitching 66 Finding the panoramas Finding the panoramas Richard Szeliski Image Stitching 67 Richard Szeliski Image Stitching 68

18 Fully automated 2D stitching Get you own free copy Demo Richard Szeliski Image Stitching 69 Richard Szeliski Image Stitching 70 Rec.pano.: system components 1. Feature detection and description more uniform point density 2. Fast matching (hash table) 3. RANSAC filtering of matches 4. Intensity-based verification 5. Incremental bundle adjustment [M. Brown, R. Szeliski, and S. Winder. Multi-image matching using multi-scale oriented patches, CVPR'2005] Richard Szeliski Image Stitching 71 Multi-Scale Oriented Patches Interest points Multi-scale Harris corners Orientation from blurred gradient Geometrically invariant to similarity transforms Descriptor vector Bias/gain normalized sampling of local patch (8x8) Photometrically invariant to affine changes in intensity Richard Szeliski Image Stitching 72

19 Feature irregularities Descriptor Vector Distribute points evenly over the image Orientation = blurred gradient Similarity Invariant Frame Scale-space position (x, y, s) + orientation (θ) Richard Szeliski Image Stitching 73 Probabilistic Feature Matching Richard Szeliski Image Stitching Richard Szeliski Image Stitching 74 RANSAC motion model 75 Richard Szeliski Image Stitching 76

20 RANSAC motion model RANSAC motion model Richard Szeliski Image Stitching 77 Richard Szeliski Image Stitching 78 Probabilistic model for verification How well does this work? Test on 100s of examples Richard Szeliski Image Stitching 79

21 Matching Mistakes: False Positive How well does this work? Test on 100s of examples still too many failures (5-10%) for consumer application Richard Szeliski Image Stitching 82 Matching Mistakes: False Positive Matching Mistake: False Negative Moving objects: large areas of disagreement Richard Szeliski Image Stitching 83 Richard Szeliski Image Stitching 84

22 Matching Mistakes Accidental alignment repeated / similar regions Failed alignments moving objects / parallax low overlap feature-less regions (more variety?) No 100% reliable algorithm? Richard Szeliski Image Stitching 85 How can we fix these? Tune the feature detector Tune the feature matcher (cost metric) Tune the RANSAC stage (motion model) Tune the verification stage Use higher-level knowledge e.g., typical camera motions Sounds like a big learning problem Need a large training/test data set (panoramas) Richard Szeliski Image Stitching 86 Image feathering Image Blending Weight each image proportional to its distance from the edge (distance map [Danielsson, CVGIP 1980] 1. Generate weight map for each image 2. Sum up all of the weights and divide by sum: weights sum up to 1: w i = w i / ( i w i ) Richard Szeliski Image Stitching 88

23 Image Feathering Feathering = 1 0 Richard Szeliski Image Stitching 89 Richard Szeliski Image Stitching 90 Effect of window size Effect of window size 1 left right Richard Szeliski Image Stitching 91 Richard Szeliski Image Stitching 92

24 Good window size Pyramid Blending 1 0 Optimal window: smooth but not ghosted Doesn t always work... Richard Szeliski Image Stitching 93 Burt, P. J. and Adelson, E. H., A multiresolution spline with applications to image Richard Szeliski Image Stitching 94 mosaics, ACM Transactions on Graphics, 42(4), October 1983, Laplacian level 4 Laplacian level 2 Laplacian level 0 Richard Szeliski Image Stitching 95 left pyramid right pyramid blended pyramid Laplacian image blend 1. Compute Laplacian pyramid 2. Compute Gaussian pyramid on weight image (can put this in A channel) 3. Blend Laplacians using Gaussian blurred weights 4. Reconstruct the final image Q: How do we compute the original weights? A: For horizontal panorama, use mid-lines Q: How about for a general 3D panorama? Richard Szeliski Image Stitching 96

25 Weight selection (3D panorama) Poisson Image Editing Idea: use original feather weights to select strongest contributing image Can be implemented using L- norm: (p = 10) w i = [w ip / ( i w ip )] 1/p Richard Szeliski Image Stitching 97 Blend the gradients of the two images, then integrate For more info: Perez et al, SIGGRAPH 2003 Richard Szeliski Image Stitching 98 Local alignment (deghosting) De-Ghosting Use local optic flow to compensate for small motions [Shum & Szeliski, ICCV 98] Richard Szeliski Image Stitching 100

26 Local alignment (deghosting) Use local optic flow to compensate for radial distortion [Shum & Szeliski, ICCV 98] Region-based de-ghosting Select only one image in regions-of-difference using weighted vertex cover [Uyttendaele et al., CVPR 01] Richard Szeliski Image Stitching 101 Richard Szeliski Image Stitching 102 Region-based de-ghosting Select only one image in regions-of-difference using weighted vertex cover [Uyttendaele et al., CVPR 01] Cutout-based de-ghosting Select only one image per output pixel, using spatial continuity Blend across seams using gradient continuity ( Poisson blending ) [Agarwala et al., SG 2004] Richard Szeliski Image Stitching 103 Richard Szeliski Image Stitching 104

27 Cutout-based compositing Photomontage [Agarwala et al., SG 2004] Interactively blend different images: group portraits PhotoMontage Technical details: use Graph Cuts to optimize seam placement Demo: GroupShot application Richard Szeliski Image Stitching 105 Richard Szeliski Image Stitching 106 Cutout-based compositing Photomontage [Agarwala et al., SG 2004] Interactively blend different images: focus settings Cutout-based compositing Photomontage [Agarwala et al., SG 2004] Interactively blend different images: people s faces Richard Szeliski Image Stitching 107 Richard Szeliski Image Stitching 108

28 More stitching possibilities Other types of mosaics Video stitching High dynamic range image stitching see demo Flash + Non-Flash Video-based rendering Next week s lecture: Computational Photography Richard Szeliski Image Stitching 109 Can mosaic onto any surface if you know the geometry See NASA s Visible Earth project for some stunning earth mosaics Richard Szeliski Image Stitching 110 Slit images Slit images: cyclographs y-t slices of the video volume are known as slit images take a single column of pixels from each input image Richard Szeliski Image Stitching 111 Richard Szeliski Image Stitching 112

29 Slit images: photofinish Final thought: What is a panorama? Tracking a subject Repeated (best) shots Multiple exposures Infer what photographer wants? Richard Szeliski Image Stitching 113 Richard Szeliski Image Stitching 114

Panoramas. CS 178, Spring Marc Levoy Computer Science Department Stanford University

Panoramas CS 178, Spring 2010 Marc Levoy Computer Science Department Stanford University What is a panorama?! a wider-angle image than a normal camera can capture! any image stitched from overlapping photographs!

Panoramas. CS 178, Spring Marc Levoy Computer Science Department Stanford University

Panoramas CS 178, Spring 2012 Marc Levoy Computer Science Department Stanford University What is a panorama?! a wider-angle image than a normal camera can capture! any image stitched from overlapping photographs!

Panoramic imaging. Ixyzϕθλt. 45 degrees FOV (normal view)

Camera projections Recall the plenoptic function: Panoramic imaging Ixyzϕθλt (,,,,,, ) At any point xyz,, in space, there is a full sphere of possible incidence directions ϕ, θ, covered by 0 ϕ 2π, 0 θ

Video Registration: Key Challenges. Richard Szeliski Microsoft Research

Video Registration: Key Challenges Richard Szeliski Microsoft Research 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Key Challenges 1. Mosaics and panoramas 2. Object-based based segmentation (MPEG-4) 3. Engineering

Creating a Panorama Photograph Using Photoshop Elements

Creating a Panorama Photograph Using Photoshop Elements Following are guidelines when shooting photographs for a panorama. Overlap images sufficiently -- Images should overlap approximately 15% to 40%.

Light Field based 360º Panoramas

1 Light Field based 360º Panoramas André Alexandre Rodrigues Oliveira Abstract This paper describes in detail the developed light field based 360º panorama creation solution, named as multiperspective

Radiometric alignment and vignetting calibration Pablo d Angelo University of Bielefeld, Technical Faculty, Applied Computer Science D-33501 Bielefeld, Germany pablo.dangelo@web.de Abstract. This paper

Homographies and Mosaics

Homographies and Mosaics Jeffrey Martin (jeffrey-martin.com) CS194: Image Manipulation & Computational Photography with a lot of slides stolen from Alexei Efros, UC Berkeley, Fall 2014 Steve Seitz and

Vision Review: Image Processing. Course web page:

Vision Review: Image Processing Course web page: www.cis.udel.edu/~cer/arv September 7, Announcements Homework and paper presentation guidelines are up on web page Readings for next Tuesday: Chapters 6,.,

Light field sensing. Marc Levoy. Computer Science Department Stanford University

Light field sensing Marc Levoy Computer Science Department Stanford University The scalar light field (in geometrical optics) Radiance as a function of position and direction in a static scene with fixed

How to combine images in Photoshop

How to combine images in Photoshop In Photoshop, you can use multiple layers to combine images, but there are two other ways to create a single image from mulitple images. Create a panoramic image with

Computational Photography and Video. Prof. Marc Pollefeys

Computational Photography and Video Prof. Marc Pollefeys Today s schedule Introduction of Computational Photography Course facts Syllabus Digital Photography What is computational photography Convergence

Sensors and Sensing Cameras and Camera Calibration

Sensors and Sensing Cameras and Camera Calibration Todor Stoyanov Mobile Robotics and Olfaction Lab Center for Applied Autonomous Sensor Systems Örebro University, Sweden todor.stoyanov@oru.se 20.11.2014

Antialiasing and Related Issues

Antialiasing and Related Issues OUTLINE: Antialiasing Prefiltering, Supersampling, Stochastic Sampling Rastering and Reconstruction Gamma Correction Antialiasing Methods To reduce aliasing, either: 1.

ON THE CREATION OF PANORAMIC IMAGES FROM IMAGE SEQUENCES

ON THE CREATION OF PANORAMIC IMAGES FROM IMAGE SEQUENCES Petteri PÖNTINEN Helsinki University of Technology, Institute of Photogrammetry and Remote Sensing, Finland petteri.pontinen@hut.fi KEY WORDS: Cocentricity,

Image Pyramids. Sanja Fidler CSC420: Intro to Image Understanding 1 / 35

Image Pyramids Sanja Fidler CSC420: Intro to Image Understanding 1 / 35 Finding Waldo Let s revisit the problem of finding Waldo This time he is on the road template (filter) image Sanja Fidler CSC420:

The ultimate camera. Computational Photography. Creating the ultimate camera. The ultimate camera. What does it do?

Computational Photography The ultimate camera What does it do? Image from Durand & Freeman s MIT Course on Computational Photography Today s reading Szeliski Chapter 9 The ultimate camera Infinite resolution

Image Sampling. Moire patterns. - Source: F. Durand

Image Sampling Moire patterns Source: F. Durand - http://www.sandlotscience.com/moire/circular_3_moire.htm Any questions on project 1? For extra credits, attach before/after images how your extra feature

Capturing and Viewing Gigapixel Images

Capturing and Viewing Gigapixel Images Johannes Kopf University of Konstanz Matt Uyttendaele Microsoft Research Oliver Deussen University of Konstanz Michael F. Cohen Microsoft Research Figure : Three

High-Resolution Interactive Panoramas with MPEG-4

High-Resolution Interactive Panoramas with MPEG-4 Peter Eisert, Yong Guo, Anke Riechers, Jürgen Rurainsky Fraunhofer Institute for Telecommunications, Heinrich-Hertz-Institute Image Processing Department

A short introduction to panoramic images

A short introduction to panoramic images By Richard Novossiltzeff Bridgwater Photographic Society March 25, 2014 1 What is a panorama Some will say that the word Panorama is over-used; the better word

Efficient Construction of SIFT Multi-Scale Image Pyramids for Embedded Robot Vision

Efficient Construction of SIFT Multi-Scale Image Pyramids for Embedded Robot Vision Peter Andreas Entschev and Hugo Vieira Neto Graduate School of Electrical Engineering and Applied Computer Science Federal

Image Interpolation. Image Processing

Image Interpolation Image Processing Brent M. Dingle, Ph.D. 2015 Game Design and Development Program Mathematics, Statistics and Computer Science University of Wisconsin - Stout public domain image from

CSC 320 H1S CSC320 Exam Study Guide (Last updated: April 2, 2015) Winter 2015

Question 1. Suppose you have an image I that contains an image of a left eye (the image is detailed enough that it makes a difference that it s the left eye). Write pseudocode to find other left eyes in

Using Line and Ellipse Features for Rectification of Broadcast Hockey Video

Using Line and Ellipse Features for Rectification of Broadcast Hockey Video Ankur Gupta, James J. Little, Robert J. Woodham Laboratory for Computational Intelligence (LCI) The University of British Columbia

Restoration of Motion Blurred Document Images

Restoration of Motion Blurred Document Images Bolan Su 12, Shijian Lu 2 and Tan Chew Lim 1 1 Department of Computer Science,School of Computing,National University of Singapore Computing 1, 13 Computing

CS535 Fall Department of Computer Science Purdue University

Omnidirectional Camera Models CS535 Fall 2010 Daniel G Aliaga Daniel G. Aliaga Department of Computer Science Purdue University A little bit of history Omnidirectional cameras are also called panoramic

Multi-perspective Panoramas. Slides from a talk by Lihi Zelnik-Manor at ICCV 07 3DRR workshop

Multi-perspective Panoramas Slides from a talk by Lihi Zelnik-Manor at ICCV 07 3DRR workshop Objectives 1. Better looking panoramas 2. Let the camera move: Any view Natural photographing Stand on the shoulders

Digital Image Processing

Digital Image Processing Digital Imaging Fundamentals Christophoros Nikou cnikou@cs.uoi.gr Images taken from: R. Gonzalez and R. Woods. Digital Image Processing, Prentice Hall, 2008. Digital Image Processing

Which equipment is necessary? How is the panorama created?

Congratulations! By purchasing your Panorama-VR-System you have acquired a tool, which enables you - together with a digital or analog camera, a tripod and a personal computer - to generate high quality

Digital Image Fundamentals. Digital Image Processing. Human Visual System. Contents. Structure Of The Human Eye (cont.) Structure Of The Human Eye

Digital Image Processing 2 Digital Image Fundamentals Digital Imaging Fundamentals Christophoros Nikou cnikou@cs.uoi.gr Those who wish to succeed must ask the right preliminary questions Aristotle Images

A Geometric Correction Method of Plane Image Based on OpenCV

Sensors & Transducers 204 by IFSA Publishing, S. L. http://www.sensorsportal.com A Geometric orrection Method of Plane Image ased on OpenV Li Xiaopeng, Sun Leilei, 2 Lou aiying, Liu Yonghong ollege of

Fast Motion Blur through Sample Reprojection

Fast Motion Blur through Sample Reprojection Micah T. Taylor taylormt@cs.unc.edu Abstract The human eye and physical cameras capture visual information both spatially and temporally. The temporal aspect

Burst Photography! EE367/CS448I: Computational Imaging and Display! stanford.edu/class/ee367! Lecture 7! Gordon Wetzstein! Stanford University!

Burst Photography! EE367/CS448I: Computational Imaging and Display! stanford.edu/class/ee367! Lecture 7! Gordon Wetzstein! Stanford University! Motivation! wikipedia! exposure sequence! -4 stops! Motivation!

Image Deblurring with Blurred/Noisy Image Pairs

Image Deblurring with Blurred/Noisy Image Pairs Huichao Ma, Buping Wang, Jiabei Zheng, Menglian Zhou April 26, 2013 1 Abstract Photos taken under dim lighting conditions by a handheld camera are usually

Geometry of Aerial Photographs

Geometry of Aerial Photographs Aerial Cameras Aerial cameras must be (details in lectures): Geometrically stable Have fast and efficient shutters Have high geometric and optical quality lenses They can

Early art: events. Baroque art: portraits. Renaissance art: events. Being There: Capturing and Experiencing a Sense of Place

Being There: Capturing and Experiencing a Sense of Place Early art: events Richard Szeliski Microsoft Research Symposium on Computational Photography and Video Lascaux Early art: events Early art: events

Image Resizing by Seam Carving in Python and Matched Masks

Image Resizing by Seam Carving in Python and Matched Masks Alexander Converse Department of Electrical Engineering and Computer Science, Case Western Reserve University, Cleveland, OH, Email: alexander.converse@case.edu

Panoramic Mosaicing with a 180 Field of View Lens

CENTER FOR MACHINE PERCEPTION CZECH TECHNICAL UNIVERSITY Panoramic Mosaicing with a 18 Field of View Lens Hynek Bakstein and Tomáš Pajdla {bakstein, pajdla}@cmp.felk.cvut.cz REPRINT Hynek Bakstein and

Rectifying the Planet USING SPACE TO HELP LIFE ON EARTH

Rectifying the Planet USING SPACE TO HELP LIFE ON EARTH About Me Computer Science (BS) Ecology (PhD, almost ) I write programs that process satellite data Scientific Computing! Land Cover Classification

1.Discuss the frequency domain techniques of image enhancement in detail.

1.Discuss the frequency domain techniques of image enhancement in detail. Enhancement In Frequency Domain: The frequency domain methods of image enhancement are based on convolution theorem. This is represented

A Study of Slanted-Edge MTF Stability and Repeatability

A Study of Slanted-Edge MTF Stability and Repeatability Jackson K.M. Roland Imatest LLC, 2995 Wilderness Place Suite 103, Boulder, CO, USA ABSTRACT The slanted-edge method of measuring the spatial frequency

Synthetic Stereoscopic Panoramic Images

Synthetic Stereoscopic Panoramic Images What are they? How are they created? What are they good for? Paul Bourke University of Western Australia In collaboration with ICinema @ University of New South

Image Processing (EA C443)

Image Processing (EA C443) OBJECTIVES: To study components of the Image (Digital Image) To Know how the image quality can be improved How efficiently the image data can be stored and transmitted How the

PSEUDO HDR VIDEO USING INVERSE TONE MAPPING

PSEUDO HDR VIDEO USING INVERSE TONE MAPPING Yu-Chen Lin ( 林育辰 ), Chiou-Shann Fuh ( 傅楸善 ) Dept. of Computer Science and Information Engineering, National Taiwan University, Taiwan E-mail: r03922091@ntu.edu.tw

IMAGE ENHANCEMENT IN SPATIAL DOMAIN

A First Course in Machine Vision IMAGE ENHANCEMENT IN SPATIAL DOMAIN By: Ehsan Khoramshahi Definitions The principal objective of enhancement is to process an image so that the result is more suitable

Physics 3340 Spring Fourier Optics

Physics 3340 Spring 011 Purpose Fourier Optics In this experiment we will show how the Fraunhofer diffraction pattern or spatial Fourier transform of an object can be observed within an optical system.

Joomla developers needed - Long term potential in India Copywriter Email Campaigns Wordpress Creative design Social media in UK More Freelance Jobs... Facebook Twitter Google+ Find us on Facebook Step

Modeling and Synthesis of Aperture Effects in Cameras

Modeling and Synthesis of Aperture Effects in Cameras Douglas Lanman, Ramesh Raskar, and Gabriel Taubin Computational Aesthetics 2008 20 June, 2008 1 Outline Introduction and Related Work Modeling Vignetting

Image Formation: Camera Model

Image Formation: Camera Model Ruigang Yang COMP 684 Fall 2005, CS684-IBMR Outline Camera Models Pinhole Perspective Projection Affine Projection Camera with Lenses Digital Image Formation The Human Eye

Image Filtering. Median Filtering

Image Filtering Image filtering is used to: Remove noise Sharpen contrast Highlight contours Detect edges Other uses? Image filters can be classified as linear or nonlinear. Linear filters are also know

Math 259 Winter Recitation Handout 9: Lagrange Multipliers

Math 259 Winter 2009 Recitation Handout 9: Lagrange Multipliers The method of Lagrange Multipliers is an excellent technique for finding the global maximum and global minimum values of a function f(x,

OFFSET AND NOISE COMPENSATION

OFFSET AND NOISE COMPENSATION AO 10V 8.1 Offset and fixed pattern noise reduction Offset variation - shading AO 10V 8.2 Row Noise AO 10V 8.3 Offset compensation Global offset calibration Dark level is

IMAGE FORMATION. Light source properties. Sensor characteristics Surface. Surface reflectance properties. Optics

IMAGE FORMATION Light source properties Sensor characteristics Surface Exposure shape Optics Surface reflectance properties ANALOG IMAGES An image can be understood as a 2D light intensity function f(x,y)

Capturing Light. The Light Field. Grayscale Snapshot 12/1/16. P(q, f)

Capturing Light Rooms by the Sea, Edward Hopper, 1951 The Penitent Magdalen, Georges de La Tour, c. 1640 Some slides from M. Agrawala, F. Durand, P. Debevec, A. Efros, R. Fergus, D. Forsyth, M. Levoy,

History of projection. Perspective. History of projection. Plane projection in drawing

History of projection Ancient times: Greeks wrote about laws of perspective Renaissance: perspective is adopted by artists Perspective CS 4620 Lecture 3 Duccio c. 1308 1 2 History of projection Plane projection

Panoramas and High-Dynamic-Range Imaging

Panoramas and High-Dynamic-Range Imaging Kari Pulli Senior Director Are you getting the whole picture? Compact Camera FOV = 50 x 35 Slide from Brown & Lowe Are you getting the whole picture? Compact Camera

Image Processing & Projective geometry

Image Processing & Projective geometry Arunkumar Byravan Partial slides borrowed from Jianbo Shi & Steve Seitz Color spaces RGB Red, Green, Blue HSV Hue, Saturation, Value Why HSV? HSV separates luma,

T I P S F O R I M P R O V I N G I M A G E Q U A L I T Y O N O Z O F O O T A G E

T I P S F O R I M P R O V I N G I M A G E Q U A L I T Y O N O Z O F O O T A G E Updated 20 th Jan. 2017 References Creator V1.4.0 2 Overview This document will concentrate on OZO Creator s Image Parameter

Image Filtering in Spatial domain. Computer Vision Jia-Bin Huang, Virginia Tech

Image Filtering in Spatial domain Computer Vision Jia-Bin Huang, Virginia Tech Administrative stuffs Lecture schedule changes Office hours - Jia-Bin (44 Whittemore Hall) Friday at : AM 2: PM Office hours

Technical information about PhoToPlan The following pages shall give you a detailed overview of the possibilities using PhoToPlan. kubit GmbH Fiedlerstr. 36, 01307 Dresden, Germany Fon: +49 3 51/41 767

Squaring the Circle in Panoramas

in: Proceedings of the Tenth IEEE International Conference on Computer Vision, pp. 1292-1299, Beijing, China, October 15-21, 2005. Squaring the Circle in Panoramas Lihi Zelnik-Manor 1 Gabriele Peters 2

Panoramic Vision System for an Intelligent Vehicle using. a Laser Sensor and Cameras

Panoramic Vision System for an Intelligent Vehicle using a Laser Sensor and Cameras Min Woo Park PH.D Student, Graduate School of Electrical Engineering and Computer Science, Kyungpook National University,

Realistic Visual Environment for Immersive Projection Display System

Realistic Visual Environment for Immersive Projection Display System Hasup Lee Center for Education and Research of Symbiotic, Safe and Secure System Design Keio University Yokohama, Japan hasups@sdm.keio.ac.jp

CSC 170 Introduction to Computers and Their Applications. Lecture #3 Digital Graphics and Video Basics. Bitmap Basics

CSC 170 Introduction to Computers and Their Applications Lecture #3 Digital Graphics and Video Basics Bitmap Basics As digital devices gained the ability to display images, two types of computer graphics

Multiresolution Histograms and their Use for Texture Classification

Multiresolution Histograms and their Use for Texture Classification E. Hadjidemetriou, M. D. Grossberg, and S. K. Nayar Computer Science, Columbia University, New York, NY 17 {stathis, mdog, nayar}@cs.columbia.edu

Lecture 19: Depth Cameras. Kayvon Fatahalian CMU : Graphics and Imaging Architectures (Fall 2011)

Lecture 19: Depth Cameras Kayvon Fatahalian CMU 15-869: Graphics and Imaging Architectures (Fall 2011) Continuing theme: computational photography Cheap cameras capture light, extensive processing produces

IMAGE STABILIZATION WITH BEST SHOT SELECTOR AND SUPER RESOLUTION RECONSTRUCTION

IMAGE STABILIZATION WITH BEST SHOT SELECTOR AND SUPER RESOLUTION RECONSTRUCTION Jing-Fu Chen ( 陳景富 ) and Chiou-Shann Fuh ( 傅楸善 ) Department of Computer Science and Information Engineering National Taiwan

How to Create a Curious Owl in Illustrator

How to Create a Curious Owl in Illustrator Tutorial Details Program: Adobe Illustrator Difficulty: Intermediate Estimated Completion Time: 1.5 hours Take a look at what we're aiming for, an inquisitive

Computational Camera & Photography: Coded Imaging

Computational Camera & Photography: Coded Imaging Camera Culture Ramesh Raskar MIT Media Lab http://cameraculture.media.mit.edu/ Image removed due to copyright restrictions. See Fig. 1, Eight major types

Digital deformation model for fisheye image rectification

Digital deformation model for fisheye image rectification Wenguang Hou, 1 Mingyue Ding, 1 Nannan Qin, 2 and Xudong Lai 2, 1 Department of Bio-medical Engineering, Image Processing and Intelligence Control

Method for out-of-focus camera calibration

2346 Vol. 55, No. 9 / March 20 2016 / Applied Optics Research Article Method for out-of-focus camera calibration TYLER BELL, 1 JING XU, 2 AND SONG ZHANG 1, * 1 School of Mechanical Engineering, Purdue

Depth Perception with a Single Camera

Depth Perception with a Single Camera Jonathan R. Seal 1, Donald G. Bailey 2, Gourab Sen Gupta 2 1 Institute of Technology and Engineering, 2 Institute of Information Sciences and Technology, Massey University,

Preserving Natural Scene Lighting by Strobe-lit Video

Preserving Natural Scene Lighting by Strobe-lit Video Olli Suominen, Atanas Gotchev Department of Signal Processing, Tampere University of Technology Korkeakoulunkatu 1, 33720 Tampere, Finland ABSTRACT

Basics of Photogrammetry Note#6

Basics of Photogrammetry Note#6 Photogrammetry Art and science of making accurate measurements by means of aerial photography Analog: visual and manual analysis of aerial photographs in hard-copy format

Camera Intrinsic Blur Kernel Estimation: A Reliable Framework

Camera Intrinsic Blur Kernel Estimation: A Reliable Framework Ali Mosleh 1 Paul Green Emmanuel Onzon Isabelle Begin J.M. Pierre Langlois 1 1 École Polytechnique de Montreál, Montréal, QC, Canada Algolux

Real Time Word to Picture Translation for Chinese Restaurant Menus

Real Time Word to Picture Translation for Chinese Restaurant Menus Michelle Jin, Ling Xiao Wang, Boyang Zhang Email: mzjin12, lx2wang, boyangz @stanford.edu EE268 Project Report, Spring 2014 Abstract--We

MULTIPLE SENSORS LENSLETS FOR SECURE DOCUMENT SCANNERS

INFOTEH-JAHORINA Vol. 10, Ref. E-VI-11, p. 892-896, March 2011. MULTIPLE SENSORS LENSLETS FOR SECURE DOCUMENT SCANNERS Jelena Cvetković, Aleksej Makarov, Sasa Vujić, Vlatacom d.o.o. Beograd Abstract -

Neuron Bundle 12: Digital Film Tools

Neuron Bundle 12: Digital Film Tools Neuron Bundle 12 consists of two plug-in sets Composite Suite Pro and zmatte from Digital Film Tools. Composite Suite Pro features a well rounded collection of visual

Panoramas and the Info Palette By: Martin Kesselman 5/25/09

Panoramas and the Info Palette By: Martin Kesselman 5/25/09 Any time you have a color you would like to copy exactly, use the info palette. When cropping to achieve a particular size, it is useful to use

Εισαγωγική στην Οπτική Απεικόνιση

Εισαγωγική στην Οπτική Απεικόνιση Δημήτριος Τζεράνης, Ph.D. Εμβιομηχανική και Βιοϊατρική Τεχνολογία Τμήμα Μηχανολόγων Μηχανικών Ε.Μ.Π. Χειμερινό Εξάμηνο 2015 Light: A type of EM Radiation EM radiation:

ME 6406 MACHINE VISION. Georgia Institute of Technology

ME 6406 MACHINE VISION Georgia Institute of Technology Class Information Instructor Professor Kok-Meng Lee MARC 474 Office hours: Tues/Thurs 1:00-2:00 pm kokmeng.lee@me.gatech.edu (404)-894-7402 Class

Double Aperture Camera for High Resolution Measurement

Double Aperture Camera for High Resolution Measurement Venkatesh Bagaria, Nagesh AS and Varun AV* Siemens Corporate Technology, India *e-mail: varun.av@siemens.com Abstract In the domain of machine vision,

Sampling and Reconstruction

Sampling and Reconstruction Many slides from Steve Marschner 15-463: Computational Photography Alexei Efros, CMU, Fall 211 Sampling and Reconstruction Sampled representations How to store and compute with

Creating Digital Illustrations for Your Research Workshop IV Illustration Demo Part II

Creating Digital Illustrations for Your Research Workshop IV Illustration Demo Part II Final Figure Workshop IV Components Topics & Techniques covered How to randomly transform a group of individual shapes.

Making a Panoramic Digital Image of the Entire Northern Sky

Making a Panoramic Digital Image of the Entire Northern Sky Anne M. Rajala anne2006@caltech.edu, x1221, MSC #775 Mentors: Ashish Mahabal and S.G. Djorgovski October 3, 2003 Abstract The Digitized Palomar

Cameras for Stereo Panoramic Imaging Λ

Cameras for Stereo Panoramic Imaging Λ Shmuel Peleg Yael Pritch Moshe Ben-Ezra School of Computer Science and Engineering The Hebrew University of Jerusalem 91904 Jerusalem, ISRAEL Abstract A panorama

6.098/6.882 Computational Photography 1. Problem Set 1. Assigned: Feb 9, 2006 Due: Feb 23, 2006

6.098/6.882 Computational Photography 1 Problem Set 1 Assigned: Feb 9, 2006 Due: Feb 23, 2006 Note The problems marked with 6.882 only are for the students who register for 6.882. (Of course, students

Joint Demosaicing and Super-Resolution Imaging from a Set of Unregistered Aliased Images

Joint Demosaicing and Super-Resolution Imaging from a Set of Unregistered Aliased Images Patrick Vandewalle a, Karim Krichane a, David Alleysson b, and Sabine Süsstrunk a a School of Computer and Communication

Understanding Focal Length

JANUARY 19, 2018 BEGINNER Understanding Focal Length Featuring DIANE BERKENFELD, DAVE BLACK, MIKE CORRADO & LINDSAY SILVERMAN Focal length, usually represented in millimeters (mm), is the basic description

Adaptive f-xy Hankel matrix rank reduction filter to attenuate coherent noise Nirupama (Pam) Nagarajappa*, CGGVeritas

Adaptive f-xy Hankel matrix rank reduction filter to attenuate coherent noise Nirupama (Pam) Nagarajappa*, CGGVeritas Summary The reliability of seismic attribute estimation depends on reliable signal.

Computational Cameras. Rahul Raguram COMP

Computational Cameras Rahul Raguram COMP 790-090 What is a computational camera? Camera optics Camera sensor 3D scene Traditional camera Final image Modified optics Camera sensor Image Compute 3D scene

Denoising and Effective Contrast Enhancement for Dynamic Range Mapping

Denoising and Effective Contrast Enhancement for Dynamic Range Mapping G. Kiruthiga Department of Electronics and Communication Adithya Institute of Technology Coimbatore B. Hakkem Department of Electronics

Color and perception Christian Miller CS Fall 2011

Color and perception Christian Miller CS 354 - Fall 2011 A slight detour We ve spent the whole class talking about how to put images on the screen What happens when we look at those images? Are there any

Perspective in 2D Games

Lecture 16 in 2D Games Drawing Images Graphics Lectures SpriteBatch interface Coordinates and Transforms bare minimum to draw graphics Drawing Camera Projections side-scroller vs. top down Drawing Primitives

Performance Evaluation of Edge Detection Techniques for Square Pixel and Hexagon Pixel images

Performance Evaluation of Edge Detection Techniques for Square Pixel and Hexagon Pixel images Keshav Thakur 1, Er Pooja Gupta 2,Dr.Kuldip Pahwa 3, 1,M.Tech Final Year Student, Deptt. of ECE, MMU Ambala,

Digital Signal Processing Techniques

Digital Signal Processing Techniques Dmitry Teytelman Dimtel, Inc., San Jose, CA, 95124, USA June 17, 2009 Outline 1 Introduction 2 Signal synthesis Arbitrary Waveform Generation CORDIC Direct Digital

Improved SIFT Matching for Image Pairs with a Scale Difference

Improved SIFT Matching for Image Pairs with a Scale Difference Y. Bastanlar, A. Temizel and Y. Yardımcı Informatics Institute, Middle East Technical University, Ankara, 06531, Turkey Published in IET Electronics,

Perception. Introduction to HRI Simmons & Nourbakhsh Spring 2015

Perception Introduction to HRI Simmons & Nourbakhsh Spring 2015 Perception my goals What is the state of the art boundary? Where might we be in 5-10 years? The Perceptual Pipeline The classical approach: