Computational Camera & Photography: Coded Imaging

Size: px
Start display at page:

Download "Computational Camera & Photography: Coded Imaging"

Transcription

1 Computational Camera & Photography: Coded Imaging Camera Culture Ramesh Raskar MIT Media Lab

2 Image removed due to copyright restrictions. See Fig. 1, Eight major types of optics in animal eyes. In Fernald, R. D. Casting a Genetic Light on the Evolution of Eyes. Science 313, no (September 29, 2006):

3 Traditional Camera Shutter is OPEN

4 H(f) Sinc Function Blurring == Convolution f Source: Raskar, Agrawal and Tumblin. Coded Exposure Photography: Motion Deblurring via Fluttered Shutter. Proceedings of SIGGRAPH Traditional Camera: Box Filter

5 Blurring Process for Linear Motion T T * = Source: Raskar, Agrawal and Tumblin. Coded Exposure Photography: Motion Deblurring via Fluttered Shutter. Proceedings of SIGGRAPH 2006.

6 Deblurring Process \ T =? Unknown Image A \ Smear Matrix b Blurred Image - Condition number for the smearing matrix is large - Thus invertibility is bad Source: Raskar, Agrawal and Tumblin. Coded Exposure Photography: Motion Deblurring via Fluttered Shutter. Proceedings of SIGGRAPH 2006.

7 Source: Raskar, Agrawal and Tumblin. Coded Exposure Photography: Motion Deblurring via Fluttered Shutter. Proceedings of SIGGRAPH 2006.

8 Source: Raskar, Agrawal and Tumblin. Coded Exposure Photography: Motion Deblurring via Fluttered Shutter. Proceedings of SIGGRAPH 2006.

9 Source: Raskar, Agrawal and Tumblin. Coded Exposure Photography: Motion Deblurring via Fluttered Shutter. Proceedings of SIGGRAPH Input Image

10 Source: Raskar, Agrawal and Tumblin. Coded Exposure Photography: Motion Deblurring via Fluttered Shutter. Proceedings of SIGGRAPH Rectified Crop Deblurred Result

11 Source: Raskar, Agrawal and Tumblin. Coded Exposure Photography: Motion Deblurring via Fluttered Shutter. Proceedings of SIGGRAPH 2006.

12 Source: Raskar, Agrawal and Tumblin. Coded Exposure Photography: Motion Deblurring via Fluttered Shutter. Proceedings of SIGGRAPH 2006.

13 Input Photo Deblurred Result Source: Raskar, Agrawal and Tumblin. Coded Exposure Photography: Motion Deblurring via Fluttered Shutter. Proceedings of SIGGRAPH 2006.

14 Traditional Camera Shutter is OPEN

15 Our Camera Flutter Shutter

16 Shutter is OPEN and CLOSED

17 Comparison of Blurred Images

18 Implementation Completely Portable

19 Mitsubishi Electric Research Labs (MERL) Coding and Modulation in Cameras Flutter Shutter On PointGrey Camera Dragonfly2 camera External Trigger Mode 5 Multiple Exposure Pulse Width Mode On Chip Fluttered Integration, no extra cost Courtesy of MERL. Used with permission.

20 Lab Setup

21 H(f) Sinc Function Blurring == Convolution f Source: Raskar, Agrawal and Tumblin. Coded Exposure Photography: Motion Deblurring via Fluttered Shutter. Proceedings of SIGGRAPH Traditional Camera: Box Filter

22 H(f) Preserves High Spatial Frequencies f Source: Raskar, Agrawal and Tumblin. Coded Exposure Photography: Motion Deblurring via Fluttered Shutter. Proceedings of SIGGRAPH Flutter Shutter: Coded Filter

23 Comparison H(f) H(f) f f Source: Raskar, Agrawal and Tumblin. Coded Exposure Photography: Motion Deblurring via Fluttered Shutter. Proceedings of SIGGRAPH 2006.

24 1/H(f) 1/H(f) Inverse Filter stable Inverse Filter Unstable f f Source: Raskar, Agrawal and Tumblin. Coded Exposure Photography: Motion Deblurring via Fluttered Shutter. Proceedings of SIGGRAPH 2006.

25 Short Exposure Long Exposure Coded Exposure Our result Matlab Lucy Ground Truth Source: Raskar, Agrawal and Tumblin. Coded Exposure Photography: Motion Deblurring via Fluttered Shutter. Proceedings of SIGGRAPH 2006.

26 Source: Raskar, Agrawal and Tumblin. Coded Exposure Photography: Motion Deblurring via Fluttered Shutter. Proceedings of SIGGRAPH Motion Blur as Convolution

27 Convolution == Linear System??

28 Solving

29 Are all codes good? H(f) All ones H(f) f Alternate f Random H(f) f Our Code f

30 Need to consider zero padded codes H(f) f H(f) f H(f) f

31 License Plate Retrieval

32 Mitsubishi Electric Research Labs (MERL) Coding and Modulation in Cameras Camera Limitations and Tradeoffs Low dynamic range Resolution vs Noise Motion blur Reduce shutter speed, but lose light Limited depth of field Reduce aperture, but lose light Increase shutter time, but motion blur Courtesy of MERL. Used with permission.

33 Mitsubishi Electric Research Labs (MERL) Coding and Modulation in Cameras Changing Aperture Size Images removed due to copyright restrictions. Samuel Hasinoff and Kiriakos Kutulakos, A Layer-Based Restoration Framework for Variable-Aperture Photography ICCV 2007

34 Mitsubishi Electric Research Labs (MERL) How to handle focus blur? Coding and Modulation in Cameras Source: Veeraraghavan, Raskar, Agarwal, Mohan, and Tumblin. Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing. Proceedings of SIGGRAPH 2007.

35 Mitsubishi Electric Research Labs (MERL) Coded Exposure (Flutter Shutter) Raskar, Agrawal, Tumblin SIGGRAPH 2006 Coding and Modulation in Cameras Coded Aperture with Veeraraghavan, Raskar, Tumblin, & Mohan, SIGGRAPH 2007 Temporal 1-D 1 D broadband code: Motion Deblurring Spatial 2-D 2 D broadband code: Focus Deblurring

36 Mitsubishi Electric Research Labs (MERL) Coding and Modulation in Cameras LED In Focus Photo

37 Mitsubishi Electric Research Labs (MERL) Coding and Modulation in Cameras Out of Focus Photo: Open Aperture

38 Mitsubishi Electric Research Labs (MERL) Coding and Modulation in Cameras Out of Focus Photo: Coded Aperture

39 Mitsubishi Electric Research Labs (MERL) Coding and Modulation in Cameras Out of Focus Photo: Coded Aperture

40 Blurred Photos Open Aperture Source: Veeraraghavan, Raskar, Agarwal, Mohan, and Tumblin. Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing. Proceedings of SIGGRAPH Coded Aperture, 7 * 7 Mask

41 Deblurred Photos Open Aperture Source: Veeraraghavan, Raskar, Agarwal, Mohan, and Tumblin. Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing. Proceedings of SIGGRAPH Coded Aperture, 7 * 7 Mask

42 Mitsubishi Electric Research Labs (MERL) Captured Blurred Photo Coding and Modulation in Cameras Source: Veeraraghavan, Raskar, Agarwal, Mohan, and Tumblin. Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing. Proceedings of SIGGRAPH 2007.

43 Mitsubishi Electric Research Labs (MERL) Refocused on Person Coding and Modulation in Cameras Source: Veeraraghavan, Raskar, Agarwal, Mohan, and Tumblin. Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing. Proceedings of SIGGRAPH 2007.

44 MERL, Northwestern Univ. Mask-Enhanced Cameras: Heterodyned Light Fields & Coded Aperture Veeraraghavan, Raskar, Agrawal, Mohan & Tumblin Blur Estimation & Segmentation Defocus blur dependent on depth Assumptions Layered Lambertian Scene Constant blur within each layer Deblur at different blur sizes k k = 1 Captured Blurred Photo k = 10

45 MERL, Northwestern Univ. Mask-Enhanced Cameras: Heterodyned Light Fields & Coded Aperture Define Cost Function k = 1 Veeraraghavan, Raskar, Agrawal, Mohan & Tumblin k = 1 k = 10 Deblurred Images k = 10 Cost Function Images Likelihood Error: (Blurred image - Sharp Image * PSF k ) 2 Gradient Error: Natural Image Statistics, Gradient Kurtosis

46 MERL, Northwestern Univ. Mask-Enhanced Cameras: Heterodyned Light Fields & Coded Aperture Veeraraghavan, Raskar, Agrawal, Mohan & Tumblin Blur Estimation & Segmentation == Labeling Graph cuts for labeling k = 1 k = 10 Error Images K = 1 K = 7

47 Captured Photo Reblur Deblur, k = 7 Fusion Source: Veeraraghavan, Raskar, Agarwal, Mohan, and Tumblin. Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing. Proceedings of SIGGRAPH 2007.

48

49 MERL, Northwestern Univ. Mask-Enhanced Cameras: Heterodyned Light Fields & Coded Aperture Weighted Deconvolution Veeraraghavan, Raskar, Agrawal, Mohan & Tumblin Source: Veeraraghavan, Raskar, Agarwal, Mohan, and Tumblin. Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing. Proceedings of SIGGRAPH 2007.

50 MERL, Northwestern Univ. Mask-Enhanced Cameras: Heterodyned Light Fields & Coded Aperture Veeraraghavan, Raskar, Agrawal, Mohan & Tumblin Source: Veeraraghavan, Raskar, Agarwal, Mohan, and Tumblin. Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing. Proceedings of SIGGRAPH 2007.

51 Mitsubishi Electric Research Labs (MERL) Coding and Modulation in Cameras Blocking Light == More Information Coded Exposure Coding in Time Coded Aperture Coding in Space

52 Mask? Mask Sensor Mask Sensor Full Resolution Digital Refocusing: Coded Aperture Camera 4D Light Field from 2D Photo: Heterodyne Light Field Camera

53 Conventional Lens: Limited Depth of Field Open Aperture Smaller Aperture Courtesy of Shree Nayar. Used with permission. Slide by Shree Nayar

54 Wavefront Coding using Cubic Phase Plate "Wavefront Coding: jointly optimized optical and digital imaging systems, E. Dowski, R. H. Cormack and S. D. Sarama, Aerosense Conference, April 25, 2000 Courtesy of Shree Nayar. Used with permission. Slide by Shree Nayar

55 Depth Invariant Blur Conventional System Wavefront Coded System Courtesy of Shree Nayar. Used with permission. Slide by Shree Nayar

56 Decoding depth via defocus blur Design PSF that changes quickly through focus so that defocus can be easily estimated Implementation using phase diffractive mask (Sig 2008, Levin et al used amplitude mask) Phase mask Image removed due to copyright restrictions. Typical PSF changes slowly Designed PSF changes fast Images removed due to copyright restrictions. Images removed due to copyright restrictions. R. Piestun, Y. Schechner, J. Shamir, Propagation-Invariant Wave Fields with Finite Energy, JOSA A 17, (2000) R. Piestun, J. Shamir, Generalized propagation invariant wave-fields, JOSA A 15, 3039 (1998)

57 Rotational PSF Images removed due to copyright restrictions. Two sequences showing rotating and standard PSF. R. Piestun, Y. Schechner, J. Shamir, Propagation-Invariant Wave Fields with Finite Energy, JOSA A 17, (2000) R. Piestun, J. Shamir, Generalized propagation invariant wave-fields, JOSA A 15, 3039 (1998)

58 Single Pixel Camera Several slides removed due to copyright restrictions. Slides by Shree Nayar

59 MIT OpenCourseWare MAS.531 / MAS.131 Computational Camera and Photography Fall 2009 For information about citing these materials or our Terms of Use, visit:

Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing

Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing Ashok Veeraraghavan, Ramesh Raskar, Ankit Mohan & Jack Tumblin Amit Agrawal, Mitsubishi Electric Research

More information

Coding and Modulation in Cameras

Coding and Modulation in Cameras Coding and Modulation in Cameras Amit Agrawal June 2010 Mitsubishi Electric Research Labs (MERL) Cambridge, MA, USA Coded Computational Imaging Agrawal, Veeraraghavan, Narasimhan & Mohan Schedule Introduction

More information

Coded Computational Photography!

Coded Computational Photography! Coded Computational Photography! EE367/CS448I: Computational Imaging and Display! stanford.edu/class/ee367! Lecture 9! Gordon Wetzstein! Stanford University! Coded Computational Photography - Overview!!

More information

Deblurring. Basics, Problem definition and variants

Deblurring. Basics, Problem definition and variants Deblurring Basics, Problem definition and variants Kinds of blur Hand-shake Defocus Credit: Kenneth Josephson Motion Credit: Kenneth Josephson Kinds of blur Spatially invariant vs. Spatially varying

More information

Coded photography , , Computational Photography Fall 2018, Lecture 14

Coded photography , , Computational Photography Fall 2018, Lecture 14 Coded photography http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 14 Overview of today s lecture The coded photography paradigm. Dealing with

More information

Coded photography , , Computational Photography Fall 2017, Lecture 18

Coded photography , , Computational Photography Fall 2017, Lecture 18 Coded photography http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2017, Lecture 18 Course announcements Homework 5 delayed for Tuesday. - You will need cameras

More information

To Do. Advanced Computer Graphics. Outline. Computational Imaging. How do we see the world? Pinhole camera

To Do. Advanced Computer Graphics. Outline. Computational Imaging. How do we see the world? Pinhole camera Advanced Computer Graphics CSE 163 [Spring 2017], Lecture 14 Ravi Ramamoorthi http://www.cs.ucsd.edu/~ravir To Do Assignment 2 due May 19 Any last minute issues or questions? Next two lectures: Imaging,

More information

Agenda. Fusion and Reconstruction. Image Fusion & Reconstruction. Image Fusion & Reconstruction. Dr. Yossi Rubner.

Agenda. Fusion and Reconstruction. Image Fusion & Reconstruction. Image Fusion & Reconstruction. Dr. Yossi Rubner. Fusion and Reconstruction Dr. Yossi Rubner yossi@rubner.co.il Some slides stolen from: Jack Tumblin 1 Agenda We ve seen Panorama (from different FOV) Super-resolution (from low-res) HDR (from different

More information

A Framework for Analysis of Computational Imaging Systems

A Framework for Analysis of Computational Imaging Systems A Framework for Analysis of Computational Imaging Systems Kaushik Mitra, Oliver Cossairt, Ashok Veeraghavan Rice University Northwestern University Computational imaging CI systems that adds new functionality

More information

Optimal Single Image Capture for Motion Deblurring

Optimal Single Image Capture for Motion Deblurring Optimal Single Image Capture for Motion Deblurring Amit Agrawal Mitsubishi Electric Research Labs (MERL) 1 Broadway, Cambridge, MA, USA agrawal@merl.com Ramesh Raskar MIT Media Lab Ames St., Cambridge,

More information

Project 4 Results http://www.cs.brown.edu/courses/cs129/results/proj4/jcmace/ http://www.cs.brown.edu/courses/cs129/results/proj4/damoreno/ http://www.cs.brown.edu/courses/csci1290/results/proj4/huag/

More information

Coded Exposure Deblurring: Optimized Codes for PSF Estimation and Invertibility

Coded Exposure Deblurring: Optimized Codes for PSF Estimation and Invertibility Coded Exposure Deblurring: Optimized Codes for PSF Estimation and Invertibility Amit Agrawal Yi Xu Mitsubishi Electric Research Labs (MERL) 201 Broadway, Cambridge, MA, USA [agrawal@merl.com,xu43@cs.purdue.edu]

More information

Near-Invariant Blur for Depth and 2D Motion via Time-Varying Light Field Analysis

Near-Invariant Blur for Depth and 2D Motion via Time-Varying Light Field Analysis Near-Invariant Blur for Depth and 2D Motion via Time-Varying Light Field Analysis Yosuke Bando 1,2 Henry Holtzman 2 Ramesh Raskar 2 1 Toshiba Corporation 2 MIT Media Lab Defocus & Motion Blur PSF Depth

More information

To Denoise or Deblur: Parameter Optimization for Imaging Systems

To Denoise or Deblur: Parameter Optimization for Imaging Systems To Denoise or Deblur: Parameter Optimization for Imaging Systems Kaushik Mitra, Oliver Cossairt and Ashok Veeraraghavan 1 ECE, Rice University 2 EECS, Northwestern University 3/3/2014 1 Capture moving

More information

When Does Computational Imaging Improve Performance?

When Does Computational Imaging Improve Performance? When Does Computational Imaging Improve Performance? Oliver Cossairt Assistant Professor Northwestern University Collaborators: Mohit Gupta, Changyin Zhou, Daniel Miau, Shree Nayar (Columbia University)

More information

Computational Photography

Computational Photography Computational photography Computational Photography Digital Visual Effects Yung-Yu Chuang wikipedia: Computational photography h refers broadly to computational imaging techniques that enhance or extend

More information

Admin Deblurring & Deconvolution Different types of blur

Admin Deblurring & Deconvolution Different types of blur Admin Assignment 3 due Deblurring & Deconvolution Lecture 10 Last lecture Move to Friday? Projects Come and see me Different types of blur Camera shake User moving hands Scene motion Objects in the scene

More information

Introduction to Light Fields

Introduction to Light Fields MIT Media Lab Introduction to Light Fields Camera Culture Ramesh Raskar MIT Media Lab http://cameraculture.media.mit.edu/ Introduction to Light Fields Ray Concepts for 4D and 5D Functions Propagation of

More information

Sensing Increased Image Resolution Using Aperture Masks

Sensing Increased Image Resolution Using Aperture Masks Sensing Increased Image Resolution Using Aperture Masks Ankit Mohan, Xiang Huang, Jack Tumblin Northwestern University Ramesh Raskar MIT Media Lab CVPR 2008 Supplemental Material Contributions Achieve

More information

Deconvolution , , Computational Photography Fall 2017, Lecture 17

Deconvolution , , Computational Photography Fall 2017, Lecture 17 Deconvolution http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2017, Lecture 17 Course announcements Homework 4 is out. - Due October 26 th. - There was another

More information

Raskar, Camera Culture, MIT Media Lab. Ramesh Raskar. Camera Culture. Associate Professor, MIT Media Lab

Raskar, Camera Culture, MIT Media Lab. Ramesh Raskar. Camera Culture. Associate Professor, MIT Media Lab Raskar, Camera Culture, MIT Media Lab Camera Culture Ramesh Raskar C C lt Camera Culture Associate Professor, MIT Media Lab Where are the camera s? Where are the camera s? We focus on creating tools to

More information

A Review over Different Blur Detection Techniques in Image Processing

A Review over Different Blur Detection Techniques in Image Processing A Review over Different Blur Detection Techniques in Image Processing 1 Anupama Sharma, 2 Devarshi Shukla 1 E.C.E student, 2 H.O.D, Department of electronics communication engineering, LR College of engineering

More information

Less Is More: Coded Computational Photography

Less Is More: Coded Computational Photography Less Is More: Coded Computational Photography Ramesh Raskar Mitsubishi Electric Research Labs (MERL), Cambridge, MA, USA Abstract. Computational photography combines plentiful computing, digital sensors,

More information

Transfer Efficiency and Depth Invariance in Computational Cameras

Transfer Efficiency and Depth Invariance in Computational Cameras Transfer Efficiency and Depth Invariance in Computational Cameras Jongmin Baek Stanford University IEEE International Conference on Computational Photography 2010 Jongmin Baek (Stanford University) Transfer

More information

Coded Aperture and Coded Exposure Photography

Coded Aperture and Coded Exposure Photography Coded Aperture and Coded Exposure Photography Martin Wilson University of Cape Town Cape Town, South Africa Email: Martin.Wilson@uct.ac.za Fred Nicolls University of Cape Town Cape Town, South Africa Email:

More information

Implementation of Image Deblurring Techniques in Java

Implementation of Image Deblurring Techniques in Java Implementation of Image Deblurring Techniques in Java Peter Chapman Computer Systems Lab 2007-2008 Thomas Jefferson High School for Science and Technology Alexandria, Virginia January 22, 2008 Abstract

More information

Computational Cameras. Rahul Raguram COMP

Computational Cameras. Rahul Raguram COMP Computational Cameras Rahul Raguram COMP 790-090 What is a computational camera? Camera optics Camera sensor 3D scene Traditional camera Final image Modified optics Camera sensor Image Compute 3D scene

More information

Computational Photography Introduction

Computational Photography Introduction Computational Photography Introduction Jongmin Baek CS 478 Lecture Jan 9, 2012 Background Sales of digital cameras surpassed sales of film cameras in 2004. Digital cameras are cool Free film Instant display

More information

Simulated Programmable Apertures with Lytro

Simulated Programmable Apertures with Lytro Simulated Programmable Apertures with Lytro Yangyang Yu Stanford University yyu10@stanford.edu Abstract This paper presents a simulation method using the commercial light field camera Lytro, which allows

More information

The ultimate camera. Computational Photography. Creating the ultimate camera. The ultimate camera. What does it do?

The ultimate camera. Computational Photography. Creating the ultimate camera. The ultimate camera. What does it do? Computational Photography The ultimate camera What does it do? Image from Durand & Freeman s MIT Course on Computational Photography Today s reading Szeliski Chapter 9 The ultimate camera Infinite resolution

More information

Coded Aperture for Projector and Camera for Robust 3D measurement

Coded Aperture for Projector and Camera for Robust 3D measurement Coded Aperture for Projector and Camera for Robust 3D measurement Yuuki Horita Yuuki Matugano Hiroki Morinaga Hiroshi Kawasaki Satoshi Ono Makoto Kimura Yasuo Takane Abstract General active 3D measurement

More information

Coded Aperture Pairs for Depth from Defocus

Coded Aperture Pairs for Depth from Defocus Coded Aperture Pairs for Depth from Defocus Changyin Zhou Columbia University New York City, U.S. changyin@cs.columbia.edu Stephen Lin Microsoft Research Asia Beijing, P.R. China stevelin@microsoft.com

More information

To Denoise or Deblur: Parameter Optimization for Imaging Systems

To Denoise or Deblur: Parameter Optimization for Imaging Systems To Denoise or Deblur: Parameter Optimization for Imaging Systems Kaushik Mitra a, Oliver Cossairt b and Ashok Veeraraghavan a a Electrical and Computer Engineering, Rice University, Houston, TX 77005 b

More information

Deconvolution , , Computational Photography Fall 2018, Lecture 12

Deconvolution , , Computational Photography Fall 2018, Lecture 12 Deconvolution http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 12 Course announcements Homework 3 is out. - Due October 12 th. - Any questions?

More information

Recent Advances in Image Deblurring. Seungyong Lee (Collaboration w/ Sunghyun Cho)

Recent Advances in Image Deblurring. Seungyong Lee (Collaboration w/ Sunghyun Cho) Recent Advances in Image Deblurring Seungyong Lee (Collaboration w/ Sunghyun Cho) Disclaimer Many images and figures in this course note have been copied from the papers and presentation materials of previous

More information

Improved motion invariant imaging with time varying shutter functions

Improved motion invariant imaging with time varying shutter functions Improved motion invariant imaging with time varying shutter functions Steve Webster a and Andrew Dorrell b Canon Information Systems Research, Australia (CiSRA), Thomas Holt Drive, North Ryde, Australia

More information

What are Good Apertures for Defocus Deblurring?

What are Good Apertures for Defocus Deblurring? What are Good Apertures for Defocus Deblurring? Changyin Zhou, Shree Nayar Abstract In recent years, with camera pixels shrinking in size, images are more likely to include defocused regions. In order

More information

Modeling and Synthesis of Aperture Effects in Cameras

Modeling and Synthesis of Aperture Effects in Cameras Modeling and Synthesis of Aperture Effects in Cameras Douglas Lanman, Ramesh Raskar, and Gabriel Taubin Computational Aesthetics 2008 20 June, 2008 1 Outline Introduction and Related Work Modeling Vignetting

More information

Blur and Recovery with FTVd. By: James Kerwin Zhehao Li Shaoyi Su Charles Park

Blur and Recovery with FTVd. By: James Kerwin Zhehao Li Shaoyi Su Charles Park Blur and Recovery with FTVd By: James Kerwin Zhehao Li Shaoyi Su Charles Park Blur and Recovery with FTVd By: James Kerwin Zhehao Li Shaoyi Su Charles Park Online: < http://cnx.org/content/col11395/1.1/

More information

Resolving Objects at Higher Resolution from a Single Motion-blurred Image

Resolving Objects at Higher Resolution from a Single Motion-blurred Image MITSUBISHI ELECTRIC RESEARCH LABORATORIES http://www.merl.com Resolving Objects at Higher Resolution from a Single Motion-blurred Image Amit Agrawal, Ramesh Raskar TR2007-036 July 2007 Abstract Motion

More information

Extended Depth of Field Catadioptric Imaging Using Focal Sweep

Extended Depth of Field Catadioptric Imaging Using Focal Sweep Extended Depth of Field Catadioptric Imaging Using Focal Sweep Ryunosuke Yokoya Columbia University New York, NY 10027 yokoya@cs.columbia.edu Shree K. Nayar Columbia University New York, NY 10027 nayar@cs.columbia.edu

More information

A Mathematical model for the determination of distance of an object in a 2D image

A Mathematical model for the determination of distance of an object in a 2D image A Mathematical model for the determination of distance of an object in a 2D image Deepu R 1, Murali S 2,Vikram Raju 3 Maharaja Institute of Technology Mysore, Karnataka, India rdeepusingh@mitmysore.in

More information

Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing

Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing MITSUBISHI ELECTRIC RESEARCH LABORATORIES http://www.merl.com Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing Ashok Veeraraghavan, Ramesh Raskar, Amit

More information

Admin. Lightfields. Overview. Overview 5/13/2008. Idea. Projects due by the end of today. Lecture 13. Lightfield representation of a scene

Admin. Lightfields. Overview. Overview 5/13/2008. Idea. Projects due by the end of today. Lecture 13. Lightfield representation of a scene Admin Lightfields Projects due by the end of today Email me source code, result images and short report Lecture 13 Overview Lightfield representation of a scene Unified representation of all rays Overview

More information

Wavelengths and Colors. Ankit Mohan MAS.131/531 Fall 2009

Wavelengths and Colors. Ankit Mohan MAS.131/531 Fall 2009 Wavelengths and Colors Ankit Mohan MAS.131/531 Fall 2009 Epsilon over time (Multiple photos) Prokudin-Gorskii, Sergei Mikhailovich, 1863-1944, photographer. Congress. Epsilon over time (Bracketing) Image

More information

Wavefront coding. Refocusing & Light Fields. Wavefront coding. Final projects. Is depth of field a blur? Frédo Durand Bill Freeman MIT - EECS

Wavefront coding. Refocusing & Light Fields. Wavefront coding. Final projects. Is depth of field a blur? Frédo Durand Bill Freeman MIT - EECS 6.098 Digital and Computational Photography 6.882 Advanced Computational Photography Final projects Send your slides by noon on Thrusday. Send final report Refocusing & Light Fields Frédo Durand Bill Freeman

More information

Computational Approaches to Cameras

Computational Approaches to Cameras Computational Approaches to Cameras 11/16/17 Magritte, The False Mirror (1935) Computational Photography Derek Hoiem, University of Illinois Announcements Final project proposal due Monday (see links on

More information

Computational Photography and Video. Prof. Marc Pollefeys

Computational Photography and Video. Prof. Marc Pollefeys Computational Photography and Video Prof. Marc Pollefeys Today s schedule Introduction of Computational Photography Course facts Syllabus Digital Photography What is computational photography Convergence

More information

Implementation of Adaptive Coded Aperture Imaging using a Digital Micro-Mirror Device for Defocus Deblurring

Implementation of Adaptive Coded Aperture Imaging using a Digital Micro-Mirror Device for Defocus Deblurring Implementation of Adaptive Coded Aperture Imaging using a Digital Micro-Mirror Device for Defocus Deblurring Ashill Chiranjan and Bernardt Duvenhage Defence, Peace, Safety and Security Council for Scientific

More information

Coded Aperture Flow. Anita Sellent and Paolo Favaro

Coded Aperture Flow. Anita Sellent and Paolo Favaro Coded Aperture Flow Anita Sellent and Paolo Favaro Institut für Informatik und angewandte Mathematik, Universität Bern, Switzerland http://www.cvg.unibe.ch/ Abstract. Real cameras have a limited depth

More information

Improving Signal- to- noise Ratio in Remotely Sensed Imagery Using an Invertible Blur Technique

Improving Signal- to- noise Ratio in Remotely Sensed Imagery Using an Invertible Blur Technique Improving Signal- to- noise Ratio in Remotely Sensed Imagery Using an Invertible Blur Technique Linda K. Le a and Carl Salvaggio a a Rochester Institute of Technology, Center for Imaging Science, Digital

More information

Analysis of Coded Apertures for Defocus Deblurring of HDR Images

Analysis of Coded Apertures for Defocus Deblurring of HDR Images CEIG - Spanish Computer Graphics Conference (2012) Isabel Navazo and Gustavo Patow (Editors) Analysis of Coded Apertures for Defocus Deblurring of HDR Images Luis Garcia, Lara Presa, Diego Gutierrez and

More information

Improving Film-Like Photography. aka, Epsilon Photography

Improving Film-Like Photography. aka, Epsilon Photography Improving Film-Like Photography aka, Epsilon Photography Ankit Mohan Courtesy of Ankit Mohan. Used with permission. Film-like like Optics: Imaging Intuition Angle(θ,ϕ) Ray Center of Projection Position

More information

Image Deblurring. This chapter describes how to deblur an image using the toolbox deblurring functions.

Image Deblurring. This chapter describes how to deblur an image using the toolbox deblurring functions. 12 Image Deblurring This chapter describes how to deblur an image using the toolbox deblurring functions. Understanding Deblurring (p. 12-2) Using the Deblurring Functions (p. 12-5) Avoiding Ringing in

More information

Sensing Increased Image Resolution Using Aperture Masks

Sensing Increased Image Resolution Using Aperture Masks Sensing Increased Image Resolution Using Aperture Masks Ankit Mohan, Xiang Huang, Jack Tumblin EECS Department, Northwestern University http://www.cs.northwestern.edu/ amohan Ramesh Raskar Mitsubishi Electric

More information

Point Spread Function Engineering for Scene Recovery. Changyin Zhou

Point Spread Function Engineering for Scene Recovery. Changyin Zhou Point Spread Function Engineering for Scene Recovery Changyin Zhou Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Graduate School of Arts and Sciences

More information

Removal of Glare Caused by Water Droplets

Removal of Glare Caused by Water Droplets 2009 Conference for Visual Media Production Removal of Glare Caused by Water Droplets Takenori Hara 1, Hideo Saito 2, Takeo Kanade 3 1 Dai Nippon Printing, Japan hara-t6@mail.dnp.co.jp 2 Keio University,

More information

Toward Non-stationary Blind Image Deblurring: Models and Techniques

Toward Non-stationary Blind Image Deblurring: Models and Techniques Toward Non-stationary Blind Image Deblurring: Models and Techniques Ji, Hui Department of Mathematics National University of Singapore NUS, 30-May-2017 Outline of the talk Non-stationary Image blurring

More information

Focal Sweep Videography with Deformable Optics

Focal Sweep Videography with Deformable Optics Focal Sweep Videography with Deformable Optics Daniel Miau Columbia University dmiau@cs.columbia.edu Oliver Cossairt Northwestern University ollie@eecs.northwestern.edu Shree K. Nayar Columbia University

More information

Lenses, exposure, and (de)focus

Lenses, exposure, and (de)focus Lenses, exposure, and (de)focus http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2017, Lecture 15 Course announcements Homework 4 is out. - Due October 26

More information

Burst Photography! EE367/CS448I: Computational Imaging and Display! stanford.edu/class/ee367! Lecture 7! Gordon Wetzstein! Stanford University!

Burst Photography! EE367/CS448I: Computational Imaging and Display! stanford.edu/class/ee367! Lecture 7! Gordon Wetzstein! Stanford University! Burst Photography! EE367/CS448I: Computational Imaging and Display! stanford.edu/class/ee367! Lecture 7! Gordon Wetzstein! Stanford University! Motivation! wikipedia! exposure sequence! -4 stops! Motivation!

More information

Reinterpretable Imager: Towards Variable Post-Capture Space, Angle and Time Resolution in Photography

Reinterpretable Imager: Towards Variable Post-Capture Space, Angle and Time Resolution in Photography Reinterpretable Imager: Towards Variable Post-Capture Space, Angle and Time Resolution in Photography The MIT Faculty has made this article openly available. Please share how this access benefits you.

More information

4 STUDY OF DEBLURRING TECHNIQUES FOR RESTORED MOTION BLURRED IMAGES

4 STUDY OF DEBLURRING TECHNIQUES FOR RESTORED MOTION BLURRED IMAGES 4 STUDY OF DEBLURRING TECHNIQUES FOR RESTORED MOTION BLURRED IMAGES Abstract: This paper attempts to undertake the study of deblurring techniques for Restored Motion Blurred Images by using: Wiener filter,

More information

High dynamic range imaging and tonemapping

High dynamic range imaging and tonemapping High dynamic range imaging and tonemapping http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2017, Lecture 12 Course announcements Homework 3 is out. - Due

More information

High resolution extended depth of field microscopy using wavefront coding

High resolution extended depth of field microscopy using wavefront coding High resolution extended depth of field microscopy using wavefront coding Matthew R. Arnison *, Peter Török #, Colin J. R. Sheppard *, W. T. Cathey +, Edward R. Dowski, Jr. +, Carol J. Cogswell *+ * Physical

More information

Motion Deblurring using Coded Exposure for a Wheeled Mobile Robot Kibaek Park, Seunghak Shin, Hae-Gon Jeon, Joon-Young Lee and In So Kweon

Motion Deblurring using Coded Exposure for a Wheeled Mobile Robot Kibaek Park, Seunghak Shin, Hae-Gon Jeon, Joon-Young Lee and In So Kweon Motion Deblurring using Coded Exposure for a Wheeled Mobile Robot Kibaek Park, Seunghak Shin, Hae-Gon Jeon, Joon-Young Lee and In So Kweon Korea Advanced Institute of Science and Technology, Daejeon 373-1,

More information

Motion Estimation from a Single Blurred Image

Motion Estimation from a Single Blurred Image Motion Estimation from a Single Blurred Image Image Restoration: De-Blurring Build a Blur Map Adapt Existing De-blurring Techniques to real blurred images Analysis, Reconstruction and 3D reconstruction

More information

Compressive Imaging. Aswin Sankaranarayanan (Computational Photography Fall 2017)

Compressive Imaging. Aswin Sankaranarayanan (Computational Photography Fall 2017) Compressive Imaging Aswin Sankaranarayanan (Computational Photography Fall 2017) Traditional Models for Sensing Linear (for the most part) Take as many measurements as unknowns sample Traditional Models

More information

2D Barcode Localization and Motion Deblurring Using a Flutter Shutter Camera

2D Barcode Localization and Motion Deblurring Using a Flutter Shutter Camera 2D Barcode Localization and Motion Deblurring Using a Flutter Shutter Camera Wei Xu University of Colorado at Boulder Boulder, CO, USA Wei.Xu@colorado.edu Scott McCloskey Honeywell Labs Minneapolis, MN,

More information

Image and Depth from a Single Defocused Image Using Coded Aperture Photography

Image and Depth from a Single Defocused Image Using Coded Aperture Photography Image and Depth from a Single Defocused Image Using Coded Aperture Photography Mina Masoudifar a, Hamid Reza Pourreza a a Department of Computer Engineering, Ferdowsi University of Mashhad, Mashhad, Iran

More information

KAUSHIK MITRA CURRENT POSITION. Assistant Professor at Department of Electrical Engineering, Indian Institute of Technology Madras, Chennai.

KAUSHIK MITRA CURRENT POSITION. Assistant Professor at Department of Electrical Engineering, Indian Institute of Technology Madras, Chennai. KAUSHIK MITRA School Address Department of Electrical Engineering Indian Institute of Technology Madras Chennai, TN, India 600036 Web: www.ee.iitm.ac.in/kmitra Email: kmitra@ee.iitm.ac.in Contact: 91-44-22574411

More information

Depth from Diffusion

Depth from Diffusion Depth from Diffusion Changyin Zhou Oliver Cossairt Shree Nayar Columbia University Supported by ONR Optical Diffuser Optical Diffuser ~ 10 micron Micrograph of a Holographic Diffuser (RPC Photonics) [Gray,

More information

Understanding camera trade-offs through a Bayesian analysis of light field projections Anat Levin, William T. Freeman, and Fredo Durand

Understanding camera trade-offs through a Bayesian analysis of light field projections Anat Levin, William T. Freeman, and Fredo Durand Computer Science and Artificial Intelligence Laboratory Technical Report MIT-CSAIL-TR-2008-021 April 16, 2008 Understanding camera trade-offs through a Bayesian analysis of light field projections Anat

More information

ELEC Dr Reji Mathew Electrical Engineering UNSW

ELEC Dr Reji Mathew Electrical Engineering UNSW ELEC 4622 Dr Reji Mathew Electrical Engineering UNSW Filter Design Circularly symmetric 2-D low-pass filter Pass-band radial frequency: ω p Stop-band radial frequency: ω s 1 δ p Pass-band tolerances: δ

More information

CVPR Easter School. Michael S. Brown. School of Computing National University of Singapore

CVPR Easter School. Michael S. Brown. School of Computing National University of Singapore Computational Photography CVPR Easter School March 14 18 18 th, 2011, ANU Kioloa Coastal Campus Michael S. Brown School of Computing National University of Singapore Goal of this tutorial Introduce you

More information

Motion-invariant Coding Using a Programmable Aperture Camera

Motion-invariant Coding Using a Programmable Aperture Camera [DOI: 10.2197/ipsjtcva.6.25] Research Paper Motion-invariant Coding Using a Programmable Aperture Camera Toshiki Sonoda 1,a) Hajime Nagahara 1,b) Rin-ichiro Taniguchi 1,c) Received: October 22, 2013, Accepted:

More information

The Flutter Shutter Camera Simulator

The Flutter Shutter Camera Simulator 2014/07/01 v0.5 IPOL article class Published in Image Processing On Line on 2012 10 17. Submitted on 2012 00 00, accepted on 2012 00 00. ISSN 2105 1232 c 2012 IPOL & the authors CC BY NC SA This article

More information

Computational Illumination

Computational Illumination Computational Illumination Course WebPage : http://www.merl.com/people/raskar/photo/ Ramesh Raskar Mitsubishi Electric Research Labs Ramesh Raskar, Computational Illumination Computational Illumination

More information

Optical image stabilization (IS)

Optical image stabilization (IS) Optical image stabilization (IS) CS 178, Spring 2010 Marc Levoy Computer Science Department Stanford University Outline! what are the causes of camera shake? how can you avoid it (without having an IS

More information

Flexible Depth of Field Photography

Flexible Depth of Field Photography TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 1 Flexible Depth of Field Photography Sujit Kuthirummal, Hajime Nagahara, Changyin Zhou, and Shree K. Nayar Abstract The range of scene depths

More information

Perceptually-Optimized Coded Apertures for Defocus Deblurring

Perceptually-Optimized Coded Apertures for Defocus Deblurring Volume 0 (1981), Number 0 pp. 1 12 COMPUTER GRAPHICS forum Perceptually-Optimized Coded Apertures for Defocus Deblurring Belen Masia, Lara Presa, Adrian Corrales and Diego Gutierrez Universidad de Zaragoza,

More information

CS354 Computer Graphics Computational Photography. Qixing Huang April 23 th 2018

CS354 Computer Graphics Computational Photography. Qixing Huang April 23 th 2018 CS354 Computer Graphics Computational Photography Qixing Huang April 23 th 2018 Background Sales of digital cameras surpassed sales of film cameras in 2004 Digital Cameras Free film Instant display Quality

More information

Get the Shot! Photography + Instagram Workshop September 21, 2013 BlogPodium. Saturday, 21 September, 13

Get the Shot! Photography + Instagram Workshop September 21, 2013 BlogPodium. Saturday, 21 September, 13 Get the Shot! Photography + Instagram Workshop September 21, 2013 BlogPodium Part One: Taking your camera off manual Technical details Common problems and how to fix them Practice Ways to make your photos

More information

An Analysis of Focus Sweep for Improved 2D Motion Invariance

An Analysis of Focus Sweep for Improved 2D Motion Invariance 3 IEEE Conference on Computer Vision and Pattern Recognition Workshops An Analysis of Focus Sweep for Improved D Motion Invariance Yosuke Bando TOSHIBA Corporation yosuke.bando@toshiba.co.jp Abstract Recent

More information

Single-view Metrology and Cameras

Single-view Metrology and Cameras Single-view Metrology and Cameras 10/10/17 Computational Photography Derek Hoiem, University of Illinois Project 2 Results Incomplete list of great project pages Haohang Huang: Best presented project;

More information

Supplementary Information

Supplementary Information Supplementary Information Simultaneous whole- animal 3D- imaging of neuronal activity using light field microscopy Robert Prevedel 1-3,10, Young- Gyu Yoon 4,5,10, Maximilian Hoffmann,1-3, Nikita Pak 5,6,

More information

Coded Exposure Photography: Motion Deblurring using Fluttered Shutter

Coded Exposure Photography: Motion Deblurring using Fluttered Shutter MITSUBISHI ELECTRIC RESEARCH LABORATORIES http://www.merl.com Coded Exposure Photography: Motion Deblurring using Fluttered Shutter Ramesh Raskar, Amit Agrawal, Jack Tumblin TR26-124 July 26 Abstract In

More information

Progressive Inter-scale and Intra-scale Non-blind Image Deconvolution

Progressive Inter-scale and Intra-scale Non-blind Image Deconvolution Progressive Inter-scale and Intra-scale Non-blind Image Deconvolution Lu Yuan 1 Jian Sun 2 Long Quan 1 Heung-Yeung Shum 2 1 The Hong Kong University of Science and Technology 2 Microsoft Research Asia

More information

Computational Photography: Principles and Practice

Computational Photography: Principles and Practice Computational Photography: Principles and Practice HCI & Robotics (HCI 및로봇응용공학 ) Ig-Jae Kim, Korea Institute of Science and Technology ( 한국과학기술연구원김익재 ) Jaewon Kim, Korea Institute of Science and Technology

More information

The multi integration mode of the 2 MP CMOS sensor from e2v used in the USB 3 ueye families opens up exciting new opportunities in machine vision.

The multi integration mode of the 2 MP CMOS sensor from e2v used in the USB 3 ueye families opens up exciting new opportunities in machine vision. Multi integration mode: Multiple exposures within one image The multi integration mode of the 2 MP CMOS sensor from e2v used in the USB 3 ueye families opens up exciting new opportunities in machine vision.

More information

Modeling and Synthesis of Aperture Effects in Cameras

Modeling and Synthesis of Aperture Effects in Cameras Computational Aesthetics in Graphics, Visualization, and Imaging (2008) P. Brown, D. W. Cunningham, V. Interrante, and J. McCormack (Editors) Modeling and Synthesis of Aperture Effects in Cameras Douglas

More information

Single Image Blind Deconvolution with Higher-Order Texture Statistics

Single Image Blind Deconvolution with Higher-Order Texture Statistics Single Image Blind Deconvolution with Higher-Order Texture Statistics Manuel Martinello and Paolo Favaro Heriot-Watt University School of EPS, Edinburgh EH14 4AS, UK Abstract. We present a novel method

More information

MAS.963 Special Topics: Computational Camera and Photography

MAS.963 Special Topics: Computational Camera and Photography MIT OpenCourseWare http://ocw.mit.edu MAS.963 Special Topics: Computational Camera and Photography Fall 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

More information

Light field sensing. Marc Levoy. Computer Science Department Stanford University

Light field sensing. Marc Levoy. Computer Science Department Stanford University Light field sensing Marc Levoy Computer Science Department Stanford University The scalar light field (in geometrical optics) Radiance as a function of position and direction in a static scene with fixed

More information

Removing Motion Blur with Space-Time Processing

Removing Motion Blur with Space-Time Processing 1 Removing Motion Blur with Space-Time Processing Hiroyuki Takeda, Student Member, IEEE, Peyman Milanfar, Fellow, IEEE Abstract Although spatial deblurring is relatively well-understood by assuming that

More information

Motion Deblurring Using Hybrid Imaging

Motion Deblurring Using Hybrid Imaging Motion Deblurring Using Hybrid Imaging Moshe Ben-Ezra and Shree K. Nayar Computer Science Department, Columbia University New York, NY, USA E-mail: {moshe, nayar}@cs.columbia.edu Abstract Motion blur due

More information

multiframe visual-inertial blur estimation and removal for unmodified smartphones

multiframe visual-inertial blur estimation and removal for unmodified smartphones multiframe visual-inertial blur estimation and removal for unmodified smartphones, Severin Münger, Carlo Beltrame, Luc Humair WSCG 2015, Plzen, Czech Republic images taken by non-professional photographers

More information

High Resolution Spectral Video Capture & Computational Photography Xun Cao ( 曹汛 )

High Resolution Spectral Video Capture & Computational Photography Xun Cao ( 曹汛 ) High Resolution Spectral Video Capture & Computational Photography Xun Cao ( 曹汛 ) School of Electronic Science & Engineering Nanjing University caoxun@nju.edu.cn Dec 30th, 2015 Computational Photography

More information

Defocus Map Estimation from a Single Image

Defocus Map Estimation from a Single Image Defocus Map Estimation from a Single Image Shaojie Zhuo Terence Sim School of Computing, National University of Singapore, Computing 1, 13 Computing Drive, Singapore 117417, SINGAPOUR Abstract In this

More information

Resolution. [from the New Merriam-Webster Dictionary, 1989 ed.]:

Resolution. [from the New Merriam-Webster Dictionary, 1989 ed.]: Resolution [from the New Merriam-Webster Dictionary, 1989 ed.]: resolve v : 1 to break up into constituent parts: ANALYZE; 2 to find an answer to : SOLVE; 3 DETERMINE, DECIDE; 4 to make or pass a formal

More information