A Framework for Analysis of Computational Imaging Systems

Size: px
Start display at page:

Download "A Framework for Analysis of Computational Imaging Systems"

Transcription

1 A Framework for Analysis of Computational Imaging Systems Kaushik Mitra, Oliver Cossairt, Ashok Veeraghavan Rice University Northwestern University

2 Computational imaging CI systems that adds new functionality Light Field Capture Structured Lighting HDR Imaging Motion deblurring system CI systems that improves performance Extended depth of field Others: Multiplexed Light field Illumination Spectography

3 How does CI improve performance? Increased light throughput Short exposure Flutter Shutter Increased light throughput but blurry Sharp, but noisy Deblurred image Slide coutesy Amit Agarwal

4 How does CI improve performance? Well conditioned optical coding Flutter Shutter Large exposure Captured image Deblurred image Slide coutesy Amit Agarwal

5 Recovered image Captured image One key flaw: Signal prior has not been taken into account Short exposure Flutter Shutter Large exposure BM3D denoising BM3D deblurring BM3D deblurring SNR= 17 db SNR= 19 db SNR= 13.4 db

6 State-of-the-art systems use signal prior Denoising using BM3D Coded exposure video using dictionary learning Inpainting using GMM Dabov et al., 2011 Hitomi et al., 2011 Yu et al., 2011

7 Our goal: A comprehensive analysis Scene Image Computational camera Multiplexed image Multiplexed image Multiplexing matrix (Read+photon) Noise Signal prior P(x) Our analysis takes into account: Signal prior Multiplexing matrix Noise characteristics Figure courtesy Oliver Cossairt

8 Prior Work: Analysis of CI systems 1. Analysis under read noise withour prior 2. Analysis under affine noise withour prior Harwitt et al Ratner et al. 2007, Wuttig 2007, Hasinoff et al. 2008, Ihrke et al. 2010, Cossairt et al Relates performance to practical considerations such as illumination, sensor characteristics, etc. Cossairt et al. 2012

9 Our analysis framework: GMM as signal prior Advantages of GMM 1.Universal approximation property 3. State-of-the-art results Image processing Yu et al LF processing Sorenson et al., Analytically tractable A special case is Gaussian prior, whose MMSE can be computed analytically Mitra et al. 2012

10 Our analysis framework: Linear system Multiplexed image Multiplexing matrix Noise Motion blur Defocus blur Single pixel camera [Raskar 06] [Levin 08] [Cho 10] Light Field Capture [Hausler 72] [Nagahara 08] [Dowski, Cathey 96] [Levin et al. 07] [Zhou, Nayar 08] Reflectance [Wakin et al., 2006] High speed video [Lanman 08] [Veeraraghavan 07] [Liang 08] [Schechner 03] [Ratner 07] [Ratner 08] [Hitomi et al. 2011][Veera et al., 2011]

11 Slide courtesy Oliver Cossairt Our analysis framework: Affine noise model Noise Variance at i th Pixel: photon noise aperture, lighting, pixel size read noise electronics, ADC s, quantization J i : i th pixel intensity Signal dependent / independent noise Ignore Dark current, fixed pattern Noise PDF: Photon noise modeled as Gaussian (good approx. if #photons > 10) Photon noise spatially averaged

12 Complete specification of the framework Multiplexed measurement Multiplexing matrix Noise Learn patch-based GMM prior GMM Cluster 1: mean and PCA components GMM Cluster 2: mean and PCA components Cluster weight Cluster mean GMM patch prior Cluster covariance

13 MMSE as a performance metric Mean Squared Error (MSE) of an estimator is defined as: MMSE estimator: Defined as the estimator that achieves the minimum MSE Given by the posterior mean MMSE is the corresponding MSE error MMSE: a scalar that characterizes the performance of a system H

14 Computation of MMSE estimator The posterior PDF is also a GMM: old weight with new weights Probability of y coming from kth cluster and with new mean and covariance: The MMSE estimator (posterior mean):

15 Interpretation of MMSE Intra-cluster error, can be computed analytically Inter-cluster error needs MC simulations We have an analytical lower bound for the MMSE: Tight bound for fully-determined system H

16 Limitations of analysis Patch-based GMM prior GMM Cluster 1 GMM Cluster 2 Local multiplexing Shift invariant blur (motion and focus) Other assumptions: Linear systems Affine noise [Nagahara 08] [Dowsky 96] [Levin 08]

17 Practical implications of the analysis

18 Practical system performance depends on 1. Illumination condition 2. Scene reflectivity 3. Camera parameters F/#, Exposure time t, quantum efficiency q, pixel size p Average signal-level is given by: Average Signal (e - ) Illumination (lux) Reflectivity Aperture Exposure Time (s) Quantum Efficiency Pixel Size (m)

19 Average signal level for three form factors SLR camera Pixel size p SLR = 8 μm Machine vision camera (MVC) p MVC = 2.5μm Smartphone camera (SPC) p SPC = 1μm Typical values of average signal level J for different illumination levels Other parameters: q=.5, R =.5, F/11, t = 6ms, σ r =4

20 Common analysis and simulation framework Learn GMM prior of patch size GMM Cluster 1 GMM Cluster 2 Analytical computations: Without prior: Under GMM prior: Simulation computations: Perform per-patch reconstruction. Let y be the observed patch. Without prior: Under GMM prior: Performance measure: SNR gain w.r.t impulse imaging

21 Analysis of Extended DOF systems

22 Depth of Field and SNR Image Lens F Small apertures have large depth of field and low SNR Slide courtesy Oliver Cossairt

23 Focal Sweep: An example EDOF system Sensor Lens (depth) Point Spread Function (PSF) [Hausler 72, Nagahara et al. 08] Slide courtesy Oliver Cossairt

24 Focal Sweep: An example EDOF system Sensor Lens (depth) = t = 1 t = 2 t = 3 t = 4 t = 5 t = 6 t = 7 (400) (600) (900) (1200) (1500) (1700) (2000) [Hausler 72, Nagahara et al. 08] Integrated PSF Slide courtesy Oliver Cossairt

25 Depth Invariant PSF Focal Sweep PSF Traditional Camera PSF mm 750mm mm mm 2000mm mm mm 2000mm Extended depth of field with a single deconvolution Slide courtesy Oliver Cossairt

26 Recovery with GMM Recovery without prior Captured image Simulation performance Impulse (F/11) Low light condition (10 lux) Focal sweep (F/1) Pixel size = 8 um Exp time = 6 ms SNR = 4.8 db ~5.5 db multiplexing gain SNR = 10.2 db ~11.5 db gain due to prior SNR = 15.7 db SNR = 22.4 db Gain due to prior is much greater than gain due to multiplexing

27 Recovery with GMM Recovery without prior Captured image Simulation performance Impulse (F/11) High light condition (1000 lux) Focal sweep (F/1) Pixel size = 8 um Exp time = 6 ms SNR = 29.5 db ~1 db multiplexing gain SNR = 30.3 db ~1.5 db gain due to prior SNR = 30.8 db SNR = 31.9 db At high light condition, gain due to both prior and multiplexing is negligible.

28 SNR gain (in db) Analytic performance: SNR gain vs. illumination level (without prior case) FS multiplexing gain without prior Impulse Photon to read noise ratio (J/σ 2 ) I SLR I MV I SP Huge multiplexing gain at low light levels

29 SNR gain (in db) Analytic performance: SNR gain vs. illumination level (with prior case) Impulse GMM FS FS GMM multiplexing gain with prior Gain due to prior alone multiplexing gain without prior Impulse Photon to read noise ratio (J/σ 2 ) I SLR I MV I SP Under signal prior moderate multiplexing gain at low light levels

30 Other EDOF systems Depth invariant PSF systems Coded aperture systems [Dowski, Cathey 96] [Cossairt et al. 10] [Levin et al. 07] [Zhou et al. 08] [Veeraraghavan et al. 07] Depth invariant PSF Coded Camera PSF mm 750mm mm mm 2000mm mm mm 2000mm

31 SNR gain (in db) Analytic performance with Prior Impulse camera: F/11 Other cameras: F/1 SNR gain vs. light level J/σ 2 I SLR I MVC I SPC Good EDOF systems perform 9 db better than impulse imaging

32 Analysis of motion deblurring systems

33 Light throughput vs. motion blur Increasing exposure time Noise decreases but motion blur increases

34 Motion deblurring CI systems Coded exposure (Flutter shutter) [Raskar 06] Motion invariant photography [Levin 08] [Cho 10] Increased light throughput and inversion better conditioned Captured image has same motion blur for different motions Captured image Deblurred image Whole image deblurred using a single blur kernel

35 Recovered Captured Simulation Performance under signal prior t CI =33 t impulse Impulse imaging Flutter Shutter Motion invariant Low light condition (10 lux) SNR= -1.7 db Motion invariant 7 db better than impulse SNR= 13.5 db SNR= 16.8 db SNR= 20.9 db High light condition (1000 lux) Motion invariant 1.2 db better than impulse Recovered Captured SNR= 25.8 db SNR= 28.2 db SNR= 24.7 db SNR= 29.4 db

36 SNR gain (in db) Analytic Performance under signal prior t CI =33 t impulse SNR gain vs. light level J/σ 2 I SLR I MVC I SPC Motion invariant camera achieves a peak SNR gain of 7.5 db

37 Conclusion: Comprehensive analysis framework of CI Scene Image Computational camera Multiplexed image Multiplexed image Multiplexing matrix Affine noise Our analysis accounts for Signal prior (GMM) Optical coding (H) Noise (affine) Signal prior P(x) GMM Cluster 1 GMM Cluster 2

38 Conclusion: Practical implications 1. Illumination condition 2. Scene reflectivity 3. Camera parameters F/#, Exposure time t, quantum efficiency q, pixel size p We analyzed EDOF and motion deblurring systems for typical values of: Illumination conditions Scene characteristics Camera parameters

39 Conclusion: Our observations More gain due to prior than multiplexing Gain due to multiplexing modest when prior is taken into account CI systems provide significant advantage over impulse imaging under various illumination and camera parameters EDOF systems provides on average 7 db gain over impulse imaging Motion deblurring systems provides on average 4.5 db gain over impulse imaging

40 Future Work Analyze compressive systems: High speed video Light Field Capture Multi/Hyper-Spectral [Hitomi et al. 2011][Veera et al., 2011] [Lanman 08] [Veeraraghavan 07] [Liang 08] Single pixel camera [Sloane 79] [Hanley 99] [Baer 99] [Wetzstein et al., 12] [Wakin et al., 2006] Design optimal CI systems

To Denoise or Deblur: Parameter Optimization for Imaging Systems

To Denoise or Deblur: Parameter Optimization for Imaging Systems To Denoise or Deblur: Parameter Optimization for Imaging Systems Kaushik Mitra, Oliver Cossairt and Ashok Veeraraghavan 1 ECE, Rice University 2 EECS, Northwestern University 3/3/2014 1 Capture moving

More information

When Does Computational Imaging Improve Performance?

When Does Computational Imaging Improve Performance? When Does Computational Imaging Improve Performance? Oliver Cossairt Assistant Professor Northwestern University Collaborators: Mohit Gupta, Changyin Zhou, Daniel Miau, Shree Nayar (Columbia University)

More information

To Denoise or Deblur: Parameter Optimization for Imaging Systems

To Denoise or Deblur: Parameter Optimization for Imaging Systems To Denoise or Deblur: Parameter Optimization for Imaging Systems Kaushik Mitra a, Oliver Cossairt b and Ashok Veeraraghavan a a Electrical and Computer Engineering, Rice University, Houston, TX 77005 b

More information

A Framework for Analysis of Computational Imaging Systems: Role of Signal Prior, Sensor Noise and Multiplexing

A Framework for Analysis of Computational Imaging Systems: Role of Signal Prior, Sensor Noise and Multiplexing SNR gain (in db) 1 A Framework for Analysis of Computational Imaging Systems: Role of Signal Prior, Sensor Noise and Multiplexing Kaushik Mitra, Member, IEEE, Oliver S. Cossairt, Member, IEEE and Ashok

More information

Coded Computational Photography!

Coded Computational Photography! Coded Computational Photography! EE367/CS448I: Computational Imaging and Display! stanford.edu/class/ee367! Lecture 9! Gordon Wetzstein! Stanford University! Coded Computational Photography - Overview!!

More information

Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing

Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing Ashok Veeraraghavan, Ramesh Raskar, Ankit Mohan & Jack Tumblin Amit Agrawal, Mitsubishi Electric Research

More information

Near-Invariant Blur for Depth and 2D Motion via Time-Varying Light Field Analysis

Near-Invariant Blur for Depth and 2D Motion via Time-Varying Light Field Analysis Near-Invariant Blur for Depth and 2D Motion via Time-Varying Light Field Analysis Yosuke Bando 1,2 Henry Holtzman 2 Ramesh Raskar 2 1 Toshiba Corporation 2 MIT Media Lab Defocus & Motion Blur PSF Depth

More information

Coded photography , , Computational Photography Fall 2018, Lecture 14

Coded photography , , Computational Photography Fall 2018, Lecture 14 Coded photography http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 14 Overview of today s lecture The coded photography paradigm. Dealing with

More information

Computational Camera & Photography: Coded Imaging

Computational Camera & Photography: Coded Imaging Computational Camera & Photography: Coded Imaging Camera Culture Ramesh Raskar MIT Media Lab http://cameraculture.media.mit.edu/ Image removed due to copyright restrictions. See Fig. 1, Eight major types

More information

Deblurring. Basics, Problem definition and variants

Deblurring. Basics, Problem definition and variants Deblurring Basics, Problem definition and variants Kinds of blur Hand-shake Defocus Credit: Kenneth Josephson Motion Credit: Kenneth Josephson Kinds of blur Spatially invariant vs. Spatially varying

More information

Coded photography , , Computational Photography Fall 2017, Lecture 18

Coded photography , , Computational Photography Fall 2017, Lecture 18 Coded photography http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2017, Lecture 18 Course announcements Homework 5 delayed for Tuesday. - You will need cameras

More information

Coding and Modulation in Cameras

Coding and Modulation in Cameras Coding and Modulation in Cameras Amit Agrawal June 2010 Mitsubishi Electric Research Labs (MERL) Cambridge, MA, USA Coded Computational Imaging Agrawal, Veeraraghavan, Narasimhan & Mohan Schedule Introduction

More information

Modeling and Synthesis of Aperture Effects in Cameras

Modeling and Synthesis of Aperture Effects in Cameras Modeling and Synthesis of Aperture Effects in Cameras Douglas Lanman, Ramesh Raskar, and Gabriel Taubin Computational Aesthetics 2008 20 June, 2008 1 Outline Introduction and Related Work Modeling Vignetting

More information

Optimal Single Image Capture for Motion Deblurring

Optimal Single Image Capture for Motion Deblurring Optimal Single Image Capture for Motion Deblurring Amit Agrawal Mitsubishi Electric Research Labs (MERL) 1 Broadway, Cambridge, MA, USA agrawal@merl.com Ramesh Raskar MIT Media Lab Ames St., Cambridge,

More information

Extended depth of field for visual measurement systems with depth-invariant magnification

Extended depth of field for visual measurement systems with depth-invariant magnification Extended depth of field for visual measurement systems with depth-invariant magnification Yanyu Zhao a and Yufu Qu* a,b a School of Instrument Science and Opto-Electronic Engineering, Beijing University

More information

Depth from Diffusion

Depth from Diffusion Depth from Diffusion Changyin Zhou Oliver Cossairt Shree Nayar Columbia University Supported by ONR Optical Diffuser Optical Diffuser ~ 10 micron Micrograph of a Holographic Diffuser (RPC Photonics) [Gray,

More information

Recent Advances in Image Deblurring. Seungyong Lee (Collaboration w/ Sunghyun Cho)

Recent Advances in Image Deblurring. Seungyong Lee (Collaboration w/ Sunghyun Cho) Recent Advances in Image Deblurring Seungyong Lee (Collaboration w/ Sunghyun Cho) Disclaimer Many images and figures in this course note have been copied from the papers and presentation materials of previous

More information

Coded Exposure Deblurring: Optimized Codes for PSF Estimation and Invertibility

Coded Exposure Deblurring: Optimized Codes for PSF Estimation and Invertibility Coded Exposure Deblurring: Optimized Codes for PSF Estimation and Invertibility Amit Agrawal Yi Xu Mitsubishi Electric Research Labs (MERL) 201 Broadway, Cambridge, MA, USA [agrawal@merl.com,xu43@cs.purdue.edu]

More information

Computational Photography Introduction

Computational Photography Introduction Computational Photography Introduction Jongmin Baek CS 478 Lecture Jan 9, 2012 Background Sales of digital cameras surpassed sales of film cameras in 2004. Digital cameras are cool Free film Instant display

More information

Deconvolution , , Computational Photography Fall 2017, Lecture 17

Deconvolution , , Computational Photography Fall 2017, Lecture 17 Deconvolution http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2017, Lecture 17 Course announcements Homework 4 is out. - Due October 26 th. - There was another

More information

Focal Sweep Videography with Deformable Optics

Focal Sweep Videography with Deformable Optics Focal Sweep Videography with Deformable Optics Daniel Miau Columbia University dmiau@cs.columbia.edu Oliver Cossairt Northwestern University ollie@eecs.northwestern.edu Shree K. Nayar Columbia University

More information

Transfer Efficiency and Depth Invariance in Computational Cameras

Transfer Efficiency and Depth Invariance in Computational Cameras Transfer Efficiency and Depth Invariance in Computational Cameras Jongmin Baek Stanford University IEEE International Conference on Computational Photography 2010 Jongmin Baek (Stanford University) Transfer

More information

Computational Cameras. Rahul Raguram COMP

Computational Cameras. Rahul Raguram COMP Computational Cameras Rahul Raguram COMP 790-090 What is a computational camera? Camera optics Camera sensor 3D scene Traditional camera Final image Modified optics Camera sensor Image Compute 3D scene

More information

Admin Deblurring & Deconvolution Different types of blur

Admin Deblurring & Deconvolution Different types of blur Admin Assignment 3 due Deblurring & Deconvolution Lecture 10 Last lecture Move to Friday? Projects Come and see me Different types of blur Camera shake User moving hands Scene motion Objects in the scene

More information

Compressive Imaging. Aswin Sankaranarayanan (Computational Photography Fall 2017)

Compressive Imaging. Aswin Sankaranarayanan (Computational Photography Fall 2017) Compressive Imaging Aswin Sankaranarayanan (Computational Photography Fall 2017) Traditional Models for Sensing Linear (for the most part) Take as many measurements as unknowns sample Traditional Models

More information

Extended Depth of Field Catadioptric Imaging Using Focal Sweep

Extended Depth of Field Catadioptric Imaging Using Focal Sweep Extended Depth of Field Catadioptric Imaging Using Focal Sweep Ryunosuke Yokoya Columbia University New York, NY 10027 yokoya@cs.columbia.edu Shree K. Nayar Columbia University New York, NY 10027 nayar@cs.columbia.edu

More information

Motion-invariant Coding Using a Programmable Aperture Camera

Motion-invariant Coding Using a Programmable Aperture Camera [DOI: 10.2197/ipsjtcva.6.25] Research Paper Motion-invariant Coding Using a Programmable Aperture Camera Toshiki Sonoda 1,a) Hajime Nagahara 1,b) Rin-ichiro Taniguchi 1,c) Received: October 22, 2013, Accepted:

More information

Deconvolution , , Computational Photography Fall 2018, Lecture 12

Deconvolution , , Computational Photography Fall 2018, Lecture 12 Deconvolution http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 12 Course announcements Homework 3 is out. - Due October 12 th. - Any questions?

More information

Image and Depth from a Single Defocused Image Using Coded Aperture Photography

Image and Depth from a Single Defocused Image Using Coded Aperture Photography Image and Depth from a Single Defocused Image Using Coded Aperture Photography Mina Masoudifar a, Hamid Reza Pourreza a a Department of Computer Engineering, Ferdowsi University of Mashhad, Mashhad, Iran

More information

Coded Aperture and Coded Exposure Photography

Coded Aperture and Coded Exposure Photography Coded Aperture and Coded Exposure Photography Martin Wilson University of Cape Town Cape Town, South Africa Email: Martin.Wilson@uct.ac.za Fred Nicolls University of Cape Town Cape Town, South Africa Email:

More information

Simulated Programmable Apertures with Lytro

Simulated Programmable Apertures with Lytro Simulated Programmable Apertures with Lytro Yangyang Yu Stanford University yyu10@stanford.edu Abstract This paper presents a simulation method using the commercial light field camera Lytro, which allows

More information

Coded Aperture for Projector and Camera for Robust 3D measurement

Coded Aperture for Projector and Camera for Robust 3D measurement Coded Aperture for Projector and Camera for Robust 3D measurement Yuuki Horita Yuuki Matugano Hiroki Morinaga Hiroshi Kawasaki Satoshi Ono Makoto Kimura Yasuo Takane Abstract General active 3D measurement

More information

Project 4 Results http://www.cs.brown.edu/courses/cs129/results/proj4/jcmace/ http://www.cs.brown.edu/courses/cs129/results/proj4/damoreno/ http://www.cs.brown.edu/courses/csci1290/results/proj4/huag/

More information

Short-course Compressive Sensing of Videos

Short-course Compressive Sensing of Videos Short-course Compressive Sensing of Videos Venue CVPR 2012, Providence, RI, USA June 16, 2012 Richard G. Baraniuk Mohit Gupta Aswin C. Sankaranarayanan Ashok Veeraraghavan Tutorial Outline Time Presenter

More information

The ultimate camera. Computational Photography. Creating the ultimate camera. The ultimate camera. What does it do?

The ultimate camera. Computational Photography. Creating the ultimate camera. The ultimate camera. What does it do? Computational Photography The ultimate camera What does it do? Image from Durand & Freeman s MIT Course on Computational Photography Today s reading Szeliski Chapter 9 The ultimate camera Infinite resolution

More information

Motion Estimation from a Single Blurred Image

Motion Estimation from a Single Blurred Image Motion Estimation from a Single Blurred Image Image Restoration: De-Blurring Build a Blur Map Adapt Existing De-blurring Techniques to real blurred images Analysis, Reconstruction and 3D reconstruction

More information

Flexible Depth of Field Photography

Flexible Depth of Field Photography TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 1 Flexible Depth of Field Photography Sujit Kuthirummal, Hajime Nagahara, Changyin Zhou, and Shree K. Nayar Abstract The range of scene depths

More information

Blur and Recovery with FTVd. By: James Kerwin Zhehao Li Shaoyi Su Charles Park

Blur and Recovery with FTVd. By: James Kerwin Zhehao Li Shaoyi Su Charles Park Blur and Recovery with FTVd By: James Kerwin Zhehao Li Shaoyi Su Charles Park Blur and Recovery with FTVd By: James Kerwin Zhehao Li Shaoyi Su Charles Park Online: < http://cnx.org/content/col11395/1.1/

More information

Toward Non-stationary Blind Image Deblurring: Models and Techniques

Toward Non-stationary Blind Image Deblurring: Models and Techniques Toward Non-stationary Blind Image Deblurring: Models and Techniques Ji, Hui Department of Mathematics National University of Singapore NUS, 30-May-2017 Outline of the talk Non-stationary Image blurring

More information

CS354 Computer Graphics Computational Photography. Qixing Huang April 23 th 2018

CS354 Computer Graphics Computational Photography. Qixing Huang April 23 th 2018 CS354 Computer Graphics Computational Photography Qixing Huang April 23 th 2018 Background Sales of digital cameras surpassed sales of film cameras in 2004 Digital Cameras Free film Instant display Quality

More information

Point Spread Function Engineering for Scene Recovery. Changyin Zhou

Point Spread Function Engineering for Scene Recovery. Changyin Zhou Point Spread Function Engineering for Scene Recovery Changyin Zhou Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Graduate School of Arts and Sciences

More information

Flexible Depth of Field Photography

Flexible Depth of Field Photography TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 1 Flexible Depth of Field Photography Sujit Kuthirummal, Hajime Nagahara, Changyin Zhou, and Shree K. Nayar Abstract The range of scene depths

More information

What are Good Apertures for Defocus Deblurring?

What are Good Apertures for Defocus Deblurring? What are Good Apertures for Defocus Deblurring? Changyin Zhou, Shree Nayar Abstract In recent years, with camera pixels shrinking in size, images are more likely to include defocused regions. In order

More information

KAUSHIK MITRA CURRENT POSITION. Assistant Professor at Department of Electrical Engineering, Indian Institute of Technology Madras, Chennai.

KAUSHIK MITRA CURRENT POSITION. Assistant Professor at Department of Electrical Engineering, Indian Institute of Technology Madras, Chennai. KAUSHIK MITRA School Address Department of Electrical Engineering Indian Institute of Technology Madras Chennai, TN, India 600036 Web: www.ee.iitm.ac.in/kmitra Email: kmitra@ee.iitm.ac.in Contact: 91-44-22574411

More information

Improved motion invariant imaging with time varying shutter functions

Improved motion invariant imaging with time varying shutter functions Improved motion invariant imaging with time varying shutter functions Steve Webster a and Andrew Dorrell b Canon Information Systems Research, Australia (CiSRA), Thomas Holt Drive, North Ryde, Australia

More information

Coded Exposure HDR Light-Field Video Recording

Coded Exposure HDR Light-Field Video Recording Coded Exposure HDR Light-Field Video Recording David C. Schedl, Clemens Birklbauer, and Oliver Bimber* Johannes Kepler University Linz *firstname.lastname@jku.at Exposure Sequence long exposed short HDR

More information

Stochastic Image Denoising using Minimum Mean Squared Error (Wiener) Filtering

Stochastic Image Denoising using Minimum Mean Squared Error (Wiener) Filtering Stochastic Image Denoising using Minimum Mean Squared Error (Wiener) Filtering L. Sahawneh, B. Carroll, Electrical and Computer Engineering, ECEN 670 Project, BYU Abstract Digital images and video used

More information

A Review over Different Blur Detection Techniques in Image Processing

A Review over Different Blur Detection Techniques in Image Processing A Review over Different Blur Detection Techniques in Image Processing 1 Anupama Sharma, 2 Devarshi Shukla 1 E.C.E student, 2 H.O.D, Department of electronics communication engineering, LR College of engineering

More information

Implementation of Adaptive Coded Aperture Imaging using a Digital Micro-Mirror Device for Defocus Deblurring

Implementation of Adaptive Coded Aperture Imaging using a Digital Micro-Mirror Device for Defocus Deblurring Implementation of Adaptive Coded Aperture Imaging using a Digital Micro-Mirror Device for Defocus Deblurring Ashill Chiranjan and Bernardt Duvenhage Defence, Peace, Safety and Security Council for Scientific

More information

Coded Aperture Pairs for Depth from Defocus

Coded Aperture Pairs for Depth from Defocus Coded Aperture Pairs for Depth from Defocus Changyin Zhou Columbia University New York City, U.S. changyin@cs.columbia.edu Stephen Lin Microsoft Research Asia Beijing, P.R. China stevelin@microsoft.com

More information

Generalized Assorted Camera Arrays: Robust Cross-channel Registration and Applications Jason Holloway, Kaushik Mitra, Sanjeev Koppal, Ashok

Generalized Assorted Camera Arrays: Robust Cross-channel Registration and Applications Jason Holloway, Kaushik Mitra, Sanjeev Koppal, Ashok Generalized Assorted Camera Arrays: Robust Cross-channel Registration and Applications Jason Holloway, Kaushik Mitra, Sanjeev Koppal, Ashok Veeraraghavan Cross-modal Imaging Hyperspectral Cross-modal Imaging

More information

THE depth of field (DOF) of an imaging system is the

THE depth of field (DOF) of an imaging system is the 58 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 33, NO. 1, JANUARY 2011 Flexible Depth of Field Photography Sujit Kuthirummal, Member, IEEE, Hajime Nagahara, Changyin Zhou, Student

More information

Tonemapping and bilateral filtering

Tonemapping and bilateral filtering Tonemapping and bilateral filtering http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 6 Course announcements Homework 2 is out. - Due September

More information

Image Denoising Using Statistical and Non Statistical Method

Image Denoising Using Statistical and Non Statistical Method Image Denoising Using Statistical and Non Statistical Method Ms. Shefali A. Uplenchwar 1, Mrs. P. J. Suryawanshi 2, Ms. S. G. Mungale 3 1MTech, Dept. of Electronics Engineering, PCE, Maharashtra, India

More information

Defocus Map Estimation from a Single Image

Defocus Map Estimation from a Single Image Defocus Map Estimation from a Single Image Shaojie Zhuo Terence Sim School of Computing, National University of Singapore, Computing 1, 13 Computing Drive, Singapore 117417, SINGAPOUR Abstract In this

More information

A Kalman-Filtering Approach to High Dynamic Range Imaging for Measurement Applications

A Kalman-Filtering Approach to High Dynamic Range Imaging for Measurement Applications A Kalman-Filtering Approach to High Dynamic Range Imaging for Measurement Applications IEEE Transactions on Image Processing, Vol. 21, No. 2, 2012 Eric Dedrick and Daniel Lau, Presented by Ran Shu School

More information

Burst Photography! EE367/CS448I: Computational Imaging and Display! stanford.edu/class/ee367! Lecture 7! Gordon Wetzstein! Stanford University!

Burst Photography! EE367/CS448I: Computational Imaging and Display! stanford.edu/class/ee367! Lecture 7! Gordon Wetzstein! Stanford University! Burst Photography! EE367/CS448I: Computational Imaging and Display! stanford.edu/class/ee367! Lecture 7! Gordon Wetzstein! Stanford University! Motivation! wikipedia! exposure sequence! -4 stops! Motivation!

More information

Analysis of Coded Apertures for Defocus Deblurring of HDR Images

Analysis of Coded Apertures for Defocus Deblurring of HDR Images CEIG - Spanish Computer Graphics Conference (2012) Isabel Navazo and Gustavo Patow (Editors) Analysis of Coded Apertures for Defocus Deblurring of HDR Images Luis Garcia, Lara Presa, Diego Gutierrez and

More information

Computational Photography

Computational Photography Computational photography Computational Photography Digital Visual Effects Yung-Yu Chuang wikipedia: Computational photography h refers broadly to computational imaging techniques that enhance or extend

More information

Lenses, exposure, and (de)focus

Lenses, exposure, and (de)focus Lenses, exposure, and (de)focus http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2017, Lecture 15 Course announcements Homework 4 is out. - Due October 26

More information

Acquisition. Some slides from: Yung-Yu Chuang (DigiVfx) Jan Neumann, Pat Hanrahan, Alexei Efros

Acquisition. Some slides from: Yung-Yu Chuang (DigiVfx) Jan Neumann, Pat Hanrahan, Alexei Efros Acquisition Some slides from: Yung-Yu Chuang (DigiVfx) Jan Neumann, Pat Hanrahan, Alexei Efros Image Acquisition Digital Camera Film Outline Pinhole camera Lens Lens aberrations Exposure Sensors Noise

More information

Tradeoffs and Limits in Computational Imaging. Oliver Cossairt

Tradeoffs and Limits in Computational Imaging. Oliver Cossairt Tradeoffs and Limits in Computational Imaging Oliver Cossairt Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Graduate School of Arts and Sciences COLUMBIA

More information

Visibility of Uncorrelated Image Noise

Visibility of Uncorrelated Image Noise Visibility of Uncorrelated Image Noise Jiajing Xu a, Reno Bowen b, Jing Wang c, and Joyce Farrell a a Dept. of Electrical Engineering, Stanford University, Stanford, CA. 94305 U.S.A. b Dept. of Psychology,

More information

An Analysis of Focus Sweep for Improved 2D Motion Invariance

An Analysis of Focus Sweep for Improved 2D Motion Invariance 3 IEEE Conference on Computer Vision and Pattern Recognition Workshops An Analysis of Focus Sweep for Improved D Motion Invariance Yosuke Bando TOSHIBA Corporation yosuke.bando@toshiba.co.jp Abstract Recent

More information

To Do. Advanced Computer Graphics. Outline. Computational Imaging. How do we see the world? Pinhole camera

To Do. Advanced Computer Graphics. Outline. Computational Imaging. How do we see the world? Pinhole camera Advanced Computer Graphics CSE 163 [Spring 2017], Lecture 14 Ravi Ramamoorthi http://www.cs.ucsd.edu/~ravir To Do Assignment 2 due May 19 Any last minute issues or questions? Next two lectures: Imaging,

More information

4 STUDY OF DEBLURRING TECHNIQUES FOR RESTORED MOTION BLURRED IMAGES

4 STUDY OF DEBLURRING TECHNIQUES FOR RESTORED MOTION BLURRED IMAGES 4 STUDY OF DEBLURRING TECHNIQUES FOR RESTORED MOTION BLURRED IMAGES Abstract: This paper attempts to undertake the study of deblurring techniques for Restored Motion Blurred Images by using: Wiener filter,

More information

Image acquisition. In both cases, the digital sensing element is one of the following: Line array Area array. Single sensor

Image acquisition. In both cases, the digital sensing element is one of the following: Line array Area array. Single sensor Image acquisition Digital images are acquired by direct digital acquisition (digital still/video cameras), or scanning material acquired as analog signals (slides, photographs, etc.). In both cases, the

More information

SUPER RESOLUTION INTRODUCTION

SUPER RESOLUTION INTRODUCTION SUPER RESOLUTION Jnanavardhini - Online MultiDisciplinary Research Journal Ms. Amalorpavam.G Assistant Professor, Department of Computer Sciences, Sambhram Academy of Management. Studies, Bangalore Abstract:-

More information

APJIMTC, Jalandhar, India. Keywords---Median filter, mean filter, adaptive filter, salt & pepper noise, Gaussian noise.

APJIMTC, Jalandhar, India. Keywords---Median filter, mean filter, adaptive filter, salt & pepper noise, Gaussian noise. Volume 3, Issue 10, October 2013 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com A Comparative

More information

Optimal Camera Parameters for Depth from Defocus

Optimal Camera Parameters for Depth from Defocus Optimal Camera Parameters for Depth from Defocus Fahim Mannan and Michael S. Langer School of Computer Science, McGill University Montreal, Quebec H3A E9, Canada. {fmannan, langer}@cim.mcgill.ca Abstract

More information

Demosaicing and Denoising on Simulated Light Field Images

Demosaicing and Denoising on Simulated Light Field Images Demosaicing and Denoising on Simulated Light Field Images Trisha Lian Stanford University tlian@stanford.edu Kyle Chiang Stanford University kchiang@stanford.edu Abstract Light field cameras use an array

More information

Recent advances in deblurring and image stabilization. Michal Šorel Academy of Sciences of the Czech Republic

Recent advances in deblurring and image stabilization. Michal Šorel Academy of Sciences of the Czech Republic Recent advances in deblurring and image stabilization Michal Šorel Academy of Sciences of the Czech Republic Camera shake stabilization Alternative to OIS (optical image stabilization) systems Should work

More information

Agenda. Fusion and Reconstruction. Image Fusion & Reconstruction. Image Fusion & Reconstruction. Dr. Yossi Rubner.

Agenda. Fusion and Reconstruction. Image Fusion & Reconstruction. Image Fusion & Reconstruction. Dr. Yossi Rubner. Fusion and Reconstruction Dr. Yossi Rubner yossi@rubner.co.il Some slides stolen from: Jack Tumblin 1 Agenda We ve seen Panorama (from different FOV) Super-resolution (from low-res) HDR (from different

More information

Noise and ISO. CS 178, Spring Marc Levoy Computer Science Department Stanford University

Noise and ISO. CS 178, Spring Marc Levoy Computer Science Department Stanford University Noise and ISO CS 178, Spring 2014 Marc Levoy Computer Science Department Stanford University Outline examples of camera sensor noise don t confuse it with JPEG compression artifacts probability, mean,

More information

High dynamic range imaging and tonemapping

High dynamic range imaging and tonemapping High dynamic range imaging and tonemapping http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2017, Lecture 12 Course announcements Homework 3 is out. - Due

More information

Sensing Increased Image Resolution Using Aperture Masks

Sensing Increased Image Resolution Using Aperture Masks Sensing Increased Image Resolution Using Aperture Masks Ankit Mohan, Xiang Huang, Jack Tumblin Northwestern University Ramesh Raskar MIT Media Lab CVPR 2008 Supplemental Material Contributions Achieve

More information

Computational Photography and Video. Prof. Marc Pollefeys

Computational Photography and Video. Prof. Marc Pollefeys Computational Photography and Video Prof. Marc Pollefeys Today s schedule Introduction of Computational Photography Course facts Syllabus Digital Photography What is computational photography Convergence

More information

Lecture 3: Linear Filters

Lecture 3: Linear Filters Signal Denoising Lecture 3: Linear Filters Math 490 Prof. Todd Wittman The Citadel Suppose we have a noisy 1D signal f(x). For example, it could represent a company's stock price over time. In order to

More information

Problem Session 6. Computa(onal Imaging and Display EE 367 / CS 448I

Problem Session 6. Computa(onal Imaging and Display EE 367 / CS 448I Problem Session 6 Computa(onal Imaging and Display EE 367 / CS 448I Topics Photo- electron shot- noise SNR calcula@ons Deconvolu@on of an image with Poisson noise Wiener deconvolu@on Richardson- Lucy Richardson-

More information

Computational Sensors

Computational Sensors Computational Sensors Suren Jayasuriya Postdoctoral Fellow, The Robotics Institute, Carnegie Mellon University Class Announcements 1) Vote on this poll about project checkpoint date on Piazza: https://piazza.com/class/j6dobp76al46ao?cid=126

More information

Resolving Objects at Higher Resolution from a Single Motion-blurred Image

Resolving Objects at Higher Resolution from a Single Motion-blurred Image MITSUBISHI ELECTRIC RESEARCH LABORATORIES http://www.merl.com Resolving Objects at Higher Resolution from a Single Motion-blurred Image Amit Agrawal, Ramesh Raskar TR2007-036 July 2007 Abstract Motion

More information

Motion Blurred Image Restoration based on Super-resolution Method

Motion Blurred Image Restoration based on Super-resolution Method Motion Blurred Image Restoration based on Super-resolution Method Department of computer science and engineering East China University of Political Science and Law, Shanghai, China yanch93@yahoo.com.cn

More information

Random Coded Sampling for High-Speed HDR Video

Random Coded Sampling for High-Speed HDR Video Random Coded Sampling for High-Speed HDR Video Travis Portz Li Zhang Hongrui Jiang University of Wisconsin Madison http://pages.cs.wisc.edu/~lizhang/projects/hs-hdr/ Abstract We propose a novel method

More information

What will be on the midterm?

What will be on the midterm? What will be on the midterm? CS 178, Spring 2014 Marc Levoy Computer Science Department Stanford University General information 2 Monday, 7-9pm, Cubberly Auditorium (School of Edu) closed book, no notes

More information

CVPR Easter School. Michael S. Brown. School of Computing National University of Singapore

CVPR Easter School. Michael S. Brown. School of Computing National University of Singapore Computational Photography CVPR Easter School March 14 18 18 th, 2011, ANU Kioloa Coastal Campus Michael S. Brown School of Computing National University of Singapore Goal of this tutorial Introduce you

More information

Optical image stabilization (IS)

Optical image stabilization (IS) Optical image stabilization (IS) CS 178, Spring 2011 Marc Levoy Computer Science Department Stanford University Outline! what are the causes of camera shake? how can you avoid it (without having an IS

More information

TRUESENSE SPARSE COLOR FILTER PATTERN OVERVIEW SEPTEMBER 30, 2013 APPLICATION NOTE REVISION 1.0

TRUESENSE SPARSE COLOR FILTER PATTERN OVERVIEW SEPTEMBER 30, 2013 APPLICATION NOTE REVISION 1.0 TRUESENSE SPARSE COLOR FILTER PATTERN OVERVIEW SEPTEMBER 30, 2013 APPLICATION NOTE REVISION 1.0 TABLE OF CONTENTS Overview... 3 Color Filter Patterns... 3 Bayer CFA... 3 Sparse CFA... 3 Image Processing...

More information

The Flutter Shutter Camera Simulator

The Flutter Shutter Camera Simulator 2014/07/01 v0.5 IPOL article class Published in Image Processing On Line on 2012 10 17. Submitted on 2012 00 00, accepted on 2012 00 00. ISSN 2105 1232 c 2012 IPOL & the authors CC BY NC SA This article

More information

Today. Defocus. Deconvolution / inverse filters. MIT 2.71/2.710 Optics 12/12/05 wk15-a-1

Today. Defocus. Deconvolution / inverse filters. MIT 2.71/2.710 Optics 12/12/05 wk15-a-1 Today Defocus Deconvolution / inverse filters MIT.7/.70 Optics //05 wk5-a- MIT.7/.70 Optics //05 wk5-a- Defocus MIT.7/.70 Optics //05 wk5-a-3 0 th Century Fox Focus in classical imaging in-focus defocus

More information

Computational Approaches to Cameras

Computational Approaches to Cameras Computational Approaches to Cameras 11/16/17 Magritte, The False Mirror (1935) Computational Photography Derek Hoiem, University of Illinois Announcements Final project proposal due Monday (see links on

More information

Image Deblurring. This chapter describes how to deblur an image using the toolbox deblurring functions.

Image Deblurring. This chapter describes how to deblur an image using the toolbox deblurring functions. 12 Image Deblurring This chapter describes how to deblur an image using the toolbox deblurring functions. Understanding Deblurring (p. 12-2) Using the Deblurring Functions (p. 12-5) Avoiding Ringing in

More information

Restoration of Blurred Image Using Joint Statistical Modeling in a Space-Transform Domain

Restoration of Blurred Image Using Joint Statistical Modeling in a Space-Transform Domain IOSR Journal of Electronics and Communication Engineering (IOSR-JECE) e-issn: 2278-2834,p- ISSN: 2278-8735.Volume 12, Issue 3, Ver. I (May.-Jun. 2017), PP 62-66 www.iosrjournals.org Restoration of Blurred

More information

A Study of Slanted-Edge MTF Stability and Repeatability

A Study of Slanted-Edge MTF Stability and Repeatability A Study of Slanted-Edge MTF Stability and Repeatability Jackson K.M. Roland Imatest LLC, 2995 Wilderness Place Suite 103, Boulder, CO, USA ABSTRACT The slanted-edge method of measuring the spatial frequency

More information

BlurBurst: Removing Blur Due to Camera Shake using Multiple Images

BlurBurst: Removing Blur Due to Camera Shake using Multiple Images : Removing Blur Due to Camera Shake using Multiple Images ATSUSHI ITO Sony Corporation and ASWIN C. SANKARANARAYANAN Carnegie Mellon University and ASHOK VEERARAGHAVAN and RICHARD G. BARANIUK Rice University

More information

2015, IJARCSSE All Rights Reserved Page 312

2015, IJARCSSE All Rights Reserved Page 312 Volume 5, Issue 11, November 2015 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Shanthini.B

More information

Coding & Signal Processing for Holographic Data Storage. Vijayakumar Bhagavatula

Coding & Signal Processing for Holographic Data Storage. Vijayakumar Bhagavatula Coding & Signal Processing for Holographic Data Storage Vijayakumar Bhagavatula Acknowledgements Venkatesh Vadde Mehmet Keskinoz Sheida Nabavi Lakshmi Ramamoorthy Kevin Curtis, Adrian Hill & Mark Ayres

More information

A Comprehensive Review on Image Restoration Techniques

A Comprehensive Review on Image Restoration Techniques International Journal of Research in Advent Technology, Vol., No.3, March 014 E-ISSN: 31-9637 A Comprehensive Review on Image Restoration Techniques Biswa Ranjan Mohapatra, Ansuman Mishra, Sarat Kumar

More information

Single Digital Image Multi-focusing Using Point to Point Blur Model Based Depth Estimation

Single Digital Image Multi-focusing Using Point to Point Blur Model Based Depth Estimation Single Digital mage Multi-focusing Using Point to Point Blur Model Based Depth Estimation Praveen S S, Aparna P R Abstract The proposed paper focuses on Multi-focusing, a technique that restores all-focused

More information

Get the Shot! Photography + Instagram Workshop September 21, 2013 BlogPodium. Saturday, 21 September, 13

Get the Shot! Photography + Instagram Workshop September 21, 2013 BlogPodium. Saturday, 21 September, 13 Get the Shot! Photography + Instagram Workshop September 21, 2013 BlogPodium Part One: Taking your camera off manual Technical details Common problems and how to fix them Practice Ways to make your photos

More information

International Journal of Advancedd Research in Biology, Ecology, Science and Technology (IJARBEST)

International Journal of Advancedd Research in Biology, Ecology, Science and Technology (IJARBEST) Gaussian Blur Removal in Digital Images A.Elakkiya 1, S.V.Ramyaa 2 PG Scholars, M.E. VLSI Design, SSN College of Engineering, Rajiv Gandhi Salai, Kalavakkam 1,2 Abstract In many imaging systems, the observed

More information