Sensing Increased Image Resolution Using Aperture Masks

Size: px

Start display at page:

Download "Sensing Increased Image Resolution Using Aperture Masks"

Loreen Julie Chase
5 years ago
Views:

1 Sensing Increased Image Resolution Using Aperture Masks Ankit Mohan, Xiang Huang, Jack Tumblin Northwestern University Ramesh Raskar MIT Media Lab CVPR 2008 Supplemental Material

2 Contributions Achieve sub-pixel image shift using a mask in front of the lens Enhance effective sensor resolution without moving the camera or sensor.

3 We intentionally blur the image so that when the aperture is open, the blur is less than one pixel, p L 0 Focus Plane B 1 A B B 0 p B A A 1 Scene Plane L 1 Open Aperture/Lens A 0 Sensor

4 Moving a pinhole in along the lens effectively moves the image in an out-of-focus sensor plane. L 0 Focus Plane B 1 B p A B B 0 L p p B A A 1 A p Scene Plane L 1 Aperture/Lens A 0 Sensor

5 Moving a pinhole in along the lens is same moving the sensor by subpixel distances. L 0 Focus Plane B 1 B p A B B 0 L p p B A A 1 A p Scene Plane L 1 Aperture/Lens A 0 Sensor

6 Moving a pinhole in along the lens is same moving the sensor by subpixel distances. Focus Plane L 0 B 1 B p A B B 0 p B L p A A 1 A p Scene Plane L 1 Aperture/Lens A 0 Sensor

7 Moving the pinhole aperture with a slightly out of focus sensor is equivalent to translation based superresolution But, aperture movement is in mm instead of µm

8 Pin holes are inefficient, collect little light, thus increasing exposure time. Instead, we use wider carefully chosen apertures. Use instead of

9 Unique finite sized aperture positions l r P p P a

10 Unique finite sized aperture positions P a p P l g

11 Unique finite sized aperture positions P a p P l b

12 We capture multiple photos with out-of-focus sensor and unique finite sized aperture positions Focus Plane l r P a p l g P l b Aperture/Lens Sensor

13 3x resolution enhancement: Capture 3 photos with aperture position l r, l g, and l b Focus Plane l r P a p l g P l b Aperture/Lens Sensor

14 Total blur size = one pixel size ( p ) Blur due to each partial aperture = a = p /3 Focus Plane l r P a p l g P l b Aperture/Lens Sensor

15 2x resolution enhancement for a 1D signal Scene, s

16 Capture 2 photos with complimentary apertures Scene, s l 0 l 1 s * l 0 s * l 1 Notice the phase shift between the two signals. For a total blur of one pixel, this corresponds to half pixel shift.

17 Anti-aliasing due to finite pixel size s * l 0 s * l 0 * p p l 0 Scene, s p l 1 s * l 1 s * l 1 * p

18 Discrete sampling due to pixels s * l 0 s * l 0 *p f 0 [x] p Sampling l 0 Scene, s p Sampling l 1 s * l 1 s * l 1 * p f 1 [x] Samples captured by the two photos are different.

19 Interleave samples from the two photos s * l 0 s * l 0 *p f 0 [x] p Sampling Interleaved Samples l 0 Scene, s p Sampling l 1 s * l 1 s * l 1 *p f 1 [x]

20 Deblur the effect of p and l s * l 0 s * l 0 *p f 0 [x] p Sampling Interleaved Samples l 0 Scene, s p Sampling l 1 s * l 1 s * l 1 *p f 1 [x] Deconvolution 2x enhanced resolution

21 Image Shifting without Moving Parts Suggested Design: Programmable Aperture with NO moving parts eliminating expensive precision or cumbersome registration P l r l g l b Camera Aperture Plane Programmable LCD Aperture Focus Plane P a Sensor p Filter Holders Focus Plane l r P Our Implementation: Masks in a Holder P l g l b Removable Mechanical Mask Sensor

22 Prototype using a conventional SLR camera Cokin filter holder Slide mask in front of the lens

23 Aperture Masks

24 Result: Radial spoke chart Mask size=12mm Mask resolution=3x3 Image scale factor=1/1.7 Input image size=471x741 Output image size=1413x1413

25 Input images (3x3)

26 Cropped and bicubic interpolated input images (4 of 9 shown) Cropped result with 3x samples Observe the jaggies in the input images. In the result, details in high spatial frequencies closer to center of the spoke are maintained upto a limit.

27 Result: Barcode Mask size=12mm Mask resolution=4x1 Image scale factor=1/3 Input image size=171x416 Output image size=684x416

28 Input Images (4x1) Result: 4x increase in horizontal resolution

29 Result: Sheets of paper Mask size=12mm Mask resolution=4x1 Image scale factor=1/8 Input image size=100x300 Output image size=400x300

30 Input images (4x1)

31 2 of the 4 input images Result: 4x increase in horizontal resolution

32 Result: Carpet tile Mask size=12mm Mask resolution=2x2 Image scale factor=1/2 Input image size=256x256 Output image size=512x512

33 Input images (2x2)

34 Result Please blink-compare with next page

35 One of the Inputs Please blink-compare with previous page

Sensing Increased Image Resolution Using Aperture Masks

Sensing Increased Image Resolution Using Aperture Masks Ankit Mohan, Xiang Huang, Jack Tumblin EECS Department, Northwestern University http://www.cs.northwestern.edu/ amohan Ramesh Raskar Mitsubishi Electric