Sensing Increased Image Resolution Using Aperture Masks Ankit Mohan, Xiang Huang, Jack Tumblin Northwestern University Ramesh Raskar MIT Media Lab CVPR 2008 Supplemental Material
Contributions Achieve sub-pixel image shift using a mask in front of the lens Enhance effective sensor resolution without moving the camera or sensor.
We intentionally blur the image so that when the aperture is open, the blur is less than one pixel, p L 0 Focus Plane B 1 A B B 0 p B A A 1 Scene Plane L 1 Open Aperture/Lens A 0 Sensor
Moving a pinhole in along the lens effectively moves the image in an out-of-focus sensor plane. L 0 Focus Plane B 1 B p A B B 0 L p p B A A 1 A p Scene Plane L 1 Aperture/Lens A 0 Sensor
Moving a pinhole in along the lens is same moving the sensor by subpixel distances. L 0 Focus Plane B 1 B p A B B 0 L p p B A A 1 A p Scene Plane L 1 Aperture/Lens A 0 Sensor
Moving a pinhole in along the lens is same moving the sensor by subpixel distances. Focus Plane L 0 B 1 B p A B B 0 p B L p A A 1 A p Scene Plane L 1 Aperture/Lens A 0 Sensor
Moving the pinhole aperture with a slightly out of focus sensor is equivalent to translation based superresolution But, aperture movement is in mm instead of µm
Pin holes are inefficient, collect little light, thus increasing exposure time. Instead, we use wider carefully chosen apertures. Use instead of
Unique finite sized aperture positions l r P p P a
Unique finite sized aperture positions P a p P l g
Unique finite sized aperture positions P a p P l b
We capture multiple photos with out-of-focus sensor and unique finite sized aperture positions Focus Plane l r P a p l g P l b Aperture/Lens Sensor
3x resolution enhancement: Capture 3 photos with aperture position l r, l g, and l b Focus Plane l r P a p l g P l b Aperture/Lens Sensor
Total blur size = one pixel size ( p ) Blur due to each partial aperture = a = p /3 Focus Plane l r P a p l g P l b Aperture/Lens Sensor
2x resolution enhancement for a 1D signal Scene, s
Capture 2 photos with complimentary apertures Scene, s l 0 l 1 s * l 0 s * l 1 Notice the phase shift between the two signals. For a total blur of one pixel, this corresponds to half pixel shift.
Anti-aliasing due to finite pixel size s * l 0 s * l 0 * p p l 0 Scene, s p l 1 s * l 1 s * l 1 * p
Discrete sampling due to pixels s * l 0 s * l 0 *p f 0 [x] p Sampling l 0 Scene, s p Sampling l 1 s * l 1 s * l 1 * p f 1 [x] Samples captured by the two photos are different.
Interleave samples from the two photos s * l 0 s * l 0 *p f 0 [x] p Sampling Interleaved Samples l 0 Scene, s p Sampling l 1 s * l 1 s * l 1 *p f 1 [x]
Deblur the effect of p and l s * l 0 s * l 0 *p f 0 [x] p Sampling Interleaved Samples l 0 Scene, s p Sampling l 1 s * l 1 s * l 1 *p f 1 [x] Deconvolution 2x enhanced resolution
Image Shifting without Moving Parts Suggested Design: Programmable Aperture with NO moving parts eliminating expensive precision or cumbersome registration P l r l g l b Camera Aperture Plane Programmable LCD Aperture Focus Plane P a Sensor p Filter Holders Focus Plane l r P Our Implementation: Masks in a Holder P l g l b Removable Mechanical Mask Sensor
Prototype using a conventional SLR camera Cokin filter holder Slide mask in front of the lens
Aperture Masks
Result: Radial spoke chart Mask size=12mm Mask resolution=3x3 Image scale factor=1/1.7 Input image size=471x741 Output image size=1413x1413
Input images (3x3)
Cropped and bicubic interpolated input images (4 of 9 shown) Cropped result with 3x samples Observe the jaggies in the input images. In the result, details in high spatial frequencies closer to center of the spoke are maintained upto a limit.
Result: Barcode Mask size=12mm Mask resolution=4x1 Image scale factor=1/3 Input image size=171x416 Output image size=684x416
Input Images (4x1) Result: 4x increase in horizontal resolution
Result: Sheets of paper Mask size=12mm Mask resolution=4x1 Image scale factor=1/8 Input image size=100x300 Output image size=400x300
Input images (4x1)
2 of the 4 input images Result: 4x increase in horizontal resolution
Result: Carpet tile Mask size=12mm Mask resolution=2x2 Image scale factor=1/2 Input image size=256x256 Output image size=512x512
Input images (2x2)
Result Please blink-compare with next page
One of the Inputs Please blink-compare with previous page