Multi-aperture camera module with 720presolution using microoptics A. Brückner, A. Oberdörster, J. Dunkel, A. Reimann, F. Wippermann, A. Bräuer Fraunhofer Institute for Applied Optics and Precision Engineering IOF, Jena, Germany
Agenda Motivation Features of electronic cluster eyes Novel microlens technology Prototyping results Summary
Motivation Applications of miniaturized imaging systems information tech. life science consumer electronics automotive requirements: small z-height low mass low cost medical imaging sensor systems machine vision security ~2 billion devices shipped in 2013* *Yole (2013) 3
Motivation Scaling limits of state-of-the-art imaging systems digital imaging + advances in decrease sensor & pixel size semiconductor manufacturing (currently 0.9µm [1] ) compact devices, slim package low cost sufficient image quality reduction of signal-to-noise ratio min. pixel size 1.3µm (SNR=32, indoor daytime, 30 fps [2] ) N p : number of photons d: pixel size [1] R. Fontaine, International Image Sensor Workshop (IISW), 2011. image source: www.clarkvision.com, March 2011 [2] M. Schöberl et. al. J. Electron. Imaging, 21 (2), 2012. 4
Motivation Scaling limits of state-of-the-art imaging systems reduction of pixel size + constant f-number loss of resolution image point ~ sum of Gaussian moments [1] : diffraction aberrations f-number: minimum pixel size according to optical resolution limit no way to further shrink size of camera module [1] A. Lohmann, Appl. Opt. 28, 4996-4998 (1989). [2] A. Brückner et. al., Proc. SPIE 8616, pp. 861617, 2013. 5
Motivation Natural & artificial imaging principles state-of-the-art cameras miniaturization limited (noise, diffraction, technology) K K further miniaturization 6
Agenda Motivation Features of electronic cluster eyes Novel microlens technology Prototyping results Summary
Features of electronic cluster eyes (ecley) array of miniature cameras each transmits partial image of a different part of FOV image processing to reconstruct final image from image segments electronic image stitching 50% optics z-height easier fabrication & assembly (wafer-level optics tech.) [1] A. Brückner et.al., Opt. Express 18 (24), 24379-24394 (2010). granted patents many others pending: 2009: US20110228142, JP5379241, KR101284414 2010: US20110098208, US8717485 Schematic working principle [1] FOV : Field of view 8
Features of electronic cluster eyes (ecley) Demonstrator ecley VGA d = 10 mm d = 25 mm Size comparison: VGA webcam lens vs. ecley VGA demonstrator d = 50 mm d = 250 mm focusing = correction for parallax software focusing / refocusing measured depth map multiaperture parallax -> disparity Object distance: 4m, 70 diagonal FOV DoF : Depth of field (proportional to depth of focus) 1.4mm optics z-height large DoF (4mm - ) depth imaging A. Oberdörster et. al., "Digital focusing and re-focusing with thin multi-aperture cameras", in Proc. SPIE 8299, p. 829907 (2012). 9
Novel microlens technology opt. design recent prototypes were based on reflow of photo resist limited fill-factor, f/# and resolution standard micro lens array chirped array of ellipsoidal lenses enhance resolution need more degrees of freedom increase optical fill-factor new microlens technology: freeform microlens segment array 10
Novel microlens technology mastering Ultra precision diamond machining + Step&Repeat imprinting milling & planing generation of masters for single microlens arrays Step&Repeat process generation of array-masters for many MLAs on common substrate 11
Novel microlens technology Fabrication process chain Data compilation & transfer Single master Imprint tool Array master Optics modul surface description alignment marks yard structures UP-machining Aluminium / NiP UV transparent polymer / epoxybased Step & Repeat multiple imprint replication Mask Aligner replication & stacking Charakterisierung tactile und interferometrical characterization pre-compensation J. Dunkel, et. al.,proc. SPIE 8763, p. 876330 (2013). F. Wippermann, et. al., 18th Microoptics Conference (MOC 13), (2013). 12
Formabweichung [nm] (rms) Fraunhofer IOF Novel microlens technology Step & Repeat replication process measured master properties surface roughness ~ 8nm (RMS) 100 90 80 70 60 50 40 30 20 10 0 8 master wafer with refractive freeform lenslet arrays 001-002-MS-005_chan00 001-002-MS-005_chan01 001-002-MS-005_chan02 001-002-MS-005_chan03 001-002-MS-005_chan04 001-002-MS-005_chan10 001-002-MS-005_chan11 001-002-MS-005_chan12 001-002-MS-005_chan13 001-002-MS-005_chan14 001-002-MS-005_chan20 001-002-MS-005_chan21 001-002-MS-005_chan22 001-002-MS-005_chan23 001-002-MS-005_chan24 001-002-MS-005_chan30 001-002-MS-005_chan31 001-002-MS-005_chan32 001-002-MS-005_chan33 001-002-MS-005_chan34 001-002-MS-005_chan40 001-002-MS-005_chan41 001-002-MS-005_chan42 001-002-MS-005_chan43 001-002-MS-005_chan44 001-002-MS-005_chan50 001-002-MS-005_chan51 001-002-MS-005_chan52 001-002-MS-005_chan53 001-002-MS-005_chan54 001-002-MS-005_chan60 001-002-MS-005_chan61 001-002-MS-005_chan62 001-002-MS-005_chan63 001-002-MS-005_chan64 001-002-MS-005_chan70 001-002-MS-005_chan71 001-002-MS-005_chan72 001-002-MS-005_chan73 av. shape deviation ~ 52nm (RMS) 001-002-MS-005_chan74 J. Dunkel, et. al.,proc. SPIE 8763, p. 876330 (2013). F. Wippermann, et. al., 18th Microoptics Conference (MOC 13), (2013). 13
Novel microlens technology Step & Repeat replication process 8 master wafer with refractive freeform lenslet arrays double-sided replicated multi-aperture objective J. Dunkel, et. al.,proc. SPIE 8763, p. 876330 (2013). F. Wippermann, et. al., 18th Microoptics Conference (MOC 13), (2013). 14
Prototyping results Demonstrator ecley 720p customized image sensor* multi-aperture lens module camera module layout camera assembly 1.9mm optics z-height * developed by partner Fraunhofer IIS, Erlangen, fabricated by TowerJazz 15
Prototyping results Demonstrator ecley 720p on-axis MTF measurement preliminary camera demonstrator cutoff frequency: 300 cycles/mm Parameter diag. FOV Fno. image res. in pixels Value 70 2.8 1344 x 768 TTL no. of cells no. of pixels per cell FOV per cell pixel pitch sensor die size 1.9mm 15 x 9 96 x 96 9.2 2.2µm 6.4mm x 4.15mm fill fact or* 31% A. Brückner, et. al., "Ultra-thin wafer-level camera with 720p resolution using microoptics," Proc. SPIE 9193, (2014). 16
Prototyping results Characterization through focus MTF measurement scanning 100µm defocus with 5µm steps MTF@Ny/2 > 30% over 40µm defocus off-axis MTF measurement intermediate field at 18 AOI MTF is in good agreement with simulation AOI : Angle of incidence 17
Prototyping results Image reconstruction Real time image reconstruction: Image acquisition (raw data) Flat-Field Correction Distortion correction Raw image (channel-wise) Pixel redistribution (Braiding pattern & parallax correction) braiding coded into optical module by design algorithms of low complexity frame rate: 30 fps possible in mobile devices Reconstructed image A. Oberdörster, et. al., "Interactive alignment and image reconstruction for waferlevel multi-aperture camera systems," Proc. SPIE 9217, (2014). 18
Prototyping results Image reconstruction Raw image Reconstructed image ongoing work: optimization of image reconstruction (correction of sensor pixel characteristics) small batch series applying precompensation integration with lens housing A. Oberdörster, et. al., "Interactive alignment and image reconstruction for waferlevel multi-aperture camera systems," Proc. SPIE 9217, (2014). 19
Agenda Motivation Features of electronic cluster eyes Novel microlens technology Prototyping results Summary
Summary multi-aperture enables new freedom for miniaturized imaging systems design by fusion of micro-optics micro-electronics image processing extreme miniaturization segmentation of field-of-view reduction of costs parallelized fabrication and partly assembly on wafer level unique features for slimmer consumer products 21
Thank you. The presented work has been funded by: in a project of the Fraunhofer Future Foundation www.facetvision.de