Depth estimation using light fields and photometric stereo with a multi-line-scan framework

Size: px

Start display at page:

Download "Depth estimation using light fields and photometric stereo with a multi-line-scan framework"

Gabriel Harrison
6 years ago
Views:

Depth estimation using light fields and photometric stereo with a multi-line-scan framework Doris Antensteiner, Svorad Štolc, Reinhold Huber-Mörk doris.

1 Depth estimation using light fields and photometric stereo with a multi-line-scan framework Doris Antensteiner, Svorad Štolc, Reinhold Huber-Mörk doris.antensteiner.fl@ait.ac.at High-Performance Image Processing Digital Safety and Security Department AIT Austrian Institute of Technology GmbH, Austria

2 Content Motivation Light fields Photometric stereo Combination of both Quantitative results using synthetic data Experimental results using AIT multi-line-scan system Conclusions OAGM & ARW, Wels, AT, May

3 Synthesis of Light fields & Photometric Stereo Combine advantages of light fields and photometric stereo by a systematic combination Fine depth measurements Good absolute depth measurements 3D reconstruction of homogeneous surfaces 3D reconstruction of highly reflective surfaces OAGM & ARW, Wels, AT, May

4 Light Field Cameras AIT multi-line-scan system Industrial acquisition setup OAGM & ARW, Wels, AT, May

5 Light Field Cameras AIT multi-line-scan system Lytro Lytro Illum Xapt Eye-sect XA Pelican PiCam OAGM & ARW, Wels, AT, May

6 Light-Field Concepts Varying viewing perspectives Plenoptic camera Matrix camera Multi-line-scan setup Multi-line-scan sensor Inspected object e.g. Lytro e.g. Xapt AIT OAGM & ARW, Wels, AT, May

Cons Good absolute depth accuracy Poor relative depth

7 Light-field Depth Estimation n α x, y = arg min α i=1 I i x α, y α I 0 x, y Skewed stack Reference frame Pros / Cons Good absolute depth accuracy Poor relative depth accuracy EPI [2] OAGM & ARW, Wels, AT, May

8 Photometric Stereo Concepts Varying illumination Light dome Multi-line-scan setup OAGM & ARW, Wels, AT, May

Photometry Depth Estimation I = ρ L N Varying

intensity vector ρ albedo L illumination vector N normal

Poor absolute depth accuracy Implemented with Frankot

9 Photometry Depth Estimation I = ρ L N Varying illumination directions Fixed viewing direction I pixel intensity vector ρ albedo L illumination vector N normal unit vector Pros / Cons Good relative depth accuracy Poor absolute depth accuracy Implemented with Frankot and Chelappa [3] Photometric depth OAGM & ARW, Wels, AT, May

10 Synthesis of Light fields & Photometric Stereo OAGM & ARW, Wels, AT, May

Combination of Light fields & Photometric Stereo D = λ lf D lf f lo u, v + λ ps D ps f hi u, v LF only LF+PS GT D final depth D lf light-field depth D ps photometric depth f lo bilateral smoothing

11 Combination of Light fields & Photometric Stereo D = λ lf D lf f lo u, v + λ ps D ps f hi u, v LF only LF+PS GT D final depth D lf light-field depth D ps photometric depth f lo bilateral smoothing filter f hi high-pass image filter λ lf, λ ps weight factors Pros / Cons Preserve coarse depth information (low frequencies) from light-field depth estimations and fine details (high frequencies) from photometric stereo depth estimations OAGM & ARW, Wels, AT, May

12 Quantitative Results Using Synthetic Data Texture LF only LF+PS MSE of disparity w.r.t. GT Light field only: Light field + Photometric stereo: OAGM & ARW, Wels, AT, May

13 AIT Multi-Line-Scan System [1] Each sensor line observes the conveyor belt in a different viewing angle During acquisition, object moves under the sensor OAGM & ARW, Wels, AT, May

14 AIT Multi-Line-Scan System OAGM & ARW, Wels, AT, May

15 Experimental Results Using AIT Multi-Line-Scan System Texture LF only LF+PS OAGM & ARW, Wels, AT, May

16 Experimental Results Using AIT Multi-Line-Scan System Texture LF only LF+PS OAGM & ARW, Wels, AT, May

17 Conclusions Light fields (pros / cons) Good absolute depth accuracy Poor relative depth accuracy Experimental results Synthetic rendered data Multi-line-scan setup coins Photometry (pros / cons) Absolute depth offset Good relative depth accuracy Quantitative results Significant improvement of accuracy Combination Improved depth map Fine surface structures Good absolute depth accuracy In-line and real-time with multi-linescan setup Future Work Further quantitative evaluation Combination through energy minimization OAGM & ARW, Wels, AT, May

18 References [1] S. Štolc, D. Soukup, B. Holländer, and R. Huber-Mörk. Depth and all-infocus imaging by a multi-line-scan light-field camera. J. of Electronic Imaging, 23(5):053020, [2] R. C. Bolles, H. H. Baker, and D. H. Marimont. Epipolarplane image analysis: an approach to determining structure from motion. Int. J. Comp. Vis., 1(1):7 55, [3] R. T. Frankot and R. Chellappa. A Method for enforcing integrability in shape from shading algorithms. IEEE Trans. Pat. Anal. and Mach. Intell., 10: , OAGM & ARW, Wels, AT, May

19 AIT Austrian Institute of Technology your ingenious partner Doris Antensteiner, Svorad Štolc, Reinhold Huber-Mörk High-Performance Image Processing Digital Safety and Security Department AIT Austrian Institute of Technology GmbH, Austria

Hochperformante Inline-3D-Messung

Hochperformante Inline-3D-Messung mittels Lichtfeld Dipl.-Ing. Dorothea Heiss Deputy Head of Business Unit High Performance Image Processing Digital Safety & Security Department AIT Austrian Institute