Structured-Light Based Acquisition (Part 1)

Transcription

1 Structured-Light Based Acquisition (Part 1) CS635 Spring 2017 Daniel G. Aliaga Department of Computer Science Purdue University

2 Passive vs. Active Acquisition Passive + Just take pictures + Does not intrude in the environment (=passive) Some surfaces cannot be acquired Robustness is problematic Active + Emit light into the scene so as to force the generation of robust correspondence Environment is intruded (=active)

3 Active Acquisition Some options: Laser scanning Structured Light

4 Laser Scanning

5 Light Stripe Scanning (Single Stripe) Light plane Source Camera Surface Optical triangulation Project a single stripe of laser light Scan it across the surface of the object This is a very precise version of structured light scanning Good for high resolution 3D, but needs many images and takes time

6 Stripe Triangulation Object Light Plane Ax By Cz D 0 Laser Camera Project laser stripe onto object

7 Stripe Triangulation Object Light Plane Ax By Cz D 0 Laser Image Point ( x', y') Camera Depth from ray-plane triangulation: Intersect camera ray with light plane x y x' z / y' z / f f z Df Ax' By' Cf

8 Example: Laser scanner + very accurate < 0.01 mm more than 10sec per scan Cyberware face and head scanner

9 Example: Laser scanner Digital Michelangelo Project

10 Example: Laser scanner Portable scanner by Minolta

11 Digital Projector Structured Light Goal: generate correspondences so as to enable a robust 3D reconstruction

12 Digital Projector Structured Light Method: Use the projector as a pattern generator Have the camera see the pattern and generate 1 or more corresponded points

13 Digital Projector Structured Light What are possible patterns? Spatial patterns Temporal patterns Color patterns And combinations of the above

14 Digital Projector Structured Light Lets focus on binary striped patterns

15 Binary Pattern Structured Light 15

16 Binary Coding Assign each pixel a unique illumination code over time [Posdamer 82] Time X coordinate

17 Binary Coding Assign each pixel a unique illumination code over time [Posdamer 82] Time Y coordinate

18 Binary Coding n 2 1 stripes in n images Projected over time Pattern 3 Example: 3 binary-encoded patterns which allows the measuring surface to be divided in 8 sub-regions Pattern 2 Pattern 1

19 Binary vs Gray Codes Decimal Binary Gray Code

20 Binary vs Gray Codes Pattern 3 Pattern 2 Pattern 1 Binary code Gray code

21 Standard Pixel Classification ON(1) OFF(0) 21 Uncertain

22 Pixel Classification Challenges? Illuminated (ON) Non-illuminated (OFF)?

23 Standard Pixel Classification Interval P off P on Common methods Simple threshold Albedo threshold Dual pattern 23

24 1. Simple Threshold User specifies one threshold t for all pixels p is ON P off t P on q is OFF

25 1. Simple Threshold User specifies two thresholds t 1, t 2 p is ON P off t 1 t 2 P on q is OFF

26 2. Albedo Threshold Compute the albedo t p for each pixel P off t p p is ON P on Albedo image

27 2. Albedo Threshold Compute the albedo t p for each pixel p is OFF P off t p P on Albedo image

28 3. Dual Pattern: Pattern and Inverse Without explicitly computing t p P off p is ON p is OFF P on Inverse pattern

29 Limitations of Standard Methods All three methods assume the two intervals do not overlap This is incorrect when there is strong indirect (global) light Haven t actually established the correct bounds

30 Example Reconstruction (lost samples are due to missing and incorrect classifications)

31 Key Observations We can estimate tight intensity value bounds for when the pixel is ON and for when it is OFF. A pixel is classifiable when its intensity value falls into one interval but not in the other. [Xu07, Xu09]

32 Example Comparison Point splatting of the reconstructed living room scene Using Using standard our pixel pixel classification

33 Pixel Intensity (p is ON) p p is directly illuminated p = d + i on Projector (direct) (indirect)

34 Pixel Intensity (p is OFF) p p is not directly illuminated p = i off Projector (indirect)

35 Pixel Intensity p = { d + i on i off If p is ON If p is OFF Chicken and egg problem is Need to know d, i on, i off to classify a pixel. Need to classify a pixel to know d, i on, i off ALL white pattern (all projector pixels on)

36 Direct and Indirect Separation Direct and indirect (global) components of each pixel under ALL white pattern can be separated easily (Nayar et al. SIGGRAPH 06). p = d + i total total

37 Direct and Indirect Separation Project high frequency binary pattern and its inverse to separate light components. Structured light patterns include the separation patterns. Thus, separation can be applied to previously captured data to obtain per pixel p = d total + i total

38 Pixel Classification Scenarios d total >i total 0 P off i total d total P on d total + i total 0 OFF ON 255

39 Pixel Classification Scenarios d total 0 0 d total P off P on i total d total + i total Uncertain

40 Pixel Classification Scenarios d total i total 0 P off i total d total P on d total+ i total 0 OFF Uncertain ON 255

41 Single Pattern Classification Rules d total < m pixel is uncertain p < min(d total, i total ) pixel is off p > max(d total, i total ) pixel is on otherwise pixel is uncertain

42 Classification Results ON(1) OFF(0) Uncertain Hand-painted ground truth Standard pixel classification Our pixel classification

43 Classification Results ON(1) OFF(0) Uncertain Hand-painted ground truth Standard pixel classification Our pixel classification

44 Increased Reconstructed Points Point splatting of the reconstructed living room scene Using standard pixel classification Using our pixel classification

45 Increased Reconstructed Points Zoom-in view of the vase Using standard pixel classification Using our pixel classification

46 Increased Reconstructed Points Zoom-in view of the giraffe Using standard pixel classification Using our pixel classification

47 Increased Reconstructed Points Point splatting of the reconstructed wooden objects Using standard pixel classification Using our pixel classification

48 Increased Reconstructed Points Zoom-in view of the reconstructed points in white Using standard pixel classification Using our pixel classification

49 Increased Reconstructed Points Point splatting of the reconstructed shiny objects Bigger splat size Smaller splat size Using standard pixel classification Using our pixel classification

50 Another issue

51 Another issue Classify pixels at stripe boundaries? pixels at strip middle? all pixels?