What the spider s eyes don t tell the spider s brain

Size: px

Start display at page:

Download "What the spider s eyes don t tell the spider s brain"

Chastity Bradford
5 years ago
Views:

1 What the spider s eyes don t tell the spider s brain

2 Depth Perception from Image Defocus in a Jumping Spider (*) Depth Perception from Image Defocus in a Jumping Spider Nagata, Koyanagi, Tsukamoto, Saeki, Isono, Shichida, Tolunaga, Kinoshita, Arikawa, and Terakita.

3 How to judge distance to prey? World is 3-D images are 2-D Can t determine distance monocularly Scale factor ambiguity

4 How to judge distance to prey? World is 3-D images are 2-D Can t determine distance monocularly Scale factor ambiguity Many depth cues Ratio image size to object size Ratio image motion to object motion...

5 How to judge distance to prey? World is 3-D images are 2-D Can t determine distance monocularly Scale factor ambiguity Many depth cues Ratio image size to object size Ratio image motion to object motion... Lens accomodation

6 How to judge distance to prey? World is 3-D images are 2-D Can t determine distance monocularly Scale factor ambiguity Many depth cues Ratio image size to object size Ratio image motion to object motion... Lens accomodation Binocular stereo

7 How to judge distance to prey? World is 3-D images are 2-D Can t determine distance monocularly Scale factor ambiguity Many depth cues Ratio image size to object size Ratio image motion to object motion... Lens accomodation Binocular stereo Defocus blur

8 Accomodation? (1/f = 1/a + 1/b)

9 Accomodation? (1/f = 1/a + 1/b)

10 Accomodation? (1/f = 1/a + 1/b)

11 Binocular stereo? (*) Jumping Spider Vision, David Hill, Wikipedia

12 Defocus blurring?

13 Defocus blurring? R PSF

14 Defocus blurring? 2J 1 (Rρ)/(Rρ) R PSF / R MTF

15 Multi-layer retina (*) Depth Perception from Image Defocus in a Jumping Spider Nagata, Koyanagi, Tsukamoto, Saeki, Isono, Shichida, Tolunaga, Kinoshita, Arikawa, and Terakita.

16 Depth from two image planes (*) Depth Perception from Image Defocus in a Jumping Spider Nagata, Koyanagi, Tsukamoto, Saeki, Isono, Shichida, Tolunaga, Kinoshita, Arikawa, and Terakita.

17 What s wrong with that model?

18 What is wrong with that model? Assumes image on back layer (L1) is always in focus But this would require accomodation; then there is no need for anything else!

19 What is wrong with that model? Assumes image on back layer (L1) is always in focus But this would require accomodation; then there is no need for anything else! Assumes blur on front layer (L2) depends on distance If back is in focus then the blur in front is fixed; blur in front merely reflects inter image layer spacing!

20 What is wrong with that model? Assumes image on back layer (L1) is always in focus But this would require accomodation; then there is no need for anything else! Assumes blur on front layer (L2) depends on distance If back is in focus then the blur in front is fixed; blur in front merely reflects inter image layer spacing! Assumes amount of blur can be ascertained from image Problem is ill posed; for example: Blurry image of sharp texture same as sharp image of blurry texture!

21 Some possible approaches Transport of Intensity Equation (TIE) ( xy I(x,y,z) ) xyφ(x,y,z) = I(x,y,z) k z

22 Some possible approaches Transport of Intensity Equation (TIE) ( xy I(x,y,z) ) xyφ(x,y,z) = I(x,y,z) k z Light-field propagation

23 Some possible approaches Transport of Intensity Equation (TIE) ( xy I(x,y,z) ) xyφ(x,y,z) = I(x,y,z) k z Light-field propagation Deconvolution...

24 System model b1(x,y) E1(x,y) E(x,y) b2(x,y) E2(x,y)

25 Solution based on this b1(x,y) E1(x,y) b2(x,y) E(x,y) b2(x,y) E2(x,y) b1(x,y)

26 Solution based on this b1(x,y) E1(x,y) b2(x,y) E(x,y) b1 b2 = b2 b1 b2(x,y) E2(x,y) b1(x,y)

27 Doing it in parallel E(x,y) b(z) E1(x,y) b(z+d) E2(x,y) b(1+d) b(1) mag b(2+d) b(2) mag b(3+d) b(3) arg min mag b(4+d) b(4) mag

28 0.00 mm

29 0.15 mm

30 0.30 mm

31 0.45 mm

32 0.60 mm

33 0.75 mm

34 0.90 mm

35 1.05 mm

36 1.20 mm

37 Recovery of in-focus distance

38 Recovering the in focus image Ill-posed problem from single defocused image: P 1 (u) = P(u)M 1 (u) can t recover frequency components where M 1 (u) = 0.

39 Recovering the in focus image Ill-posed problem from single defocused image: P 1 (u) = P(u)M 1 (u) can t recover frequency components where M 1 (u) = 0. But with two images defocused to different degrees: P 2 (u) = P(u)M 2 (u) P 1 (u)m1 (u) + P 2(u)M2 (u) = P(u)( M 1 (u) 2 + M 2 (u) 2) works as long as, for any u, either M 1 (u) 0 or M 2 (u) 0. (actually, use Wiener filtering)

40 What the spider s eyes don t tell the spider s brain

41 Pillbox convolved with pillbox is not a pillbox

42 Calibration of lens motion f (mm) steps

43 Lens motion from estimates of zeros in DFT mm frame

Coded Aperture for Projector and Camera for Robust 3D measurement

Coded Aperture for Projector and Camera for Robust 3D measurement Yuuki Horita Yuuki Matugano Hiroki Morinaga Hiroshi Kawasaki Satoshi Ono Makoto Kimura Yasuo Takane Abstract General active 3D measurement