Adaptive Vision Leveraging Digital Retinas: Extracting Meaningful Segments

Size: px

Start display at page:

Download "Adaptive Vision Leveraging Digital Retinas: Extracting Meaningful Segments"

Liliana Davis
5 years ago
Views:

1 Adaptive Vision Leveraging Digital Retinas: Extracting Meaningful Segments Nicolas Burrus and Thierry M Bernard <firstnamelastname@enstafr> September 20, 2006 Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 1/ 15

2 [1/3] Robust vision algorithms generally require: Carefully managing a priori In particular, avoiding quantitative ones (parameters) Deducing them from image properties This motivates the use of statistical frameworks Statistical parameters can be estimated from images Statistical decisions provide objectivity A lot of image features/characteristics deserve statistical processing Heavy computations, difficult in real-time Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 2/ 15

3 [1/3] Robust vision algorithms generally require: Carefully managing a priori In particular, avoiding quantitative ones (parameters) Deducing them from image properties This motivates the use of statistical frameworks Statistical parameters can be estimated from images Statistical decisions provide objectivity A lot of image features/characteristics deserve statistical processing Heavy computations, difficult in real-time Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 2/ 15

4 [2/3] This leads to exploiting more adapted architectures Giving processors a faster and less power-consuming access to memory Allowing fast computation of global quantities Digital retinas seem well-adapted Their massive parallelism allows efficient data processing They usually have highly efficient aggregation operators Fast computation of global or regional quantities Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 3/ 15

5 [2/3] This leads to exploiting more adapted architectures Giving processors a faster and less power-consuming access to memory Allowing fast computation of global quantities Digital retinas seem well-adapted Their massive parallelism allows efficient data processing They usually have highly efficient aggregation operators Fast computation of global or regional quantities Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 3/ 15

6 [3/3] Aim of this work Assessing the combination of statistical processing and digital retinas Implementing a case study: meaningful segment detection Tools used A contrario statistical framework (CMLA, 2000) Events are meaningful if they cannot occur by chance Almost parameterless framework Some image properties can be taken into account Pvlsar34, home-made programmable retina Each pixel embeds a photocell and a boolean processor A global adder can add all pixel values in constant-time Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 4/ 15

7 [3/3] Aim of this work Assessing the combination of statistical processing and digital retinas Implementing a case study: meaningful segment detection Tools used A contrario statistical framework (CMLA, 2000) Events are meaningful if they cannot occur by chance Almost parameterless framework Some image properties can be taken into account Pvlsar34, home-made programmable retina Each pixel embeds a photocell and a boolean processor A global adder can add all pixel values in constant-time Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 4/ 15

Bit planes + Boolean CPU Photocell Light

Commands - 40 bits of memory per pixel -

8 Bit planes + Boolean CPU Photocell Light - SIMD architecture Scalar descriptor (via global adder) or serialized image Cortex : scalar processor Nicolas Burrus Commands - 40 bits of memory per pixel - Boolean local operations, each pixel can access its 4 neighbors - Analog global adder Adaptive Vision Leveraging Digital Retinas 5/ 15

9 Helmholtz principle An event is meaningful if its probability of occurrence in a random environment is very low Number of false alarms (NFA) The number of false alarms NFA(E) of an event E is the expectation of its number of occurrences in a random environment Meaningfulness of an event An event E is meaningful if NFA(E) < 1 Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 6/ 15

10 Helmholtz principle An event is meaningful if its probability of occurrence in a random environment is very low Number of false alarms (NFA) The number of false alarms NFA(E) of an event E is the expectation of its number of occurrences in a random environment Meaningfulness of an event An event E is meaningful if NFA(E) < 1 Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 6/ 15

11 Helmholtz principle An event is meaningful if its probability of occurrence in a random environment is very low Number of false alarms (NFA) The number of false alarms NFA(E) of an event E is the expectation of its number of occurrences in a random environment Meaningfulness of an event An event E is meaningful if NFA(E) < 1 Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 6/ 15

12 Example E: observing a 6-pixel long horizontal segment P(black) = 05 P(black) = 023 P(black) = 01 NFA(E) 63 NFA(E) 05 NFA(E) 10 3 Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 7/ 15

Segment candidate extraction Segment 1 Length = 20 pixels Gradient mean = 30 3 Candidate selection - Could we observe such a

13 Overview Candidate extraction Meaningful candidate selection Overview of our segment extraction algorithm 1 Global statistics computation - Gradient distribution over the whole image - Number of points sharing the same local directions - 2 Segment candidate extraction Segment 1 Length = 20 pixels Gradient mean = 30 3 Candidate selection - Could we observe such a long segment? - Could we observe such a contrasted segment by chance? Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 8/ 15

Iteration 1 l=2 l=3 l=4 l=2 L=3 L=1 l=5 l=6

14 Overview Candidate extraction Meaningful candidate selection Massively parallel segment extraction Local directions extraction 1 image = 1 direction For each direction image Segment thinning Info gathering in extremities Iteration 0 l=1 l=1 Iteration 1 l=2 l=3 l=4 l=2 L=3 L=1 l=5 l=6 L= Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 9/ 15

Overview Candidate extraction Meaningful candidate selection Meaningful segment selection:

white pixel density d { 500 400 300 200 100 0 Compute white pixel density thanks to the global

9 10 11 Lookup the corresponding threshold value in the precomputed table (here threshold=12)

k such as NFA(L > k) < 1 for this density Store the computed value in a lookup table Density 001

15 Overview Candidate extraction Meaningful candidate selection Meaningful segment selection: length-based Online part For each direction image Offline part Generate 1000 random images with white pixel density d { Compute white pixel density thanks to the global adder (here d=009) For each density d Compute the histogram of maximal observed lengths Lookup the corresponding threshold value in the precomputed table (here threshold=12) Threshold 12 This threshold ensures NFA(Segment) < 1 Statistical criterion: choose the threshold k such as NFA(L > k) < 1 for this density Store the computed value in a lookup table Density Length Threshold Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 10/ 15

16 Results Performance Runs at video rate, 10 times faster than a PC Power consumption is a few tens milliwatts Qualitative evaluation Well-known images were transferred into Pvlsar34 Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 11/ 15

17 Results Performance Runs at video rate, 10 times faster than a PC Power consumption is a few tens milliwatts Qualitative evaluation Well-known images were transferred into Pvlsar34 Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 11/ 15

18 Results: "desk" image Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 12/ 15

19 Results: adaptation House Horiz segments Horiz segments ("desk" threshold) (deduced threshold) Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 13/ 15

20 Meaningful segment selection: other criteria Limitations of synchronous retinas More complex -regional- criteria are desirable Example: contrast-based selection Regional computations are very inefficient on synchronous retinas Let µ and σ be the mean and deviation of the image Under a contrario assumptions, the mean of a segment with n pixels follows N(µ, σ n ) We can deduce an analytical NFA Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 14/ 15

Meaningful segment selection: other criteria Limitations of synchronous retinas More complex -regional- criteria are desirable Example: contrast-based selection Regional computations are very

21 Meaningful segment selection: other criteria Limitations of synchronous retinas More complex -regional- criteria are desirable Example: contrast-based selection Regional computations are very inefficient on synchronous retinas Towards asynchronism? Asynchronous retinas can perform regional measures very efficiently We are currently designing one Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 14/ 15

22 Discussion About the statistical framework Image adaptation is too global No free parameter Rational and robust decisions Exploitation of aggregative computation could be more intensive About the retinal implementation We already have a real-time implementation Most segments are successfully detected Asynchronism should enable faster and more complex regional analysis Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 15/ 15

23 Discussion About the statistical framework Image adaptation is too global No free parameter Rational and robust decisions Exploitation of aggregative computation could be more intensive About the retinal implementation We already have a real-time implementation Most segments are successfully detected Asynchronism should enable faster and more complex regional analysis Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 15/ 15

24 Discussion About the statistical framework Image adaptation is too global No free parameter Rational and robust decisions Exploitation of aggregative computation could be more intensive About the retinal implementation We already have a real-time implementation Most segments are successfully detected Asynchronism should enable faster and more complex regional analysis Nicolas Burrus Adaptive Vision Leveraging Digital Retinas 15/ 15

Detection of Compound Structures in Very High Spatial Resolution Images

Detection of Compound Structures in Very High Spatial Resolution Images Selim Aksoy Department of Computer Engineering Bilkent University Bilkent, 06800, Ankara, Turkey saksoy@cs.bilkent.edu.tr Joint work