Intelligent Sensor systems

Size: px

Start display at page:

Download "Intelligent Sensor systems"

Audrey Owens
6 years ago
Views:

1 Intelligent Sensor systems Kiril Alexiev 25A, Acad. G. Bonchev Str., Sofia 1113, Bulgaria Brazil 21 1

2 Intelligent surveillance system (our old definition) The system solves different tasks automatically; System is adaptive to the changes; System allows fast reorganization and tuning; System doesn t require permanent human monitoring and control. Brazil 21 2

3 Open problems to be solved 1. Unique feature selection. 2. 3D scene reconstruction. 3. Image registration. 4. Extended object tracking. 5. Dynamic camera calibration. 6. Optimal sensor control for multi-target tracking. 7. Face localization in human silhouette. 8. Image and multimedia data bases. 9. Face recognition. 1.Abnormal behavior detection. 11.Optimal resource allocation. 12.Object selection. Brazil 21 3

4 Open problems (after 3 years) OK Near to OK Some results Some results Negative res. Next year Several meth. Some results Some results Next year Some results Next year Brazil Unique feature selection. 2. 3D scene reconstruction. 3. Image registration. 4. Extended object tracking. 5. Dynamic calibration. 6. Optimal sensor control for multitarget tracking. 7. Face localization in human silhouette. 8. Image and multimedia data bases. 9. Face recognition. 1.Abnormal behavior detection. 11.Optimal resource allocation. 12.Object selection. 4

5 Where we are? John Weber, Avnet Classical security systems Brazil 21 5

6 John Weber, Avnet, Smart camera basics The image pipe Brazil 21 6

7 John Weber, Avnet, Smart camera basics Compression Brazil 21 7

8 John Weber, Smart camera basics Information processing Brazil 21 8

9 1. Super-resolution 2. Depth recovery N.B. Using one only controllable IP camera Brazil 21 9

10 Let denote signal by f Rn. This signal can be decomposed in an arbitrary orthonormal basis Ψ = [ψ 1ψ 2 ψ n ] n as follow: f (t ) = xiψ i (t ) i =1, xi = f,ψ i where are the coefficients of the corresponding orthonormal functions. Brazil 21 1

11 Sparse signals are signals for which almost all coefficients are small or zero and relatively small quantity of coefficients may restore almost ideally the signal. Let the restored signal is presented as a sum of the S biggest coefficients in decomposition: f S (t ) = xiψ i (t ) i S The signal f Rn will be called S sparse, if f S approximate very good f and the approximation error ε = f fs l is small enough. For audio- and video2 signals it is well-known that only about 2.5% of the biggest coefficients may be used for signal recovery and the discrimination between the restored signal and the original one can not be easily found. Brazil 21 11

12 Mixed Signal (two sin signals - 32Hz and 256Hz) Brazil time (seconds)

13 Single-Sided Amplitude Spectrum of y(t) with sampling frequency 124Hz Y(f) Brazil Frequency (Hz)

14 Single-Sided Amplitude Spectrum of y(t) with sampling frequency 612Hz i =.25 Y(f) δ (t it ) = ш(t ) Brazil Frequency (Hz) x 1 14

15 Single-Sided Amplitude Spectrum of y(t) with sampling frequency 4Hz Y(f) Brazil Frequency (Hz)

16 Brazil 21 16

17 How to use alias information? 1. Generate in plane camera rotation on random angle π (angle k, k = 1, 2,... 2 ). When such images are registered, the sampling rate is changed. 2. Using camera zoom. The change of zoom rescale the scene in the same sampling rate, hence the sampling rate of one and same scene element is changed. Brazil 21 17

18 zoom In plane rotation Brazil 21 18

19 Low-resolution images Brazil 21 19

20 Enhanced image Brazil 21 2

21 Brazil 21 21

22 Brazil 21 22

23 Brazil 21 23

24 Brazil 21 24

25 "We are creating aliasing just what we teach EEs not to do," said Eldar. "We are folding the highfrequency signal into the lower-frequency domain, all aliased, but modulated by a known highfrequency signal. The key to the post-processing step is to recognize that all the information is still there, just aliased into a low-frequency domain, [and that] by using all four channels together, the original [signal] can be reconstructed. Brazil 21 25

26 3D scene reconstruction is important for: Object localization; Object tracking; Discovering space-time relations between participants in the scene; Object behavior estimation; Future events prediction. Brazil 21 26

27 Miltisensor (stereo) approach stereopsis/depth perception Brazil 21 27

28 Depth recovering by focusing Today, all cameras use different auto-focusing systems and algorithms. The auto-focus system, established in most PTZ IP cameras is based on one of the approaches of the passive focus. Usually they use the fact that the accurately focused image has the highest contrast among all images in the same scene. Brazil 21 28

29 Depth from defocus The algorithm steps: Several frames (2-8) are required from camera on one and same scene. In every frame a different camera focus is set; Edge detection algorithm is applied (Canny); The blur spot diameter is estimated for every line of interest in the processed frames; The estimates for blur diameter for a particular line from all processed frames are input parameters for an optimization procedure for line fitting (Levenberg Marquardt). Brazil 21 29

30 Blur spot diameter estimation Blur spot estimation is carried out onto detected edges. This reduces the number of analyzed image fields. The brightness profile on the processed edge is built and blur width is estimated. The integration of results for many points is applied to reduce the influence of Gaussian additive noise. Also, it is considered the gradient of intensity, not the intensity itself to diminish the role of intensity. Brazil 21 3

31 Focused object Focussed image plane Object out of focus Real image plane Focussed image plane Object Object f f σ2 D fob Dfim Plane of best focus Brazil 21 Dob Dfim D rim Plane of best focus 31

32 Blur spot diameter estimation cont. Brazil 21 32

33 Mathematical model of defocus blur The main equation describing dependencies in this model is based on the Gaussian lens law: = + f D fob D fim B2 σ2 = abs ( Drim D fim ) D fim D fim = Drim fd fob fdob B2 σ2 = abs D fim D fob f Dob f Brazil 21 fd fob D fob f fdob = Dob f 33

34 Mathematical model of defocus blur 6 5 σ2 (mm) Dfob 2 1 Brazil Dob (mm)

35 Experimental results Scene 1 templates Brazil 21 Scene 2 real scene Scene 3 real scene 35

36 Experimental results Test templates scene 1 Zoom 9x Real Scene 2 Zoom 6x Real Scene 3 Zoom 6x Inside Edges Outside Edges Real distance [m] Estimated distance [m] Estimated distance [m] Real distance [m] Estimated distance [m] Real distance [m] Estimated distance [m] Brazil 21 36

37 Plane depth recovery P I =K 2 2 K 4 Dl Dc ( Depth ) where Brazil 21 Dl -Distance between object and light source Dc - Distance between object and camera 37

38 Brazil

39 Brazil

40 Acknowledgments 1.I m much obliged to the organizers of the conference and personally to Dr. Reinaldo Rosa for opportunity to be here, to participate in the conference and to see this wonderful country! 2.This presentation is partially supported by the Bulgarian Ministry of Education and Science under grants VU-MI-24/6 Brazil 21 4

Stable 3D Scene Restoration Using One Active PTZ Camera

Stable 3D Scene Restoration Using One Active PTZ Camera First International Conference Modelling and evelopment of Intelligent Systems Sibiu - Romania, -5 Octer, 009 Wasp based algorithms and applications Stable 3 Scene Restoration Using One Active PTZ Camera