SMARTSCAN Smart Pushbroom Imaging System for Shaky Space Platforms Klaus Janschek, Valerij Tchernykh, Sergeij Dyblenko SMARTSCAN 1
SMARTSCAN Smart Pushbroom Imaging System for Shaky Space Platforms Klaus Janschek, Valerij Tchernykh, Sergeij Dyblenko Contents: Problem Definition SMARTSCAN Principle Image Motion Measurement Airborne Test Results Conclusions & Outlook SMARTSCAN 2
Problem definition Satellite attitude motion effects on pushbroom camera attitude instability Distance, pixels Stable image motion In flight direction Perpendicular to the flight direction non-parallel scanning Distance, pixels Time, line exposure intervals Disturbed image motion In flight direction Perpendicular to the flight direction Time, line exposure intervals Simulated images (10 lines TDI) SMARTSCAN 3
Solution: SMARTSCAN image motion compensation Features in-situ focal plane image motion measurement posteriori motion compensation allowsless costly remote sensing from simple platforms SMARTSCAN 4
Image motion tracking by 2D correlation Focal Plane Area Sensor image 1 t=t 1 image 2 t=t 2 image motion s 2D Correlation s image shift vector 2D correlation weakly structured image texture (no features used!) robustness to small signal-to-noise ratio image shift vector s high computational effort image shift vectors s i flight direction raw image data (linear sensor) SMARTSCAN 5
JTOC Joint Transform Optical Correlator S CCD + { I ( x, y) I ( x, } 2 ( u, v) = F 1 1 2 y) S { S ( u, v) } 2 2 CCD ( l, m) = F 2 CCD = I 1 1( x, y) I2( x, y) Image 1 Image 2 Optical Fourier Processor 1 Optical Fourier Processor 2 I 1( x, y) + I2( x, y) input image(s) S CCD ( u, v) 1 spectrum image S CCD ( l, m) 2 correlation image JTOC [Goodman 1968] 60 corr./sec. Patent DE 100 47 504 B4, (2005) SMARTSCAN 6
JTOC Accuracy & Robustness Subpixel Image Shift Measurement Image 1 D 0 Y Correlation function C = a n, m b n + i m + c i, j, + j n m j Correlation maximum Correlation i CCD D D = = D 00 + + EE X Image 2 E=(e x, e x y), e y ) Measurement error Error subpixel accuracy RMS error, pixels RMS error, pixel 0.4 0.2 Image Noise horizontal vertical 0.3 0.1 0 0 10 20 30 40 0 10 20 30 40 Signal to noise ratio, db signal to noise ratio, db RSS of RMS error, pixels RSS of RMS error, pixel 0.3 0.2 0.1 0 0 Window Size 80 60 120 80 160 120 240 160 Image size, pixel image size, pixels 80x60 120x80 160x120 240x160 SMARTSCAN 7
JTOC Accuracy & Robustness Subpixel Image Shift Measurement Image Texture Image 1 D 0 Y Correlation function C = a n, m b n + i m + c i, j, + j n m j Correlation maximum RMS error, pixels RMS error, pixel X 0.4 0.3 0.2 0.1 Image 2 Correlation i CCD D D = = D 00 + + EE E=(e x, e x y), e y ) Measurement error Error subpixel accuracy Image Noise horizontal vertical JTOC block size 128x128 pixels greyscale 0 255 random additive image noise 20 greyscale bit (1σ) Test image Successful correlation s Standard deviation of error [pixels] Urban scene (A) 100.0% 0.0127 Rural scene (B) 91.9% 0.0166 Mountain with snow and clouds (C) 93.1% 0.0332 Water surface 98.2% 0.0062 0 80x60 10 20 30 40 60 120x80 160x120 240x160 80 120 160 with waves (D) Signal to noise ratio, db Image size, pixel signal to noise ratio, db image size, pixels Water surface 28.6% 0.1061 w/o waves (E) SMARTSCAN 8 0 0 10 20 30 40 0 RSS of RMS error, pixels RSS of RMS error, pixel 0.3 0.2 0.1 Window Size A B D C E
SMARTSCAN breadboard Sensors configuration in the focal plane CCD cameras (auxiliary matrix sensors) Lens Linear Sensor Camera breadboard Lens Tessar type: f = 75 mm; f/4; angular resolution - 187 μrad/pixel Linear sensor Resolution: 2048 pixels in line; lines rate: 240 lines per second Optical correlator breadboard 60 correlations per second Input two video streams Output image motion record Double optical unit Self-calibration Auxiliary matrix sensors Dimensions / mass Frame size: 640 x 480 pixels; frames rate: 30 frames per second 110 x 58 x 50 mm / 900 g SMARTSCAN 9
Airborne Test Department of Electrical Engineering and Information Technology Matrix Sensors Camera Mount Optical Correlator Model Linear sensor Portable PC (notebook) Chair for Automation Engineering Altitude: 2400 m Velocity: 240 km/h = 66.7 m/s Ground resolution: 0.45 m/pixel Lens Camera mount (suppresses vibrations above 10 Hz) SMARTSCAN 10
Airborne Test Results Department of Electrical Engineering and Information Technology SMARTSCAN Chair for Automation Engineering 11
Airborne Test Results Department of Electrical Engineering and Information Technology Chair for Automation Engineering Image motion (pixels) Ideal IMR in the flight direction Real IMR in the flight direction Real IMR perpendicular to the flight direction Number of the line SMARTSCAN 12
Airborne Test Results 2D-correlator Image Motion Time History + Image Correction Original (distorted) image Corrected image SMARTSCAN 13
Airborne Test Results 2D-correlator Image Motion Time History + Image Correction Original (distorted) image Corrected image SMARTSCAN 14
Conclusions & Outlook Achieved SMARTSCAN principle successfully demonstrated method: image motion tracking using 2D-correlation key technology: embedded optical correlator, in-flight tested application: high accuracy imaging from shaky platforms SMARTSCAN 15
Conclusions & Outlook Achieved SMARTSCAN principle successfully demonstrated method: image motion tracking using 2D-correlation key technology: embedded optical correlator, in-flight tested application: high accuracy imaging from shaky platforms Next Steps advanced optical correlator development space qualification of OC technology micro mirror SLM navigation images image motion record SMARTSCAN 16
-- end of presentation -- www.et.tu-dresden.de/ifa SMARTSCAN 17