Geo-localization and Mosaicing System (GEMS): Enabling Precision Image Feature Location and Rapid Mosaicing General: info@senteksystems.com www.senteksystems.com 12/6/2014
Precision Agriculture Multi-Spectral Sensing GEMS Sensor Overview GEMS Software Tool Overview GEMS Sample Imagery Spectral Band Alignment GEMS Mosaic Software Benchmarks Conclusions
Market Challenges & Needs: Precise Alignment of imagery across multiple spectral bands to sub-pixel level accuracy Rapid, reliable, robust stitching of images into mosaics Accurate registration of images and mosaics to an absolute reference ellipsoid such as GPS WGS-84 3
Precise Alignment of Imagery Across Spectral Bands High quality vegetation index calculations (i.e. NDVI) requires sub-pixel level spectral band alignment Prevents mixing soil pixels with crop pixels Separates soil and field residue artifacts from crop Factory alignment of camera system is not sufficient to ensure sub-pixel level spectral image alignment Sub-pixel level alignment requires a height above ground estimate in addition to image processing and optimization 4
Stitching Images into Mosaics Rapid, Reliable, Robust Stitching of images into Mosaics Requires provably convergent stitching algorithms converging to a unique mosaic solution Requires computationally tractable stitching algorithms which rapidly converge to the unique mosaic solution Requires the computer throughput and software optimization for fast computation of mosaics 5
Accurate Absolute Registration Accurate Registration of Images and Mosaics with the GPS WGS-84 Absolute Reference Ellipsoid Requires precise alignment between cameras and position and orientation sensors Requires precise time synchronization of image capture times with camera position and orientation sensors Position errors caused by a lack of time synchronization can greatly exceed the position errors of the onboard GPS receiver For a 30 knot UAV flying at 400 ft a 1 second time synchronization error with the position sensor results in a 15 meter error For a UAV flying at 400 ft and rolling 20 degrees per second a 1 second time synchronization error with the orientation sensor results in a 44.3 meter position error!! 6
Totally self-contained End-to-End System (Hardware & Software) Hardware: Collects RGB, NIR, & NDVI imagery in a single flight Aligns RGB & NIR images to sub-pixel level accuracy to produce high quality NDVI imagery High Frame rate allows for high quality NDVI at lower altitudes No need to integrate with external autopilot or navigation system. GPS and inertial sensors already included with GEMS. Auto detects take-off and landing events to initiate start and stop of cameras (default mode of operation) Software: Stitches images together to form RGB, NIR & NDVI mosaiced imagery aligned to sub-pixel level accuracy Exports the GPS coordinates corresponding to any image pixels to a file to enable the return to those image points in the field Auto generates and utilizes image metadata file Auto generates mosaics and NDVI imagery 7
8
GEMS is a precision agriculture sensor payload for remote sensing on small UAVs Multispectral VIS/NIR UAV Payload Dual simultaneously triggered 1.3MP global shutter cameras Fully integrated inertial and GPS sensors on the same PCB Tri-axial gyros, magnetometer, and accelerometers Carrier phase GPS receiver Relative NDVI (crop stress, leaf area, crop cover, biomass, vegetation vigor, field grazing potential for livestock) Weight: 170 grams (including shields) Di e sio s: 3.5 x5 Voltage input: 9-40V Power Consumption: 4.8 Watts 9
Feature Dual RGB & NIR cameras with NDVI optical filters Relative NDVI sensor Benefit Simultaneously triggered global shutter cameras -Images precisely (<1ms) time synchronized with GPS and inertial data to enable precision registration and mosaicing -Global shutter eliminates image artifacts caused by rolling shutter cameras (enables operation in fast flying fixed wing platforms) Continuously high image frame rates Self-contained hardware system: cameras, GPS, inertial navigation sensors integrated on same board 1.4 fps (0.7 sec between images maintained continuously) Ensures consecutive images overlap to enable high accuracy mosaics -No external integration with Autopilot or GPS required -Metadata tightly integrated -Precise alignment of camera and navigation sensors on PCB enables accurate mosaicing and registration 10
Feature Tightly integrated GEMS hardware platform with post-processing mosaicing & registration software Geo-location to pixel level Field of View Exposure duration Benefit No need to purchase 3 rd party mosaicing and registration software -3-5m* absolute accuracy without field markers -1.5cm** relative accuracy with field markers 30 degrees ~1ms (average) (Built in logic to aid camera exposure settings on the fly) *Altitude dependent; ~ 3-5m at 50-100ft **Assumes field markers have been surveyed in to 1.5cm accuracy relative to a GPS RTK base station. The absolute position error of the base station must be added to the 1.5cm relative error to obtain the absolute position error of a pixel location. If field markers have been surveyed in with precise point positioning (PPP) GPS receivers absolute WGS84 pixel position accuracies of 10-20cm (single frequency PPP) or 1-5cm (dual frequency PPP receiver) are achievable. Achievable accuracy is GSD dependent. Release 1 GEMS Tool allows measurement through the GUI to 1-2 GSD. 11
2D image footprint on ground Automatically loads 2D flight trajectory overlaid on satellite imagery Blue dots indicate camera trigger points Right click and view: RGB, NIR, or NDVI for each camera trigger point Pixel geo-location available on every image and in satellite imagery 2D footprints of camera capture point images are displayed on satellite imagery Pixel geo-location 12
RGB, NIR, NDVI images available for every image capture point NDVI aligned to sub-pixel level precision Markers can be placed on any image to indicate points of interest Distance measurement tool included Export all GP points of interest and measurement distances to CSV file to easily go back in the field to examine plant health 13
14
15
16
17
Image Number: 246 Height above ground = 2.113626 m (6.934468 ft) Speed = 9.266337 m/s (20.728241 mph) GSD = 0.1029 cm ( 0.0405 inches) RGB Info: Exposure = 1.000000 ms Distance Traveled During Exposure = 0.926634 cm Translation-based smearing = 9.002395 pixels Mono Info: Exposure: 1.000000 ms Distance Traveled During Exposure = 0.926634 cm Translation-based smearing = 9.002395 pixels GEMS Sample Imagery: NIR 18
GEMS Sample Imagery: RGB 19
GEMS Sample Imagery: NDVI GEMS NDVI Imagery detects photosynthetic imagery. Weeds, partially ripened pumpkins, and pumpkin stems are photosynthetically active. Pumpkin leaves ripened pumpkins 20 are no longer photosynthetically active.
Image Number: 245 Height above ground = 3.482895 m (11.426820 ft) Speed = 9.454248 m/s (21.148585 mph) GSD = 0.1696 cm ( 0.0668 inches) RGB Info: Exposure = 1.000000 ms Distance Traveled During Exposure = 0.945425 cm Translation-based smearing = 5.573971 pixels Mono Info: Exposure: 1.000000 ms Distance Traveled During Exposure = 0.945425 cm Translation-based smearing = 5.573971 pixels GEMS Sample Imagery: NIR 21
GEMS Sample Imagery: RGB 22
GEMS Sample Imagery: NDVI GEMS NDVI Imagery detects photosynthetic imagery. Weeds, partially ripened pumpkins, and pumpkin stems are photosynthetically active. Pumpkin leaves ripened pumpkins 23 are no longer photosynthetically active.
Image Number: 238 Height above ground = 2.457909 m (8.064008 ft) Speed = 11.910979 m/s (26.644145 mph) GSD = 0.1197 cm ( 0.0471 inches) RGB Info: Exposure = 1.000000 ms Distance Traveled During Exposure = 1.191098 cm Translation-based smearing = 9.950837 pixels Mono Info: Exposure: 1.000000 ms Distance Traveled During Exposure = 1.191098 cm Translation-based smearing = 9.950837 pixels GEMS Sample Imagery: NIR 24
GEMS Sample Imagery: RGB 25
GEMS Sample Imagery: NDVI GEMS NDVI is aligned to sub-pixel level accuracy. Notice the clarity of imagery 26 at a relatively low altitude of 8 ft.
Quickview Fine Mosaic 27
Fine Mosaic RGB 28
Fine Mosaic NIR 29
Fine Mosaic NDVI 30
GEMS is the most self contained end-to-end low cost high performance precision agriculture multispectral imaging system on the market today Sentek is dedicated to developing advanced sensors and software packages to simplify the end users workflow. 31
1. Higher Camera Resolution- When it Helps and When it Hurts, Sentek Systems Technical Report November 2014 2. Prasad S. Thenkabail, John G. Lyon, and Alfredo Huete, "Hyperspectral Remote Sensing of Vegetation", CRC Press, 2012. 3. California Agriculture 50(4):14-18. DOI: 10.3733/ca.v050n04p14. July-August 1996. 32