Geo-localization and Mosaicing System (GEMS): Enabling Precision Image Feature Location and Rapid Mosaicing General:

Transcription

1 Geo-localization and Mosaicing System (GEMS): Enabling Precision Image Feature Location and Rapid Mosaicing General: 12/6/2014

2 Precision Agriculture Multi-Spectral Sensing GEMS Sensor Overview GEMS Software Tool Overview GEMS Sample Imagery Spectral Band Alignment GEMS Mosaic Software Benchmarks Conclusions

3 Market Challenges & Needs: Precise Alignment of imagery across multiple spectral bands to sub-pixel level accuracy Rapid, reliable, robust stitching of images into mosaics Accurate registration of images and mosaics to an absolute reference ellipsoid such as GPS WGS-84 3

4 Precise Alignment of Imagery Across Spectral Bands High quality vegetation index calculations (i.e. NDVI) requires sub-pixel level spectral band alignment Prevents mixing soil pixels with crop pixels Separates soil and field residue artifacts from crop Factory alignment of camera system is not sufficient to ensure sub-pixel level spectral image alignment Sub-pixel level alignment requires a height above ground estimate in addition to image processing and optimization 4

5 Stitching Images into Mosaics Rapid, Reliable, Robust Stitching of images into Mosaics Requires provably convergent stitching algorithms converging to a unique mosaic solution Requires computationally tractable stitching algorithms which rapidly converge to the unique mosaic solution Requires the computer throughput and software optimization for fast computation of mosaics 5

6 Accurate Absolute Registration Accurate Registration of Images and Mosaics with the GPS WGS-84 Absolute Reference Ellipsoid Requires precise alignment between cameras and position and orientation sensors Requires precise time synchronization of image capture times with camera position and orientation sensors Position errors caused by a lack of time synchronization can greatly exceed the position errors of the onboard GPS receiver For a 30 knot UAV flying at 400 ft a 1 second time synchronization error with the position sensor results in a 15 meter error For a UAV flying at 400 ft and rolling 20 degrees per second a 1 second time synchronization error with the orientation sensor results in a 44.3 meter position error!! 6

7 Totally self-contained End-to-End System (Hardware & Software) Hardware: Collects RGB, NIR, & NDVI imagery in a single flight Aligns RGB & NIR images to sub-pixel level accuracy to produce high quality NDVI imagery High Frame rate allows for high quality NDVI at lower altitudes No need to integrate with external autopilot or navigation system. GPS and inertial sensors already included with GEMS. Auto detects take-off and landing events to initiate start and stop of cameras (default mode of operation) Software: Stitches images together to form RGB, NIR & NDVI mosaiced imagery aligned to sub-pixel level accuracy Exports the GPS coordinates corresponding to any image pixels to a file to enable the return to those image points in the field Auto generates and utilizes image metadata file Auto generates mosaics and NDVI imagery 7

8 8

9 GEMS is a precision agriculture sensor payload for remote sensing on small UAVs Multispectral VIS/NIR UAV Payload Dual simultaneously triggered 1.3MP global shutter cameras Fully integrated inertial and GPS sensors on the same PCB Tri-axial gyros, magnetometer, and accelerometers Carrier phase GPS receiver Relative NDVI (crop stress, leaf area, crop cover, biomass, vegetation vigor, field grazing potential for livestock) Weight: 170 grams (including shields) Di e sio s: 3.5 x5 Voltage input: 9-40V Power Consumption: 4.8 Watts 9

10 Feature Dual RGB & NIR cameras with NDVI optical filters Relative NDVI sensor Benefit Simultaneously triggered global shutter cameras -Images precisely (<1ms) time synchronized with GPS and inertial data to enable precision registration and mosaicing -Global shutter eliminates image artifacts caused by rolling shutter cameras (enables operation in fast flying fixed wing platforms) Continuously high image frame rates Self-contained hardware system: cameras, GPS, inertial navigation sensors integrated on same board 1.4 fps (0.7 sec between images maintained continuously) Ensures consecutive images overlap to enable high accuracy mosaics -No external integration with Autopilot or GPS required -Metadata tightly integrated -Precise alignment of camera and navigation sensors on PCB enables accurate mosaicing and registration 10

11 Feature Tightly integrated GEMS hardware platform with post-processing mosaicing & registration software Geo-location to pixel level Field of View Exposure duration Benefit No need to purchase 3 rd party mosaicing and registration software -3-5m* absolute accuracy without field markers -1.5cm** relative accuracy with field markers 30 degrees ~1ms (average) (Built in logic to aid camera exposure settings on the fly) *Altitude dependent; ~ 3-5m at ft **Assumes field markers have been surveyed in to 1.5cm accuracy relative to a GPS RTK base station. The absolute position error of the base station must be added to the 1.5cm relative error to obtain the absolute position error of a pixel location. If field markers have been surveyed in with precise point positioning (PPP) GPS receivers absolute WGS84 pixel position accuracies of 10-20cm (single frequency PPP) or 1-5cm (dual frequency PPP receiver) are achievable. Achievable accuracy is GSD dependent. Release 1 GEMS Tool allows measurement through the GUI to 1-2 GSD. 11

12 2D image footprint on ground Automatically loads 2D flight trajectory overlaid on satellite imagery Blue dots indicate camera trigger points Right click and view: RGB, NIR, or NDVI for each camera trigger point Pixel geo-location available on every image and in satellite imagery 2D footprints of camera capture point images are displayed on satellite imagery Pixel geo-location 12

13 RGB, NIR, NDVI images available for every image capture point NDVI aligned to sub-pixel level precision Markers can be placed on any image to indicate points of interest Distance measurement tool included Export all GP points of interest and measurement distances to CSV file to easily go back in the field to examine plant health 13

14 14

15 15

16 16

17 17

18 Image Number: 246 Height above ground = m ( ft) Speed = m/s ( mph) GSD = cm ( inches) RGB Info: Exposure = ms Distance Traveled During Exposure = cm Translation-based smearing = pixels Mono Info: Exposure: ms Distance Traveled During Exposure = cm Translation-based smearing = pixels GEMS Sample Imagery: NIR 18

19 GEMS Sample Imagery: RGB 19

20 GEMS Sample Imagery: NDVI GEMS NDVI Imagery detects photosynthetic imagery. Weeds, partially ripened pumpkins, and pumpkin stems are photosynthetically active. Pumpkin leaves ripened pumpkins 20 are no longer photosynthetically active.

21 Image Number: 245 Height above ground = m ( ft) Speed = m/s ( mph) GSD = cm ( inches) RGB Info: Exposure = ms Distance Traveled During Exposure = cm Translation-based smearing = pixels Mono Info: Exposure: ms Distance Traveled During Exposure = cm Translation-based smearing = pixels GEMS Sample Imagery: NIR 21

22 GEMS Sample Imagery: RGB 22

23 GEMS Sample Imagery: NDVI GEMS NDVI Imagery detects photosynthetic imagery. Weeds, partially ripened pumpkins, and pumpkin stems are photosynthetically active. Pumpkin leaves ripened pumpkins 23 are no longer photosynthetically active.

24 Image Number: 238 Height above ground = m ( ft) Speed = m/s ( mph) GSD = cm ( inches) RGB Info: Exposure = ms Distance Traveled During Exposure = cm Translation-based smearing = pixels Mono Info: Exposure: ms Distance Traveled During Exposure = cm Translation-based smearing = pixels GEMS Sample Imagery: NIR 24

25 GEMS Sample Imagery: RGB 25

26 GEMS Sample Imagery: NDVI GEMS NDVI is aligned to sub-pixel level accuracy. Notice the clarity of imagery 26 at a relatively low altitude of 8 ft.

27 Quickview Fine Mosaic 27

28 Fine Mosaic RGB 28

29 Fine Mosaic NIR 29

30 Fine Mosaic NDVI 30

31 GEMS is the most self contained end-to-end low cost high performance precision agriculture multispectral imaging system on the market today Sentek is dedicated to developing advanced sensors and software packages to simplify the end users workflow. 31

32 1. Higher Camera Resolution- When it Helps and When it Hurts, Sentek Systems Technical Report November Prasad S. Thenkabail, John G. Lyon, and Alfredo Huete, "Hyperspectral Remote Sensing of Vegetation", CRC Press, California Agriculture 50(4): DOI: /ca.v050n04p14. July-August