Mapping with RedEdge on a 3DR Solo Platform Jerry Davis, Institute for Geographic Information Science, San Francisco State University

Transcription

1 Mapping with RedEdge on a 3DR Solo Platform Jerry Davis, Institute for Geographic Information Science, San Francisco State University The purpose of this guide is to provide an overview of the process of mapping a site with the MiceSense RedEdge 5 band sensor mounted on a 3DR Solo quadcopter platform, along with providing some important details that might be missed by reading the manufacturers' manuals and some discoveries from our experience in test and operational runs. The manuals are also useful references and provide other details. To be successful collecting imagery requires understanding both the sensor and how to safely and effectively fly the drone, as well as how to communicate between the two. MicaSense RedEdge sensor The MicaSense RedEdge 5 band sensor was designed primarily for precision agriculture applications, where farmers can look for moisture and other stress on their fields, but also works well for vegetation mapping. It captures narrow bands of blue (centered on 475 nm), green (560 nm), red (668 nm), near infrared (NIR, 840 nm), and red edge (717 nm), allowing for accurate detection of vegetation health and discernment of species based on their spectral signatures. Useful guides on the sensor have been provided at equipment/micasenserededge 5 band sensor and include: RedEdge User Manual RedEdge Downwelling Light Sensor Integration Guide probably others Also see Greg Crutsinger's guide notes on mapping withthe sequoia camera on 3dr solo/ Figure 1. Five spectral bands (1)blue, (2) green, (3) red, (5) red edge, (4) near Infrared of the MicaSense RedEdge camera, and typical plant spectral signature. 1

2 3DR Solo platform The platform for the RedEdge is the 3DR Solo quadcopter, chosen for its customizability for mounting the sensor and its components. We are using the Tower app to plan and execute missions. Useful guides are to be found at equipment/3dr solo 3DR Solo User Manual 3DR Solo and Tower Mapping Guide Figure 2. 3D Robotics Solo quadcopter with the MicaSense RedEdge mounted below and downwelling light sensor and GPS mounted above. 2

3 There are two apps that communicate with the 3DR Solo: Solo and Tower. Both are from 3DR. Both use WiFi to communicate from your Android phone/tablet. Only one can be on at a time. Solo app: For controlling manually, controlling a GoPro Hero 4 or some other cameras, but for just flying you can just use the remote controller. Tower app: Designed for planning a mission for 3D image capture or mapping. We'll focus on mapping needs basically capturing vertical aerial photography. This guide will provide a basic look at Tower for capturing aerial photography, choosing options that work well with the MicaSense Rededge multispectral camera. To learn more about Tower, there's not much in the way of a manual, but users have created YouTube videos, which you can find via Google or YouTube searches. Here are some I've found useful: Tower mission planning video (50 minutes): Video of spline survey method, including whole flight: We'll use a spline survey method. Mapping with the MicaSense RedEdge sensor mounted on the Solo platform requires a combined use of communicating with the Solo with Tower to create and run missions and communicating with the RedEdge to set up imaging. You'll want to use separate mobile devices to talk to the RedEdge and the Solo, each using WiFi. The RedEdge really just gets power from the Solo they don't communicate. Tower runs only in Android, so that may determine your choice. To collect images in the RedEdge, we're going to use the Overlap which starts taking pictures automatically when it reaches 50 m below the target altitude, so once this is set up you can just create and run a mission using the Tower app. Essentials of the Tower App The two most frequently used views in Tower are: Vehicle Data, where you'll link to the aircraft, check its status and fly the mission; and Editor, where you'll create mission plans. You can create mission plans on site or elsewhere, and it's obviously easier to work not out in the bright sun so you can see the screen. 3

4 Vehicle Data View Connecting, arming, and controls in flight. Connect mode, connect, settings Readings and menu, including to upload and download mission. Arm (to start props) and Dronie. To execute the misssion after arming, slide the bar. During flight, return home, land in place, pause, run Auto mission (do this after initial launch), follow me, settings. Aircraft attitude 4

5 Editor View Controlling the display Zoom to GPS Zoom to aircraft Zoom to mission Toggle display of feature properties Creating and editing waypoints Create Waypoint, Spline Waypoint, Circle, Land, Region of Interest, Structure Scanner Draw a line or poly to create multiple waypoints: Waypoints, Spline Waypoints, Survey, Spline Survey Select all Deleted selected or entire mission Undo 5

6 Editing waypoints Select waypoint to get to choices of type of waypoint or event: Waypoint types can be simple or more complex, like a point the aircraft moves around at a set radius. Events may be associated with a point or just a change of settings like speed. For a survey, start with a Change Speed setting and create a Survey or Spline Survey to create a set of waypoints. Save a survey by exporting to file. Useful when trying find your study site on a cached basemap image. Waypoints Definition Properties to modify Waypoint Waypoint connected from last and to next waypoint Altitude, Delay (before next waypoint) with a straight path Spline Waypoint Waypoint connected with Altitude, Delay spline curves around bends Circle Center waypoint with circlue Altitude, Radius, N turns at radius Region of Interest (ROI) Point at the focus of a Altitude survey. Copter points toward or aims camera. Structure Scanner Map a building Radius, Start altitude, Steps Events Change Speed Takeoff Land Return to Launch Camera Trigger EPM Gripper Set Yaw Set Servo Do Jump Reset ROI From after the last waypoint, change the speed From wherever the aircraft is (so isn't recorded as a point) to a specified altitude Land at current location. You must manually exit Auto mode to disarm motors Return to location the aircraft was armed Trigger the camera at regular distance intervals Release the payload Set condition to reach a certain target angle Move a servo to a PWM value Jump to a particular waypoint Reset the ROI lock Speed Altitude, Pitch Length Target angle Channel, PWM Waypoint, Repeat count 6

7 Setting up the RedEdge for Mapping with Tower 1. Prep the site with tarp, etc., and install an SD card in the RedEdge. 2. Power up the Solo to get it finding GPS and establish a connection. 3. This will power up the RedEdge, which will progress through a sequence of LED flashes on the front (facing down) of the camera: Red 1 flash : No SD card Red 2 flashes : SD card is full or close to full Red 3 flashes : GPS timeout Red 4 flashes : Lower power source Yellow 1 flash: GPS unit detected, but no fix yet Green 1 flash: Camera is ready for operation GPS is good and SD has > 2 GB And on the back of the camera (facing up), there are two other LEDs: Rearward on the Solo, Blue continuous: Power on Forward on the Solo, Red 2 flashes: Capturing picture 4. Enable WiFi on mobile device and look for rededge signal. Password micasense. You may need to turn off cellular data to keep it from using that connection from the browser. It can take a while to connect, and may need rebooting the camera by turning off power from the Solo and back on. Fortunately, once it finds it, it tends to be reliable. Then enter in a browser to connect to the onboard site. Note that if you've run this before, there may be a cached version still showing even though it's not communicating with the RedEdge. The home page will have no data in it, and the camera page will look like it's set to manual exposures with 5 rows Exposure 1 Exposure 5. 7

8 The four menu areas in the RedEdge page Main Camera Map Configuration Shows the current conditions for the camera and the SD card. You can use this to capture what it is seeing in the 5 separate band images. Note: Streaming is just for live viewing of what the camera is seeing and doesn't save. Use the Pre Flight Estimator to determine entries for the configuration. Enter the altitude, flight speed, area of the field, foreward overlap, cross overlap, camera orientation (landscape), image output (TIFF 16bit), aircraft type (multi copter). Use this to set the auto capture mode. We'll use Overlap and provide the target altitude of up to 120 m, and along track overlap of 80% and cross track overlap of 70%. The camera will start capturing when it reaches an altitude 50 m below the target altitude, and stop when it drops below. See Triggering Options at us/articles/ RedEdge Automatic Triggering Options 8

9 Camera settings to start the mission We'll use the Overlap method, which is easy enough to set up in the configuration. It seems to take effect when you but no clue about how to start it. If we use the recommended auto exposure and Overlap mode, and then enter a suitable camera (with a similar horizontal field of view (HFOV) in Tower, we shouldn't need to do much. In fact, just leaving the saved Overlap settings should mean that we don't need to go into the program at all just use the manual shutter button to take a picture of the calibration panel. Pre-Flight Estimator Optionally, determine the spacing of the flight in advance using the Pre Flight Estimator to display various parameters based on a % overlap and height setting. Simpler is to just use a similar HFOV camera (e.g. the Sentek GEMS V1) in Tower. Footprint Width m Footprint Height m Distance between Capture m Distance between Track m Time between Capture s Flight Time min Number of Captures Number of Images Area per Hour Storage Space Reqt GB Overlap Mode Set up overlap mode survey in the Configuration screen, with the percentage (alongtrack) overlap you've chosen (e.g. 75%). Saving this will initiate photos 50 m below the target altitude, so just set that. See us/articles/ Best practices Collecting Data with MicaSense RedEdge and Parrot Sequoia. Leave as Auto Exposure not manual According to MicaSense, manual exposure settings are not needed, and good calibration is achieved with auto exposure. Bands will actually need different exposures. Capture the calibration panel Either with the manual button (may want to watch the light forward on the Solo to see if it takes it) or with the Camera tab in the web page, capture a panel image. Follow the instructions shown in the calibration panel. 9

10 Tower 1. Now power up the controller and go to Tower on the tablet. You may want to first check things in the Solo app. 2. In Tower, prepare a mission and save it (export it) to your tablet. A good type of survey is a spline survey, as demonstrated in : but basically involves these steps: 3. In Editor mode, either create a new mission or import a saved mission. It's a lot easier to create a mission where you have some shade so you can read the screen. You don't have to be at the site to create it, but you do need to be there to run it, and your first waypoint should be accessible from the home point where you power up the aircraft. You should also confirm that there are no obstructions in the path of the mission. a. Create a waypoint and convert it into a speed change, setting that to maybe 5 m/s. You could also make this the default in advanced settings. b. Use the draw tool and change the type to Spline Survey and draw the boundary of what you want to survey. Alternatively, place waypoints manually using the footprint info in the table. c. In the settings for the spline survey, set the flight altitude, overlap, sidelap and camera type. If the RedEdge is still not listed (as "MicaSense Landscape"), use a close approximate in terms of horizontal field of view (HFOV), the Sentek GEMS V1 (48 HFOV), close to the 47.2 HFOV of the RedEdge. Alternatively, copy the text from the textbox below to a micasense.xml file and store it in a folder (you need to create) at: Internal storage/3drservices/camerainfo. The field of view is really the only camera property that matters here, since it determines what is in the scene for determining overlap. d. See Creating Waypoints in ArcGIS section below for an alternative method that provides more control. Altitude (m) <camerainfo> <Name>Micasense Landscape</Name> <SensorWidth>4.8</SensorWidth> <SensorHeight>3.6</SensorHeight> <SensorResolution>1.23</SensorResolution> <FocalLength>5.5</FocalLength> </camerainfo> 10 Footprint width Footprint length Track 75% Capture 75%

11 4. Vehicle Data mode: o o Make sure you're connected via WiFi to the aircraft. You may need to select it in your tablet settings. There will be a WiFi password ours is solo. Note that when you connect, it will display a message that it's not connected to the internet, which it of course isn't. That's ok. When connected, you should see the home position where the aircraft first powered up, and the current location of the aircraft, as well as the location of your tablet (blue dot). You'll also see the current aircraft attitude which you can turn on or off using the icon. o o Upload the mission to Solo using the menu dots in the upper right. It will prompt you "Do you want to append a Takeoff and RTL to your mission?" which you might want to Ok. You should hear a message "Waypoints saved to drone." The mission will be ready to arm and run. You'll see an "Arm" icon on the screen next to the "Dronie" icon. When you arm it, a slider will appear confirming this o choice. During the flight, you'll see a new set of icons that provide access to return home, land in place, pause, auto, and follow. When ready, click Auto to go through the waypoints. Pause either here or on the controller. According to an entry in 3drpilots.com, either the FLY or the PAUSE button on the controller takes over manual control. I've also programmed the A button to go to Fly:Manual control (no GPS) which should only be used if the GPS is causing the drone to fly away to the wrong place. The pause button in Tower just pauses the flight, but it should continue. Note: Review the Fail safe Procedures before all flights. 11

12 Creating Waypoints in ArcGIS Tower is a decent app for combining controlling the drone and creating a flight mission, however it is missing a lot of features you'll find in a GIS program like ArcGIS that can really help: like a scale bar, more choices of basemaps, and especially the ability to edit features related to other features through the use of snapping and other editing tools. Getting these into a waypoint file suitable for input to Tower takes a bit of manual editing and knowledge of ArcGIS and Excel, but the results illustrate the level of control possible. Here are some basic steps: 1. Creating Waypoints along a Flight Line in ArcGIS You'll need to know how to use ArcGIS to create feature classes or shapefiles and how to use its editing capabilities and various geoprocessing tools. First use the editing tools in ArcGIS to create a flight line. Note you create it and the following waypoints in the coordinate system of the map you're editing in. If you're creating parallel up and back flight lines, use the measure tool to set their spacing to the right distance for overlap. Then create waypoints along it using the snapping function built into the ArcGIS editor. Create waypoint ids in the order you want to fly them. Note that you don't need to make waypoints at the photo spacing for overlap because the RedEdge will take care of that, taking photos along the track at a distance to ensure the specified overlap, though make sure all of your overlap settings and choices coincide. Project into GCS Add GCS coordinates with the Add XY Coordinates tool. Open the attribute table, and export to a text file. 2. In Excel, convert these to the waypoint format used in Tower You'll need a template waypoint file, so copy one exported from Tower and stored in the Tower/Waypoints folder on the tablet. Open the text file exported from ArcGIS and the template waypoint file in Excel. Note that the former will be comma delimited and the latter will be tab delimited. Fit the ArcGIS exported data into the template. Note you'll have to reverse latitude and longitude (otherwise you'll send your drone into the ocean!) which are exported as longitude and latitude by ArcGIS. The column after longitude in the template is the flight AGL, so set that to the height you're using that matches the overlap spacing. There are several columns before latitude and longitude that I haven't research what they they do, but just set them the same unless you've learned what they do. Save the waypoint file you've created based on the template, as text format. Excel will warn you that you're losing features, but that's ok, it's just a tab delimited text file. Copy the waypoint file to the Tower/Waypoints folder on the tablet. Check Tower to see if you can Import it in the Editor. 12

13 Caching Basemaps for Offline Use In order to see basemaps like satellite imagery when offline, you need to be able to cache image basemaps. This requires using an online map data service that allows for caching. In Tower at least, the standard Google map service doesn't support this, but Mapbox does. In order to do this, the following method works at least currently: 1. Create a Mapbox custom map with a satellite image and streets style, and get the map id and access token: a. Get a free Mapbox account if you don't have one already. b. Download the Mapbox Studio Classic on your laptop, and build a map with just a satellite and streets style. There may be a way to use their more current app, but this produces the map id and access token that is needed in Tower and described in which is about creating offline maps in Tower using Mapbox. The map id is your Mapbox user name followed by a dot and an 8 character (actually 4 bytes in hexadecimal) code. c. As suggested in the YouTube video, the best way to get the very long access token is to copy it in Mapbox Studio Classic and it to yourself to bring up on your tablet. 2. Also following the YouTube video, go to settings in Tower, then User Interface, then Map Provider Preferences, and set Mapbox as Map Tile Provider. Check "Enable offline map layer" and "Add Download Map to menu" and enter the map id in Mapbox Id and the long Mapbox Access Token. 3. Then once you have the area of interest displayed, you can pull down the upper right 3 dot menu and download the map while online. The map downloaded will cover the whole area shown at all finer resolutions. 4. When offline, this should result in being able to see the cached basemap typically in the Editor screen only, but it may not immediately appear. Try zooming in and out. Sometimes you have to restart Tower. It's a good idea to export a mission for each cached site area to a file, even if you're going to create a new one in the field. It can sometimes be hard to find your site and cached basemaps when offline at the site. When you import an exported mission from file, the map zooms to the area of the mission. You can then edit it with the cached basemap visible. 13

14 Fail Safe procedures to review before flying. It can be nerve wracking during a mission when something goes wrong. You need to know what to do without having to look it up. So it can be critically important to review some emergency procedures in advance to know what to do, and to know what the normal response of the Solo will be to any failures in the guidance system. This information is all covered in the Solo manual. Important: make sure your antennas are pointed correctly; you'll have no control if you lose this. What happens when Solo loses GPS position? If GPS position drops beneath a reliable threshold during flight, Solo will automatically switch to Fly:Manual mode. This mode does not provide the flight stabilizing GPS functionality, and will drift with the wind, but you can control Solo visually. Visual flight should be practiced, and it helps to know that the tail lights are red and that the antennas are correctly set. What happens when Solo loses controller connection? Displays "Controller signal lost" and Solo initiates Return Home. If signal is recovered, displays "Signal recovered" and standard manual control (includes GPS) can be regained with FLY button. Note that this is not Fly:Manual (no GPS), which you can force it into by programming the A button to do this. To return to normal FLY mode, press either FLY, Pause, or RTH. What happens when battery failsafe triggers? At 10% battery, Solo initiates Return Home. Warnings at 25% and 15%. What is the responsibility of the pilot when the above happen? First, if the pilot in command (PIC) is not at the controls, that pilot should take control. What are all the options for return to home and how are they set/used? RETURN HOME button If the Solo loses controller connection and it's in normal FLY mode, it should return home. See the Cone Zone discussion (p. 32) defining where it can safely move while returning home (Or to land at current location, press and hold the FLY button.) What are the characteristics of solos flight modes? Standard flight : manual with GPS lock to maintain position. Control with joysticks. Fly:Manual : a version of standard flight without GPS lock. Control with joysticks, but note that the aircraft will drift with the wind and momentum. Requires constant adjustment to maintain (right stick) position and (left stick) orientation. Should be set to Preset A. Can be overridden by pressing FLY, Pause, or RETURN HOME. Guided : programmed by app (Tower, Solo). Can be overriden 14

15 Control buttons: FLY : The Fly button lets you control Solo s main flight functions: starting motors, takeoff, land, and activating GPS flight. You can land in place by holding the FLY button. RETURN HOME : If initial GPS position established in power up, will return to that location. PAUSE : will hover in place, including interrupting Return Home. As with RETURN HOME, it depends on GPS however, so will have no effect if GPS signal is lost. A : Assuming you're an experienced pilot, should be programmed to go to FLY:Manual mode, which does not use GPS for stabilization and will drift in the wind. On the 3DRPilots Forum, this is recommended by some, but not by others who note that if the GPS signal is lost the aircraft will automatically go to FLY:Manual mode and forcing this could create problems for inexperienced pilots. Your connection via antenna becomes critical if you're in FLY:Manual mode since it's not using GPS. One purpose of this setting is to practice flying in this mode, which you will probably need someday. Note that in this mode, similar to standard, the throttle stick corresponds to altitude. You must Enable Advanced Flight Modes in Advanced Settings in the Solo app to be able to assign this or other advanced modes to the preset. B : Also assuming you're an experienced pilot, should be programmed to go to Stabilize mode. In Stabilize mode, the throttle stick controls power and acceleration directly. Should be practiced, but presumably could work to overcome situations where the altitude is not detected by the aircraft. See the Fly manual conundrum thread at 3DRpilots.com: manual conundrum.11189/ 15

16 Checklist for RedEdge Mapping Note: This checklist assumes you're already familiar with the details described in the rest of this document, and just lists the basic steps as a reminder. Preparation o Confirm < 25 mph winds o Install tarp o Clear of people. o Review emergency protocols, assign roles (PIC, VOs) o Review manual override, return to home methods. Install SD card in RedEdge Power up the Solo, but not the controller yet, on the tarp. Wait for and ensure that RedEdge light facing down is blinking green, indicating good GPS. RedEdge Setup. Only done to change settings, not needed if Overlap mode has been previously set and are settings are configured to the right overlap percentage. o Enable WiFi on mobile device and look for rededge (or with different serial number) signal. Password micasense. o In browser: to start up the RedEdge Camera app by MicaSense program. o In the RedEdge app: Home: should see storage, GPS data, a good indicator that you're all hooked up. Configuration: Overlap Set Target Alt (at least 60 m). Along track overlap : 75% recommended. Do not set to manual. Save Settings Camera: Capture the calibration panel from the Camera menu. o At this point, the RedEdge is ready to automatically initiate image collection, so close down that tablet and we'll shift to Tower and the RC. Power up remote controller (RC) and position the antennas at 20 out from down. Connection is best if perpendicular to the sight line. Open Tower and Connect in Vehicle Data mode. Should see Aircraft attitude. 16

17 o May need to go to WiFi settings on tablet o Look for Arm icon In Editor, open saved mission (import from file) or create a new one. If a basemap image was saved, it should show. If missing, try restarting Tower. Tower is buggy with images so you may have to cycle back and forth from Editor to Vehicle data. o The first event (created initially as a waypoint then changed) should be a change speed to 5 m/s. While you can change this default setting (normally at 11 m/s), this doesn't seem to stick so do this for each flight. If you're adding this event, move it to step 1. Upload to aircraft, and reply yes to adding takeoff and landing, which will add numbered events as waypoints at the beginning and end of the mission. Make sure you're not flying a mission that's somewhere else they stay stored in the aircraft! o Good practice is to try downloading a mission from the aircraft to see what it has stored. You really don't want to send it off on a very long mission by mistake. Arm and Fly mission, going to Auto to initiate survey. Note buttons to go back to manual (A should also do this), Return Home, Land in Place, Pause options. Remember, Pause lets you continue a mission, while going to manual puts you in manual mode. Note that you'll probably need to go to manual to land on the pad accurately. Remember that the RedEdge camera is in the front of the aircraft, and the red lights are in the rear of the aircraft, but this is not as easy to see. o If about to land in the wrong place, climb back up to let it hover, then adjust. It's best not to run the power all the way down so you have sufficient power to make these adjustments. 17