Autonomous Surface Vehicle EE424 Senior Design Group #8 Date Submitted: March 19, 2013 Faculty Technical Advisor: Professor Yan Meng Yan.meng@stevens.edu Group Members: Alex Cihanowyz Francis Garcia Charles Steiner Rover/Turret Development Sensor Development System Integration We pledge our Honor that we have abided by the Stevens Honor System
Table of Contents Abstract 1 Acknowledgement 1 Project Progress Introduction 2 Updated Design Requirements and Competition Approaches 2 Task 1-4 Navigation and Localization Tests 2 Task 5 Catch the Ball 3 Task 6 Sneaky Sprinkler 4 Task 7 Rock, Paper, Scissors, Lizard, Spock 4 Task 8 Capture the Flag 6 Task 9 Shoot through the Hoops 6 Task 10 Return to Dock 7 Component Updates 7 Software Progress 10 Design Risks 10 Conclusion 10 References 11 Appendices Updated Task Layout A Channel Navigation Flowchart B Rock Paper Scissors Lizard Spock Flowchart C Shoot Through the Hoops Flowchart D OpenCV Color Differentiation E Financial Budget F Gantt Chart G
Abstract There are many problems in the world that are too great in magnitude for humans to solve. One recent example of this is the Deepwater Horizon oil spill in 2010, where an explosion caused millions of barrels of oil to leak into the ocean through a hole in a pipe underwater. This caused a massive oil spill that required more manpower and time to clean as time passed. This need for a solution caused several research groups to develop robotic platforms to help clean the oil spill. Most successful of these is a robot developed by MIT known as the SeaSwarm, which autonomously navigates through the oil slick and absorbs the oil with a nano-fiber net. Other recent problems include Hurricane Sandy, particularly in Hoboken and in the New Jersey shore area where towns were flooded and required high water vehicles or even boats to traverse through the flooded areas. With autonomous boats that can navigate to a specific point in an area, it is possible for emergency responders to deliver a payload such as food, water or medicine while rescuers can prioritize evacuating people rather than food delivery. In order to foster the design process and develop skills in systems engineering by completing missions with autonomous robotic boats, the Association for Unmanned Vehicle Systems International (AUVSI) developed the RoboBoat Competition. This competition is comprised of ten tasks that simulate problems that an autonomous boat may encounter, from channel navigation to receiving a payload from a dock. For the 3 rd year in a row, Stevens Institute of Technology is submitting an entry to this competition in hopes to win the competition and allow its students to develop skills to design a solution to a series of problems. Unique to this year s entry is the inclusion of a team of electrical engineers, who will be tasked with the programming and sensor integration of the boat. They will use sensors such as a Kinect sensing device, high definition cameras, sonar sensors, and infrared sensors. After completion of the project, the team will have an autonomous surface vehicle (ASV) that utilizes all these sensors to complete the tasks set out by the AUVSI. Acknowledgement The electrical team would like to acknowledge Professor Yan Meng for her advice and guidance throughout both semesters, Professor Fisher from the mechanical engineering department for allowing a team of electrical engineers to join in on the AUVSI Robo-boat team, and the electrical engineering department for funding the project. The team as a whole would also like to thank Davidson Lab for allowing the use of their wave tanks for testing. Page 1
Introduction The purpose of this project is to design and create an autonomous boat to compete in the AUSVI Roboboat competition. This competition is an obstacle course designed to be completed by using multiple sensors on the ASV. It is comprised of several challenges that must be completed sequentially. This is the third year that Stevens is submitting an entry for this competition. As of now the boat that will be used in the competition is being re-constructed with lighter materials and a majority of the sensors that will be used in to complete the tasks have been purchased by previous teams. Using these components as wells as concepts learned in class at Stevens Institute of Technology, the electrical team will design the ASV to complete the competition course successfully and attempt to achieve the highest score that it can. Testing and final implementation for the sensors is currently being performed by the team, and the final program is being developed using algorithms from the previous LabView program and updated algorithms developed by the electrical team. Once all the sensors are configured and completed, testing will be done in the Davidson Lab on the Stevens Institute of Technology s campus prior to the competition. Updated Design Requirements and Approaches In the beginning of February, the AUVSI released the preliminary rules for the 2013 competition. There were no significant changes to the design requirements for the ASV. However, new challenge stations were added to the competition and old challenge stations were combined. One major change from the 2012 competition is that the Channel Navigation task is now optional, but may be completed to earn score multipliers based on performance. Two new stations were added, the Capture the Flag task and the Shoot through the Hoop task. In the Capture the Flag task, the ASV must retrieve a flag from a floating or slow moving remote controlled boat. The Shoot through the Hoop task involves the ASV shooting a foam missile through large plastic hoops to earn points. The same sensor suite used in the 2012 competition is able to complete the newer tasks, but the new tasks will also require new actuators, which are being developed by the mechanical team members. First Set: Task 1 4: Navigation and Localization Tests Updated Contest Tasks There have been no changes made to the format of the navigation and localization tasks. The significant change made for the 2013 Competition is that the Channel Navigation task is now entirely optional. This may be because of last year s competition, where only two teams were able to complete the Channel Navigation successfully. For this year s competition, a score multiplier is awarded to a team upon completion of part or the entire Channel Navigation task. If a vehicle navigates at least one pair of Page 2
buoys, the team s challenge station points are doubled. Completion of half of the course and avoiding one yellow buoy awards a triple multiplier, while successful navigation of the entire channel will result in a quadruple multiplier. Only the highest multiplier earned will be applied to the team s final score. To complete this task, the group will use the same method described in the Final Paper for EE423 submitted on December 11, 2012, as well as developed in the WeBots simulation. The 2012 ASV did not complete the Channel Navigation, so the 2013 team spent time last semester to develop a way to successfully navigate the channel. Several methods will be implemented and tested during the testing phase to determine the most effective solution. The current solution being developed and tested is navigation with the Kinect sensor and USB webcam. The ASV will use the Kinect s depth sensor to find the closest set of buoys, use the webcam to determine the buoy colors, and then navigate through the barriers by keeping the center of the processed image between the closest obstacles. This will be completed with an OpenCV program that captures an image from a USB webcam, processes the image to detect the closest red and green buoys, and logs the approximate centers of each buoy. The program will then send a serial command to the Arduino communicating with the motor controller to control the boat so that it navigates between the buoys. A flowchart of this process is available in Appendix B. Second Set Task 5 Catch the Ball The Catch the Ball task is simply the 2012 Poker Chip task renamed. The format of the task remains the same the ASV will locate a landing zone and dock with it. Once docked, it will deploy a mini-rover to navigate through the landing zone and capture a payload, which will be a tennis ball. The rover will then navigate back to the ASV and deposit the tennis ball. Upon completion of the task, the ASV will then un-dock and proceed to the next task. One change made to this task is that after the competition, the team has to submit the video used by the rover to navigate to the competition officials. Figure 1: Catch the Ball Page 3
To complete this task, the rover will use a camera and send an image to the boat through Xbee Wireless USB to perform image processing. Just like the navigation task, the robot will detect the circular tennis ball with an OpenCV program, and then control the rover to capture the ball. To help in navigating back to the ASV, rotary encoders may be mounted on the rover. A secondary GPS unit for the rover may also be purchased if the team decides it is necessary. This will also provide extra functionality in that a second GPS can be responsible for tracking the boat s coordinates if the main GPS loses connection with satellites or if it stops working. The original strategy may also work, by marking the boat with a unique color so that the rover can then simply find the color on the ASV and navigate towards it. Task 6 Sneaky Sprinkler The Sneaky Sprinkler task is the 2012 Jackpot task. There have been no changes made to this task from the 2012 competition other than the name. One solution to the task may be to mount a submersible webcam under the boat and use an OpenCV function to detect where the white buoy is. One problem with this is that the water may be too murky for the camera to see anything beyond a couple of feet. Task 7- Rock Paper Scissors Lizard Spock Figure 2: Sneaky Sprinkler The Rock Paper Scissors Lizard Spock task is a combination of the 2012 Cheater s Hand and Hot Suit tasks. The concept was invented by Sam Kass, a student at Carnegie Mellon University and popularized by the TV show The Big Bang Theory. By adding Lizard and Spock, the game is theoretically made more fair by provide alternative outcomes. A diagram depicting the different outcomes is shown in Figure 3. Page 4
Figure 3: Rock Paper Scissors Lizard Spock Outcomes In the 2013 competition challenge, there are four metal signs like in the 2012 Cheater s Hand task. Each sign will have one of the outcomes of Rock Paper Scissors Lizard Spock. The ASV will be tasked with detecting the hot target, as in the 2012 Hot Suit task, and then reporting not only its GPS position but also the outcome that would beat it. Figure 4: Rock Paper Scissors Lizard Spock Challenge Station In the diagrams provided in the rules, each symbol will be outlined by a colored circle. The infrared sensor will be mounted on a turret that will allow it to scan each sign. There will also be a USB webcam mounted on this turret to make it easier for the robot to complete this task. The camera will take an image of the sign and then send commands through the program to the Arduino controlling the turret to center the symbol in the image. The program will then process the image, determine what color and what symbol is located on the sign, and then use the IR sensor to determine the temperature of the sign. After the ASV scans each sign, it will then determine which sign is hot, record its GPS position, and then report both the GPS position and the symbol that would beat it to the competition organizers. A full program flowchart is available in Appendix C. Page 5
New Tasks Task 8 Capture the Flag The first new task, Capture the Flag consists of a floating or slow moving boat around the given GPS coordinate. This boat will have a small blue flag located at the stern. The ASV will have to approach the boat and steal the flag from the boat Figure 5: Capture the Flag The mechanical team is now developing a grabber arm that will be used to steal the flag. To better develop a solution to this task, the electrical team will work on a new simulation. The initial strategy will be to have the ASV travel to the challenge station s GPS coordinates, and then rotate until it detects the blue flag. There is a possibility of it recognizing the blue buoy from the navigation challenge, so it will then also perform a check to see if the object is circular. If it is not, then the ASV will have successfully located the boat with the flag and can then approach it. Task 9 Shoot through the Hoops The second new task, Shoot through the Hoops, involves shooting a foam arrow or missile through 24 inch plastic colored hoops that will be mounted on poles close to the shore. The launcher for this task will be mounted to the turret mentioned in the Task 7 requirements. Like this task, the turret will scan the image until it is centered on a hoop. It will then launch a Nerf missile to try to shoot through the hoop. A flowchart of this process can be found in Appendix D. Figure 6: Shoot through the Hoops Page 6
The mechanical team is developing the launcher by reverse engineering a toy Nerf gun which uses flywheels to pull a missile and launch it at high speed. An initial thought would be to modify the Nerf dart by gluing a ball bearing on it so that it isn t affected by the wind as much, but the rules state that the projectile must be made entirely out of foam. This may pose a challenge to not only the Stevens team, but also all teams, as high wind conditions may make this challenge station require lucky timing or may even render it impossible to complete. Task 10 Return to Dock The final task is the same Return to Dock task as the 2012 Competition. If the ASV has attempted at least three challenge stations, the boat can then attempt to return to dock through the navigation channel. The same procedure the team will use to complete the Navigation Channel task will be used to complete this task. Microsoft HD Web Camera 6000 Component Updates Since the updated rules were released, the team focused more on GPS navigation to guarantee that the ASV will at least be able to navigate to each of the challenge stations. The group is now refining the circle detection code provided in the EE423 Final Report. Improvements that will be made to the program include a blurring function to remove noise from the processed images, and a dilating function to enlarge the thresholded image. The group will also determine the necessary HSV values to isolate certain colors, such as red, green, and blue. Images showing the OpenCV program differentiating between different colors are available in Appendix E. Xbox Kinect Sensor The group is currently researching the most effective way to use the Kinect. After the preliminary rules were released, there has been less focus on the Kinect and more focus on the GPS. Now that the GPS has been fully developed, the group is now returning to development of the USB webcams and the Kinect. GlobalSat BU-353 GPS Sensor The GPS sensor has been tested multiple times near the Stevens campus. These initial tests gave GPS coordinates of a park about half a mile east and in South Jersey. After further research, the group discovered that based on magnetic deviation, the GPS will provide technically correct coordinates that do not correspond with coordinates used on mapping services like Google maps. After refining the program, readings were then produced from the GPS that were accurate to within twenty feet. However, this accuracy was only achieved after taking 4000 samples, which took up to half an hour to complete. Page 7
In order to improve the effectiveness of the GPS navigation, the team will also use the Ocean Server compass kit to provide heading to the robot. It is possible to calculate bearing with only GPS coordinates but this will provide another set of data for the robot to use. Raytek CM IR Sensor Figure 7: Predicted GPS location before magnetic deviation correction and after The IR sensor is currently being re-worked; the previous mechanical team extended the wire from the IR sensor, but used illogical color choices, such as a white and green wire extending a black wire and a blue wire extending a purple wire. These splices are being remade so that the extension wire will be the same color as the original wire. After that, the team will develop Arduino code to take the input from the sensor and convert it to a temperature. Ocean Server OS 5000 US Kit Unfortunately, the Ocean Server Kit is still in the possession of one of the 2012 team members. This fact was not brought to either team s knowledge until quite recently. Both the electrical and mechanical teams are working to get this sensor back. Luckily, the particular team member who has the sensor is still in New Jersey and has prior friendships with members of both teams, so this component should prove simple to reclaim. XBee Transmitter and Receiver The Xbee units only arrived just before spring break. Alex and Francis will both be doing research to determine how to communicate through Arduino to the main computer and back. Page 8
Humminbird Fish Finder 570 DI The Humminbird Fish Finder uses a proprietary cable to log data to a computer. Both teams are either trying to find the cable or a replacement, or contact the manufacturer to determine a better way to get data. If neither is possible, the electrical team will either look into finding a submersible USB camera or will use a regular USB camera to get images from the fish finder display. Both cases will require OpenCV functions to detect the round object, either directly from the submersible USB camera or from an image on the fish finder. Seabotix BTD-150 Thrusters and Sabertooth Motor Controllers Both thrusters have been tested and confirmed to work properly. This is important because these thrusters are among the most expensive components on the both, at approximately $1000 each. The mechanical team will have a frame built within the next week so that testing of motor controls may begin. In the meantime, the team will be using an older ASV frame and testing its control algorithms with that. A video of the thruster testing is available at http://stevens.edu/asv/gallery.html. Arduino Uno The group is able to get the computer communicating with an Arduino through serial. This is an important step because the Arduinos will control the IR sensor, the turret, the rover, the thrusters, and the launcher. The team is now developing and finalizing the programs for each Arduino. Figure 8: The Arduino sending 45 to Serial and the C++ program outputting the value received from the Arduino Page 9
Software Progress A GUI will be developed for testing and for the competition. This will display output of the GPS, output of the IR sensor, as well as the original and processed images for each camera. If there is time after testing, the group will attempt to use the Google Maps API to map where the ASV is in the world. Additionally, the main computer on the ASV will be updated with the tools the team is using to develop now, such as Visual Studio 2010 and OpenCV. Design Risks There are several design risks that will affect the ASV with regards to the EE424 timeline. There has not been time to test the ASV as a whole as the mechanical team only recently ordered the parts for the new frame and the electrical team only recently acquired the older frame. Additionally, the team has not been granted access to the Davidson lab to test. With regards to the EE424 timeline, the ideal plan is to have the boat systems working as intended by the Senior Design expo day, or at the very least have all of the components functioning. Outside of that, there will still be two whole months to test and complete the ASV afterwards before the competition. The rules released on the Roboboat website are listed as preliminary, so there may be further changes or tasks added to the rules that the team will have to develop solutions to. The mechanical team is still designing several mechanisms, such as the grabber mechanism for the Capture the Flag task and a turret for the Shoot through the Hoops task. Another design risk is present in that each member of the electrical team has a full time job lined up after graduation. This may necessitate training of replacements, should none of the electrical team be able to take time off for the competition and the testing period. Conclusion With all things considered, the group is on track to have a completed and functional ASV for Senior Design expo day. At the time of writing of this report, a majority of the parts are becoming functional and it is time to consolidate all of them onto the ASV. Now that the 2013 Competition rules have been released, both teams are able to develop solutions to the new challenges presented. Some of the older challenge stations have even been made easier with the newer rule set. While the different functions, such as GPS readout, image processing, and turret control, will be completed by the expo day, the ASV will not be fully optimized for the competition at this time. This optimization will unfortunately come in the two months between the expo and the competition during the testing phase for both groups. However, due to the progress that both teams are making, the ASV team as a whole is on track to having a functional, competition ready ASV by the time of the competition. Page 10
References 2013 AUVSI RoboBoat Competition Rules http://www.auvsifoundation.org/foundation/competitions/roboboat/ 2012 Stevens ASV Journal Paper http://stevens.edu/asv Open CV http://opencv.willowgarage.com/wiki/ Webots Components Tutorial: Real-Time Object Tracking Using OpenCV (http://www.youtube.com/watch?v=bsefrprqz2a) Introduction to Autonomous Mobile Robots Siegwart and Nourbakhsh http://www.cyberbotics.com/ http://www.seabotix.com/products/pdf_files/btd150_data_sheet.pdf http://www.dimensionengineering.com/products/sabertooth2x25 http://www.usglobalsat.com/store/download/62/bu353_ds_ug.pdf GPS Tutorial (http://www.instructables.com/id/gps-for-lazy-old-geeks/) http://www.robotshop.com/oceanserver-os5000-us-compass.html http://www.arduino.cc/en/main/arduinoboarduno http://www.raytek.com/raytek/enr0/productsandaccessories/infraredpointsensors/compactseries/raytekcm/ Page 11
Appendix A: Updated Task Layout Page 12
Appendix B: Channel Navigation Flowchart Begin Channel Navigation Capture image Image processing to isolate red buoy Get coordinates of red buoy center Image processing to isolate green buoy Get coordinates of green buoy center Image processing to isolate yellow buoy Get coordinates of yellow buoy center Rotate boat until center of image is between nearest obstacles Use thrusters to move through nearest obstacles Capture image Image processing to isolate blue buoy Is the blue buoy present? No Yes Channel navigation complete Page 13
Appendix C: Rock Paper Scissors Lizard Spock Flowchart Arrive at Challenge Station Capture Image Yes Is symbol centered in image? Perform image processing to determine symbol No Scan with turret Determine temperature with the IR sensor Record servo position Rotate turret to next sign No Have all four signs been scanned? Yes Determine the hottest sign Return servo to position corresponding with hottest sign Actuate both thrusters and servo until ASV is near the sign Transmit GPS coordinates and winning symbol Challenge station complete Page 14
Appendix D: Shoot Through the Hoops Flowchart Arrive at Challenge Station Capture Image Perform image processing to determine hoop color Yes Is hoop centered in image? No Shoot two projectiles Scan with turret Rotate turret to next hoop No Have all hoops been shot at? Yes Challenge station complete Page 15
Appendix E: Open CV Color Differentiation Yellow Red Blue Pink Green Orange Page 16