Diggle Smalls. Nick Cox University of Florida, ECE IMDL EEL4665C Instructors: Dr. Arroyo, Dr. Schwartz TAs: Josh Weaver, Andy Gray, Devin Hughes

Transcription

1 1 Diggle Smalls Nick Cox University of Florida, ECE IMDL EEL4665C Instructors: Dr. Arroyo, Dr. Schwartz TAs: Josh Weaver, Andy Gray, Devin Hughes

2 2 Table of Contents 0. Abstract 3 1. Introduction Executive Summary Integrated System Software Mobile Platform 6 6. Actuation Sensors and Sensor Integration 7 8. Behaviors Experimental Layout and results Conclusion..10 References.. 11

3 3 0. Abstract The goal of the project was to create an autonomous robot that performs some useful task. Diggie smalls mainly functions as a front loader, often seen at construction sites digging up piles of dirt. However, it can be used to accomplish any task that requires moving bulk quantities of any relatively small item (toxic waste cleanup, waste management, etc.). The end result for this project was a robot that can autonomously navigate to a pile of aquarium gravel, scoop the gravel into its bin, locate a storage receptacle and finally dump the gravel into the storage receptacle. It uses high level concepts in computer vision to complete its tasks, which is all performed on board by a single board linux computer. 1. Introduction As time goes on, more and more every day tasks are becoming automated. Like Google s self driving car, people are beginning to trust robots to perform tasks that they would normally do themselves. Something as simple as moving dirt from one place to another is the perfect task for a robot, freeing up humans to do more productive things. Also in the case of toxic waste cleanup, no humans have to endanger themselves to get the job done. I wanted to make something that could be used in everyday life, albeit it would have to be on a much larger scale. I took IMDL so I could learn about robotics and de mystify the process, and I definitely accomplished that goal. 2. Executive Summary Diggie Smalls came out exactly as intended: it is a functioning skid steer loader powered by an embedded Linux computer which handles all decision making. The computer communicates with lower level boards that control motors and servos, and also interface with sensors. Infrared sensors are used for obstacle avoidance and general rangefinding, while a force sensitive resistor is used to detect weight in the bucket. It is powered by a three cell Lithium Polymer battery using voltage regulators to provide 5V for electronic boards. For demonstration purposes, a pile of aquarium gravel represents dirt or some other substance that needs to be moved. Diggie smalls pivots, while the onboard computer receives information from a video camera mounted on the front of the chassis. It uses color detection to find the pile of pink aquarium gravel, then navigates toward it. It drives into the pile until weight is detected on the force sensitive resistor, then it turns around to look for a receptacle (in this case, a Mountain Dew box). It finds the box using object recognition, navigates toward it, and then dumps the gravel into the box. After, it will repeat the process until the pile is gone. Admittedly, it isn t perfect; every once in a while it will miss the box or or overshoot it by a little bit. However, the majority of the time it works perfectly.

4 4 3. Integrated System 3.1. Pololu Mini Maestro Servo Controller This servo controller was chosen for its convenience. Its most obvious use is to control the two servos for my actuation system, but all 12 channels can also be used as analog inputs. These analog inputs already work on 5V, so there is no need to convert a 3.3V output to 5V. I have my infrared sensors and force sensitive resistors hooked up to pins on the Mini Maestro Atmel XMega128A4U development board The Xmega was chosen because of its familiarity. I developed it into a fully functional brushed motor controller. It allows the user to set duty cycle, direction, and other variables to control the motors. Any time it receives a set motor command through serial communication, it sets the duty cycle to a set point. Using timer interrupts, the motor controller accelerates (or decelerates) the motor speed to the set point at a user defined rate. Unfortunately with the nature of Diggie Smalls design, it was often more practical to instantaneously set duty cycle values so this feature was not used much.

5 ODROID U2/Robot Operating System The ODROID U2 is a powerful single board computer with an quad core 1.7GHz ARM processor. It runs Ubuntu Linaro 12.11, which is a specialized Ubuntu distro designed for ARM processors. This is the brains of the system, and it allows all decision making to be done on board without the need for a laptop and XBEE. Robot Operating system (ROS) is installed on the ODROID to provide hardware abstraction that allows for more elegant and robust robotic designs. The main advantage of ROS is that it allows programs to be split up into nodes. A node is a single program that runs connected to a network of other nodes. While a node is running, it can publish to and receive information from other running nodes through TCP/IP or UDP. It generally works in this fashion: 1. A node creates a topic, for example IR_sensors and creates an object that publishes to this topic. 2. A separate node creates a subscriber object and subscribes to the IR_sensors topic 3. The ROS Master will detect when there is a publisher and subscriber on the same topic. Once detected, the ROS Master creates a socket connection between the two nodes, allowing communication at runtime. 4. The publisher node sends a message, which is just a set of variable values of pre defined type. 5. The subscriber calls the functions spin() or spinonce(), and if there is a message available a callback is processed. Callback functions often update global variables, but they can be made to do anything a normal function can do. Some important things to note about this publisher/subscriber communication process is that it is completely anonymous. A node doesn t know (or care) about the origins of the messages it receives. This makes each node behave as a truly independent program, which increases fault tolerance, simplifies code, and allows for easier and more thorough testing. Although my project only includes the very basics of ROS, there are many other benefits like established code libraries and multi computer robotic systems. 4. Software Software on the ODROID is written entirely in C++, and software on the Xmega board is written

6 6 in C. The Xmega is running the motor controller programs described previously. The ODROID has 4 nodes running at all times 1. Main node: This node performs all the decision making. It checks sensor values against thresholds to decide what action to perform, then sends commands in the form of messages to the other nodes. 2. Maestro node: This node receives command messages from the Main node and relays the information serially to the Maestro servo controller. At the same time, it continuously reads data from infrared sensors and the force sensitive resistor and publishes their data on the topic sensor_data. 3. Xmega node: Similar to the Maestro node, the Xmega node receives command messages from the Main node and sends the information down to the ATxmega128A4U microcontroller which is used as a motor controller. 4. Camera node: The camera node listens to the topic state_data to see if the robot is in the pile phase or bin phase of execution. If it is in pile phase, meaning searching or navigating to the pile, it performs color detection. If it is in bin phase, it performs object recognition. More information on this can be found in the Sensors and Sensor Integration section of this report. All source code is available at 5. Mobile platform At first the plan was to create the mobile platform out of polycarbonate, but that proved to be too difficult. It can be cut easily, but with little mechanical experience I was limited to straight cuts with minimal fine detail. Luckily, balsa wood in conjunction with the T tech provided a much better alternative. All parts on the platform were designed using SolidWorks and cut out using the T tech. The most eye catching aspects of the platform are the large tracks used in lieu of wheels. The tracks and all accessories were purchased from LynxMotion.com. The design includes two elongated pill shaped brackets separated by 1.5 standoffs. Around each standoff is a nylon bushing that acts as a roller for the track. At each end there is a sprocket which fits the inner contours of the track. The sprocket in the back is attached directly to the motor using a clamping hub which is bolted to the sprocket and clamped to the tapered end of the shaft.

7 7 The sprocket on the other end rotates freely inside bearings that rest within the wooden brackets. Each track bracket is attached to the body of the robot using one motor mount and one L bracket. The design of the body itself is mostly arbitrary, except for holes cut out near the back that fit the motor and slits cut out of the front to allow room for the arms. Rounded edges in the front and back are purely for aesthetics. The body is made of two identical platforms separated by 2 standoffs, with extra space cut out of the top layer to fit the ODROID and allow easy battery removal. 6. Actuation Each track is turned by a 12V DC motor at a 30:1 gear ratio. The GHM 12, purchased from Lynxmotion, have a stall torque of 10.0 Kgcm, which is important for driving into piles of dirt and driving over obstacles. A non geared motor within my budget would not have provided enough torque to overcome the frictional force holding the tracks in place, especially when pivoting. The motors are powered by a dual H bridge motor driver (Pololu MC33926). For the arms that raise the bucket, a standard Hi tec servo was used (HS645MG) with 107oz in of torque at 4.8V. One bucket arm is attached to the servo horn, while the other arm rotates freely inside a bearing and serves only to constrain the system to stay level. It would have been better to go with a more powerful servo, but at the time of purchase my design required the use of a standard sized servo. That being said, there has never been a time when the servo failed to lift gravel. At the end of the arms is the aluminum bucket. A standard Hi Tec servo (HS485 HB) is attached to the bucket to provide scooping and dumping capabilities. 7. Sensors and Sensor Integration 7.1. Infrared sensors Sharp GP2Y0A21YK infrared sensors are used for obstacle avoidance. When driving, if reading on the left IR sensor reaches a threshold value the robot will pivot right. The robot continues reading the infrared data until another threshold is reached ( FAR threshold). Then the robot continues moving forward and returns to its normal function. The same applies to the right IR reaching threshold. When the middle IR is high, the robot stops and backs up until the middle IR reaches the FAR threshold. Then, it decides which direction to turn based on which IR reading is

8 8 higher (left or right). Then it pivots until the middle IR reaches the VERY FAR threshold, essentially meaning the object is nowhere in sight. the robot moves forward then returns to its original mission Force Sensitive Resistor (FSR) As the name suggests, this sensor responds to force. 5V is sent across the resistor, and the other terminal connects to a pulldown resistor and an analog input pin. This setup acts as a voltage divider circuit, with small FSR readings allowing a higher voltage on the input pin. When there is nothing resting on top of the resistor, it has a very high (~1MOhm) resistance. As force is applied, resistance drops. When resistance drops, the voltage drop between 5V and the input pin becomes smaller, thus a higher voltage is read on the input pin. The FSR is used to determine when there is gravel in Diggie Smalls bucket Camera All computer vision is done with a library called OpenCV, which is short for Open Source Computer Vision. All vision processing is done on the ODROID U2. The camera is used for two different purposes: color tracking and object recognition. The camera used was a Creative Live! Chat webcam Color Tracking Color tracking is a very simple process using OpenCV. The first step is to capture an image from a camera and convert it from BGR to HSV color space. After that, it is helpful to apply a gaussian blur (or any blur type) to reduce noise. Then, using library functions, filter out the appropriate color by only accepting colors in a certain HSV range. All color within the range in the image will be converted to white, and everything else will be black. Once again, use library functions to calculate the moments of the white space in the frame (moments have the exact same definition as they would in an introductory physics class, with white analogous to mass ). Dividing the x moment by the area gives the x position of the detected color blob. In the case of Diggie Smalls, once the x coordinate of the pink blob is centered the robot will move toward it. It also uses the area of the blob to roughly determine its distance. On the ODROID, this can all be easily done at above 15 frames per second Object recognition Diggie Smalls uses a Keypoint/Descriptor based approach to object recognition. Specifically, the SURF extractor and matcher are used. Before any image processing, the image must be converted to grayscale. Once a matrix of grayscale values is obtained, areas in the matrix with high curvature are chosen as key points. In this context, high curvature is very similar to saying high contrast. Key points are assigned vector quantities based on the nature of the image (using integral image techniques), and a matcher compares the key points of a frame to the reference

9 9 image (the object to be detected). If it finds a match, a distance value is given that represents how strong the match is. High distance matches are discarded and low distance matches are kept. Finally, homography is used to transform the reference object into the plane of the detected object. SURF is very powerful because it is rotation and scale invariant, and it is also much more repeatable and accurate compared to other descriptors and matchers. Unfortunately, it is also much slower than some other object recognition algorithms. Even so, I managed to run object detection at around 6 8fps on the ODROID which is satisfactory. However, the SURF algorithm is very susceptible to motion blur, which means the robot has to move in jerky motions to accurately find the bin. Example of SURF detecting a pop tart box 8. Behaviors Diggie smalls is a state machine. In the FIND_PILE state, it rotates around looking for the color pink. When pink is detected and its area is above a certain threshold (to deter false positives from noise), it will drive toward the pile while maintaining the x coordinate near the center. During this time, it will avoid obstacles as described in the Sensors and Sensor Integration section. Once the area of the pile reaches a larger threshold, the robot will dig its bucket into the ground and drive toward the pile until weight is detected on the FSR. At this moment the robot lifts the gravel, backs up for 1.5 seconds and begins to pivot in a jerking motion to the left. It will pivot until it sees the Mountain Dew Box used as a storage receptacle. Once located and centered, it drives toward the box until the infrared sensor reaches a threshold which means the bin is close enough to dump the load. Then the robot tilts the bucket downward to drop the gravel, backs up, and repeats the process from the first state. 9. Experimental Layout and Results Infrared sensors are loosely calibrated. Since Diggie Smalls doesn t need to know any exact distances, all threshold values were found experimentally while mounted on the platform. I just held out my hand at a suitable triggering distance, noted the value, and recorded that value in a threshold definition. The same applies to the FSR; since it is not very sensitive to the low weight of the gravel, I just found a reliable cutoff value and stuck to it. I noticed the FSR can be very noisy, so I implemented a running average of its readings in the main program. I did the same for the infrared sensors. I created a program called CalibrateCam which allows me to adjust HSV

10 10 thresholding values while the video continuously updates, so it is very easy to calibrate my color detection code on the fly. Most of my sensor issues came from running object recognition. The first major issue was the blur, which I already mentioned. Secondly, there are a lot of false positives. To get rid of false positives, I only publish data that adheres to very strict rules. Since the SURF algorithm calculates the corners of the image, all filtering must be done by examining the corners. In order for the data to be deemed good, the matched object must be rectangular, not too elongated, and it must have reasonable coordinates for all corners of the image. Furthermore, the area cannot be too large or too small. However, just filtering out certain images is not enough. For example, one condition is that 9/5 > (y2 y3)/(y0 y1) > 5/9, where y2 and y3 represent the right corners y coordinates and y0 and y1 represent the left corners. This had to be included due to a peculiarity in the SURF algorithm that causes some matches to be represented highly off axis. In this situation, one side of the bin placed around the detected object becomes very long and it misrepresents the x coordinate of the bin. This leads to false positives and false negatives, which are both frustrating to deal with. On demo day, 9/5 and 5/9 threshold values were too strict, because sometimes it would throw out data that could have been used to detect the box. Diggie Smalls would hit its target, but overshoot the bin slightly when aligning. I decided to loosen the restrictions on the ratio, but apply a running average to the data to filter out some of the stray elongated matches. 10. Conclusion IMDL is not easy and it is definitely not cheap. However, I learned more in this class than I have in any other class I have taken so far. I learned so much about embedded computers, linux, computer vision, C++, mechanical design and more. I learned so much that, if I were to take this class again, I could build an entire robot in the span of a couple weeks. It s funny how after spending so much time programming a robot, you can think what it is thinking and see what it is seeing. When there is a glitch in the code and my robot isn t moving, I know instantly what I left out because I can follow the code in my head as Diggie Smalls runs. One of the major things I learned is that no design is going to be perfect. Sometimes rather than coming up with the perfect design in theory, it is better to create your design then tailor it to your needs. For example, I spent so long designing my actuation system in my head, but in the end I scrapped it and just kind of winged it. It seemed to work out for the best, because I don t think I could have made this robot any better given the time frame. I would still like to make some changes to Diggie Smalls. I hoped to add machine learning concepts that would speed up the process of locating objectives after the first cycle, but I didn t have time. Also, I would still like to set my robot up to have a remote control option.

11 11 References Information about ROS: Information about ODROID: Information about SURF [pdf]: Pololu electronics and information: Lynxmotion products and information: