Mindstorms RoboCup Player Carl Beattie BSc Computer Science Session (e.g., 2006/2007)

Transcription

1 Mindstorms RoboCup Player Carl Beattie BSc Computer Science Session (e.g., 2006/2007) The candidate confirms that the work submitted is their own and the appropriate credit has been given where reference has been made to the work of others. I understand that failure to attribute material which is obtained from another source may be considered as plagiarism. (Signature of student)

2 Summary This report documents the design, construction and implementation of a robot that can play a simple version of the RoboCup soccer game, that is a version of football that is specifically designed for robots to compete in. The official version of the tournament uses much more highly engineered solutions to the problem. There is however a Lego Mindstorms league that uses the simple light sensors to navigate. In order to provide more of a challenge this project was undertaken with the aims of using an intelligent vision system rather than the simple light sensors. The developed system evolved from my initial designs as their advantages and limitations became apparent; a clear example was the initial idea to use a top down camera system that could observe the whole playing field. This proved to be unfeasible due to the height required to view the whole area and the lack of detail at a sufficiently high vantage. The finished robot has a point of view vision system provided by a webcam and a pair of arms similar to mandibles on the front used for capturing the ball. The image processing for the vision system is carried out by a laptop that communicates with the robot using the infra red USB tower that is part of the Mindstorms kit. i

3 Acknowledgements I would like to thank fully: Tony Cohn- For his support and ideas during the project. Marc De Kamps- For the feedback on my mid-project report. Drew - For providing excellent grammatical support. Clare - Intensive proof reading skills. Bagnall et al- For developing such useful tools. Leigh- For putting up with all my late nights. ii

4 Contents 1 Introduction Aim Objectives Minimum Requirements Detect the ball s location in the environment The robot should be able to move to a desired location The robot should be able to direct the ball along a desired heading Deliverables RoboCup Relevence Project Components Hardware The Lego Mindstorms Kit The RCX Brick The Sensors The Motors Locomotion Acquiring the Ball Vision System Motion Detection Object Detection using Shapes Pixel Values iii

5 2.3.5 Holding the Ball Software LeJOS BrickOS NQC LegoLog Navigation Final Solution System Evaluation Testing Test Methods Test Results Evaluation Comparing system to initial objectives Problems faced along the way Overall Possible Extenstions Full Wireless Functionality Self Localisation Adding more Players Other Uses Conclusion 25 Bibliography 27 iv

6 Chapter 1 Introduction 1.1 The aims of the project The aim of the project is to use the Lego Mindstorms platform as a base for the construction of a robot that would, using some programmed intelligence, play a rudimentary version of football. This includes the design and construction of the robot, and the software that will govern its actions. 1.2 The objectives of the project The project has a few objectives, ranging from personal to deliverable goals: To teach me the basics of producing a fully functional product from design to implementation, including testing and evaluation. This is also an exercise in timekeeping and project management. Though not a software development project in the traditional sense, there is a software product and a development process that needs to be followed. To actually produce the finished robot is the main objective of the project, however I would also 1

7 like to see that the robot is capable, having enough intelligence to behave as desired. I feel that the results of this project could be something I will really be proud of. My overally objective is to develop upon skills I have obtained throughout my degree and can take forward into my professional life 1.3 The Minimum Requirements The base requirements of the project are threefold and are described in the following section Detect the ball s location in the environment This is both the most vital and the most complicated requirement to achieve; for this to be implemented the robot must first be able to sense the ball. For that it needs sensors and the ability to interpretate the input from them to make an intelligent decision as to where the ball is located. Secondly the ball will not always be in view of the sensors (unless a 360 degree field of view is implemented) and so the robot must be able to move in order to scan its location The robot should be able to move to a desired location This is a much simpler requirement which involves two parts; the knowledge of which direction to move in and the means to get there. After fulfilling the above requirement this section is mostly covered. The next idea is to simply be able to move forward so it can intercept the ball, here it will make the decision of locomotion The robot should be able to direct the ball along a desired heading This can be carried out in two different ways; one way would be to hold on to the ball and then move the robot to the desired location. Another is to hold the ball and turn to the correct heading then somehow kick the ball along that path. 2

8 1.4 The Deliverables There are three deliverables associated with the project. Most important of these is the project report. This is the main method for evaluating the success of the project as a whole and this will require a significant portion of time to be devoted to it. The robot should be able to carry out the demands of the project, e.g. it should be physically capable of moving, sensing the ball in some way, and knowing when it is in possession of the ball. The final deliverable is the code which is to be executed on the robot. This should simply be able to do the objectives outlined in the minimum requirements. 1.5 Just what is RoboCup? RoboCup is an international project with the goal of improving the fields of AI and robotics. It hopes to do this by providing both an incentive and common ground for developers and researchers to develop new and better robots. The idea is that the competition will drive people to come up with new solutions to old problems in the hopes of beating the other team. To this end the RoboCup Federation chose a game of football as the testing ground. The foundation have ambitious goals, the main one being By the year 2050, develop a team of fully autonomous humanoid robots that can win against the human world soccer champion team. [3] The RoboCup competition has expanded since then and now covers some other competitions as well, these include RoboCup Rescue and RoboCup Junior. RoboCup Rescue is a competition that aims to challenge the best rescue drones for the emergency services in hazardous environments. RoboCup Junior is an introductory level of the RoboCup league and is open to children to help them interested in robotics and programming. The junior version of the RoboCup league uses Lego Mindstorms rather than the complex and expensive robots, to provide tools which are not intimidating to the children who take part. 3

9 The football is heavily simplified with robots either undertaking 1v1 or 2v2 games.robots take part on a pitch that has been shaded to allow the light sensors to provide the means of navigation on the pitch. An infra red emitting ball is used to allow the robots to avoid using complex visual recognition systems. 1.6 Relevence to a Computer Science degree While studying for my computer science degree I have found myself covering a very wide range of subject areas. These included database design, algorithms, maths, theory of computation, interface design, as well as a large section of artificial intelligence. The artificial intelligence (AI) modules have covered a wide range of the subject including computer vision and biologically inspired AI alongside the traditional AI. This project incorporates many of these areas, notably the artificial intelligence, software development and the problem solving of the computational theory and algorithms work I have done. The construction of the robot had questionable relevance, however it does provide an integral part of the solution to the problem. It also provided an important incite to all aspects of the robotics and artificial intelligence sectors. and the impact of one on the other, problems that can arise and their solutions. 4

10 Chapter 2 Project Components 2.1 Hardware The project mainly uses the Lego Mindstorms kit but also contains some other notable items. The main image processing is provided by my laptop which is a Toshiba Tecra A4. The processed images are taken from a Logitech QuickCam 3000 Pro which is attached to the robot and passes the information to the laptop via a USB cable. Finally we have the ball which is the same as the balls used in the real RoboCup soccer league[12]. This ball contains IR emitting diodes which are detectable from the light sensor that is available in the Mindstorms kit and webcams. 2.2 The Lego Mindstorms Kit The Lego Mindstorms kit is a platform developed for both hobbyists and for education. This is apparent with Lego s support for open source firmware and development tools [9] It is mainly marketed towards school children with the default development package being a drag and drop system for commands that while being surprisingly powerful would lack the capability of the various third party offerings. The set contains around 700 pieces the heart of it being the RCX brick, which is the processing unit for 5

11 the system. Along with that there are 2 motors, 2 touch sensors, a light sensor and an IR communication tower for loading programmes onto the RCX brick. The rest of the Lego is used as a frame work for the other components and as such are mostly pieces from Lego Technique[8]. The main advantages of using the Mindstorms kit is the low cost and ease with which functional robots and vehicles can be built. The robot can be taken apart and rebuilt any number of ways using the same parts, this reduces the need for custom engineered parts allowing me to concentrate on the computing side of the project without getting bogged down in the engineering The RCX Brick The powerhouse for the Mindstorms system is the Robotic Command explorer, or the RCX often simply referred to as the Brick. Containing 32Kb of external RAM, 16Kb of space for the firmware and running at 16MHz,[10] the brick is not the most powerful of computing devices. It is however sufficient, for most of the demands placed upon it for this level of AI applications. The brick has 3 input ports used for taking input from the sensors and 3 output ports used for the control of the motors. These ports can also be overloaded with two inputs plugged into a single port, and threshold appropriately. For example, if you had a sensor that returned only 0% or 100%, and a light sensor that returns values from 0 to 100, it is unlikely that the light sensor would receive nothing or be fully saturated and so you threshold out the 100 and the 0 response from one sensor and the rest is the light sensor. It isn t perfect but it can be done. There is an IR receiver which is used as the communication point, this is how the programmes are loaded onto the brick and can also be used as a receiver in a remote control application that takes instructions from a computer. This means I was able to use a computer to perform all the image processing that would be required to find the ball in a scene using the webcam, and transmit the instructions via the IR link to the brick. Using the system in this way allows the processing power of the brick to be used for simpler task, such as reacting to the touch sensor to close the gripping arms without relying on the slow IR link to react to simple stimulus. 6

12 Lastly there is an LCD screen that can display text and can be used to display the running status of the program. The firmware that runs the code is contained on the brick and can easily be flashed and replaced with third party firmware that can execute code written in other languages such as C, Java or Prolog The Sensors There are four sensors provided by the Mindstorms system. There are rotational sensors that are able to detect the amount one of the axles have been turned, which is useful for guaging the distance a robot has travelled using a dead reckoning system. It detects rotation on 16th s of a rotation, allowing for a reasonable amount of accuracy. Touch Sensors can be used to detect collisions or impacts, and have a boolean output registering either 0% or 100% voltage based on a threshold. There is a Light Sensor that returns the brightness of the area directly ahead of it. Returning a value between 0% and 100% and is able to detect the infra red light emitted from the RoboCup soccer ball. The interesting thing is that it is entirely possible to produce your own sensors which can be passive or powered and can be used to do new things like detect heat, magnetic sources or many other stimuli[13] [7] The Motors The Motors are the main output from the system. They provide all the functionality that is required for this project. The main problem with the motors is that (probably due to age and use) there are mechanical differences between the motors that results in different levels of torque. This makes dead reckoning hard to manage without using the rotational sensors to confirm the exact movements made. Consequently, going forward in a straight line becomes a much more computationally intensive process. 7

13 2.2.4 Locomotion There are a number of different methods to move the robot, the biggest decision is to either choose wheels or tracks. Tracks have inherent advantages such as a zero turning circle and the ability to traverse uneven terrain. While the zero turning circle would be extremely useful for this project, the extra traction could have been a problem since the robot was going to be quite heavy. This is because it had to carry the webcam and the RCX brick (including it s 6 batteries). The tracks also have a limited set of configurations, there is a limit on the length of the tracks and also the design of the chassis. There are multiple sets of wheels provided by the set, from large spoked wheels to small dense ones. The large wheels prove to be ideal; since the robot would have always been acting on a flat surface, there was very little need for large amounts of torque and at least a fair amount of speed is a requirement of playing football. Next came the issue of the wheel placement, a full steering system would be inappropriate as the robot needed to be agile. Using a steering system similar to a car would have resulted in an extended turning circle as well as the need for a separate motor to turn the wheels. This would not have contributed to the overall power output, since the vehicle was going to be heavy this would have been a problem. There is a simple solution to this problem which uses one motor to drive the robot, which either powers one or both of the wheels. If one is used then the robot moves in a circle around the stationary wheel. If both wheels are powered then the robot will move forward. This however is very limited and the robot would have quite a large turning circle as well as a low power output using only one motor. A better system is to have two wheels on a central (split) axle that controls both the steering and the motion. Using the wheels in this way you can have a zero turning circle, that is basically the space required to turn around. This can be achieved by turning the wheels in opposite directions so that the robot turns on the spot which is useful for scanning the environment and avoiding complicated navigational issues in the coding. 8

14 There is a mechanical system known as a dual differential that could have been useful here. A dual differential is a system of cogs and axles that allow one motor to power two wheels in the same direction. Figure 2.1: The Dual Differential Gear System [19] A second motor is attached and when powered will turn the wheels in opposite directions. The combination of the two will allow you to vary the speed of each of the wheels which allows you to ignore differences in the speeds and torque of the motors. [5] The disadvantages of this system are that there is only one motor to power forward motion and there is a significant amount of stress and friction on the axles due to the overall weight of the robot. Despite the usefulness of the dual differential system, it had to be abandoned as the motors were simply not powerful enough. Instead a less complicated option was used with one motor driving each wheel.(see fig 5.2) 2.3 Acquiring the Ball There are two issues to consider within this section of the project. Firstly, the robot needs to be able to find the ball and get close to it. Then the robot must be able to guide the ball, that is it must be able to turn while staying in contact with the ball and be in a position to move to a new location without the ball rolling off. 9

15 2.3.1 Vision System It was clear that the robot would need some method of seeing the ball. In the official league the robots use the light sensors provided by the Mindstorms kit to locate the ball, which contains IR diodes the light sensors are capable of detecting. However, this would result in a very simple robotic system. Having studied both computer vision and image processing I wanted to build something that was a bit more interesting, therefore I took the same idea and added to it. Instead of a system that used a simple set of statements e.g. If (light){ goforward(); } else{ turnright(); } I would introduce some real image processing to the project. For this I would use a webcam that could take images and then decide if the ball was there or not. After consulting notes for both the computer vision and image processing modules[20][16] I developed some rough ideas of how to proceed. Initially, I had proposed that the system would use a top down camera that would be able to observe the entire playing field and then software which could locate the player in relation to the ball and the goal. The Robot would have two markers on its top that would allow the system to calculate the location and the orientation of the robot. Using this information the system would work out which way the robot would have to go. Following this it would pass on the details and instructions via the IR link to the robot, then the robot would carry them out. It became apparent quickly that this idea was flawed. Using the standard dimensions of the RoboCup soccer field 1220 x 1830[12] the camera would have to be situated at such a height that not only made it difficult to develop (only being able to test it in specific locations) but it also meant that it was almost 10

16 impossible to make out any of the lights on the ball and the reference colours on the robot. Using this system with more than one robot would have been easier requiring only one camera, the code running on the individual robots would have to do much more however, since the robot would have to both locate itself and its orientation to the ball. Which the RCX brick was capable of doing the maths there were some small performance issues when transmitting the data from the camera to the RCX via the IR link due to the low baud rate of the connection. Instead I decided to mount the camera on the top of the robot giving the webcam a point of view orientation, which I was much more interested in working with anyway. The robot would essentially look for the ball and then the goal Motion Detection I looked into many different ways of using the webcam to recognise the ball in an image, one of the first ideas to be dismissed was the idea of motion detection. There are many reasons why motion detection is unsuitable, the first, and most obvious is that the ball may not be in motion, and that if the robot is moving then the whole image changes making any motion detection algorithm almost useless Object Detection using Shapes A system that used image processing to find shapes seemed to be a more viable option. The idea was to use image processing to find circles, with the ball hopefully being the most obvious circle in the field of vision. I had explored for a while, researching information from the from a computer vision module. I produced a few test programmes in Matlab that would use Hough transform[6] to find the best example of a circle in images with the ball in. While this worked to a reasonable degree, there were problems. The system was not 100% reliable and a separate system would be needed to then find the goal once the ball had been acquired. While working on the birds eye view camera processing I came across the Lejos vision libraries. The Lejos vision system contained functions that supported detection of motion, light and colour. Usefully it also contained the ability to send commands to the RCX as well as displaying the output of the camera to the PC. After getting to grips with this system I decided to use the colour detection functions to locate 11

17 the ball Pixel Values The RoboCup ball emits quite a bright white light, therefore it should be possible to use the pixel value detection functions in the josx.vision libraries. Value analysis in image processing works by breaking down each pixel into its constituent RGB values. To find a particular colour or brightness in the image you perform the same process on that colour and then move pixel by pixel through the image looking for values that are the same. It is possible to apply a threshold to these values that allow colours that are not an exact match to be accepted by the system. The LeJOS vision system allows the image to be split into sections and then each section can be evaluated, returning the average RGB values in that area. From this you can employ your own checking routine to make the comparison Holding the Ball When I initially built the robot I had constructed simple arms that were fixed in the hope that the ball would be held in place by them as the robot rotated but it quickly became clear that I would need something a bit more sophisticated. When the robot rotated, the arms would knock the ball off at a tangent to the circular motion. Even when I had built the arms in a way that would better encircle the ball it was still a problem. Worse still was the issue of colliding with the ball, as soon as the robot made contact with the ball, it would be knocked. A solution was devised fairly quickly. Motorised arms would be used to replace the ineffective fixed arms. A motor was attached to a gearing system that allowed the arms to close around the ball holding it in place while the robot rotated and moved around. A touch sensor was placed on the front of the robot, positioned so that when the robot collides with the ball the motor engages trapping the ball in position. Using this method rather than using the vision 12

18 system, allowed the robot to react faster to the sensor input. This meant that the robot didn t rely on the vision system realising the ball was close enough and then waiting for the instructions to be broadcast over the IR link, which take significantly longer. This is comparable to the human central nervous system that is able to react faster to certain stimuli by bypassing the brain for certain uncontrolled reactions, usually as a reaction to danger or pain. 2.4 Software Thanks to the Lego corporations support for education the software of on the RCX is open source and can be replaced by third party firmware allowing people to develop other platforms to control Mindstorms robots. The default software that comes with the mindstorms kits is called ROBOLAB, it is based on a drag and drop system targeted to school aged children and it is capable of quickly producing simple functions. While this can be used to do a lot with the Mindstorms kits it does not contain all the functionality that I would require. There are four other languages that are appropriate for this project which I will discuss below. [17] LeJOS LeJOS is an implementation using Java. There are many libraries available for LeJOS including a very useful library called josx.vision that is now even supplied with the LeJOS source code and examples. This library contains useful functions that make remote control and vision processing possible. [15] LeJOS and most of the languages available for the RCX brick come with their own firmware that is a replacement for the standard Lego firmware. The new firmware that comes with LeJOS increases the functionality of the RCX brick, making it possible to use floating point numbers and arrays. It was this extended functionality that enticed me to use LeJOS when looking at available languages. My prior experience in the Java programming language was also a factor in choosing this over other options such as NQC. 13

19 Later in the project I also found a LeJOS plugin for the eclipse development tool, this was found at Eclipse is a very powerful application that aims to increase programming productivity, the location of this tool made a great difference when compiling and running code during the development BrickOS BrickOS was formally LegOS this is a C/C++ system using the gcc and g++ compilers. This however is not as complete as LeJOS it lacks as much support for video feeds and remote control. This would not be my ideal choice of development environment since my knowledge of C is not as good as that of Java. [2] NQC NQC is a C based language that has been developed for the Mindstorms system. This might have been an option but after some investigation appears to be a little too simplistic, as its inability to deal with floating point numbers arrays and a limit of 32 variables in a program. This is because it uses the built in firmware of the RCX brick, however it would make it possible to use vision command, the Lego webcam software and remote control software that is available [18] LegoLog Legolog is a Prolog based system which is a high level programming language that was designed for cognitive systems. I am wary of using this system due to the lack of support, both online and within the department so further investigation seemed inappropriate. [14] 14

20 2.5 Navigation There is area that is important in all robotics projects and that is the issue of navigation. There are several different methods that can be considered when starting any project, these are categorised as dead reckoning, which is useful in a deterministic and discreet world. An appropriate method in these circumstances would be to subdivide the playing area into a virtual grid, assigning each grid reference a property e.g. ball, player, goal. To navigate in this world you would need to use trigonometry, the power output and the running time of each engine in order to determine which section of the pitch the robot is actually in. This method is unsuitable since the world is actually stochastic and continuous, just because the robot has made contact with the ball it does not follow that the robot has possession with 100% certainty. There is instead the chance that the ball bounced off one of the arms, or simply rolled away. I took an almost opposite approach. By constantly reacting to its surroundings and never relying on a remembered image of the world the robot is able to adapt to changes in the environment almost instantly. Since the robot has only two major goals; to find the ball and to take or propel the ball into the goal, the need for navigation around the becomes minimal. The robot only has to find one static reference within the world and that is the goal. I had thought about two ways of fulfilling this task; the use of the grey scale mat and the vision system. The gradient of the mat can be used to determine the direction of the goal. By using one of the light sensors pointed at the floor, the robot, having acquired the ball should move in a direction turning until a constant value is returned. At this point the robot would be travelling parallel to the goal. The robot could then determine the direction to the goal by turning 90 degrees and moving forward to monitor the change in value then propel the ball forward. This however is not perfect, resulting in a lot of unnecessary motion and wasted time. Wouldn t it be better if the robot just turned on the spot to face the gold and started towards it? To this end I decided to use a colour marker as the robots reference point, this proved to have a few benefits. Firstly, I did not have to worry about having the vinyl mat at my testing locations and secondly the robot became more robust, being able to operate in new environments without significant downtime. 15

21 2.6 Final Solution The image analysis is mostly taken care of by the josx.vision package. This is made available in the LeJOS package from the website and covered all the things that I needed it to do. The way I used the vision system was to split the screen into three vertical sections. If the ball was to be detected on the right hand side of the screen then a command to turn right was issued to the robot. In the same way; if the ball was found in the left hand side of the screen then the robot was ordered to turn left. Had the robot not received any of these commands then it would continue to rotate on the spot until the vision system provided other instructions. In actuality the screen is split into 16 sections in a grid with average RGB values calculated for each of them. There was an upper limit to the number of sections it could be split into due to the processing requirements, any more than the 16 would result in there being a noticeable lag in the system, coupled with the lag of the IR communication this was unacceptable. The large sections that were generated by this meant that the lights from the ball were lost - the ball appeared black lowering the average brightness of the region, counteracting the small contribution provided by a bright light over a small number of pixels. To counteract this effect, the portion of the image that was analysed was reduced to the bottom half since that was where the ball always appeared due to the positioning of the camera. This allowed the light to take up a higher proportion of the pixels in a given area, which proved effective in increasing the accuracy. While this is going on the vision system is also looking for the goal. The way the system works is this; if the goal is detected then the system will pass a command to the robot, however the robot will only obey this command if the ball is in its possession. The robot uses the input from the touch sensor to decide if it has the ball or not. 16

22 Figure 2.2: The grid overlay on the vision system The final robot (See fig 5.4) consisted of a direct drive system with one motor per wheel, this provided the extra power to move the heavy robot. The RCX brick was attached to the frame of the robot with the IR reciever pointed straight up, allowing the laptop to stay in constant contact by mounting the USB tower near the cealing. A third motor was attached to the gearing system that allowed the robots arms to move to close on the ball. (See fig 5.1) Difficulties associated with mounting the gear set vertically meant that there was a discrepency in the gearing ratios of the two arms though this did not effect the arms ability in any noticable way. The camera was held in place on top of the robot with some creativly placed Legobricks. There was also a mounting station build for the IR tower as holding it in place soon became tiring.(see fig 5.3) 17

23 Chapter 3 System Evaluation 3.1 Testing Initial testing was performed as development continued. I was using a variation of the Dynamic Systems Development Model (DSDM) which is a form of rapid actions development process, it is a cyclic methodology that employs prototyping and periodic development cycles going through the stages of development, evaluation and improvement Test Methods The first stages of the testing came after the initial build - could the robot move. This was tested on the two different drive chains I had used. This was a simple pass fail kind of test, either it moved properly or it didn t. The test of the initial arms are not really documented. The actual test carried out was a simple one, the ball was placed in proximity to the robot, within the two arms and then the robot was rotated on the spot to see if the ball would stay near the robot. It didn t. 18

24 The new arms could be properly tested. Firstly the ball was placed directly in front of the robot which was told to move forward. A direct test of the touch sensor if was completed. If the positions of the arms and the touch sensor are correct then the arms should fire every time, holding the ball in place. The next test was a test of the vision system and of tracking the ball. In this test the robot had been programmed not to move to the ball, this meant that if the robot was to behave correctly then it would rotate until it spotted the ball, and from then on (provided the ball didn t move too quickly) it should pan to always face the ball. For this test the lighting levels were varied. They were all carried out at the same time in the interest of consistency. The light levels were denoted by having; the light on and the blinds open, the blinds closed with the lights on and then with both the lights out and the blinds shut. For the robot to pass it must track the ball for 360 degrees at a distance of 50cm from the ball. The final test was the whole system working together. Again this was carried out in three levels of brightness. The results of this and the other tests follow Test Results Drive Chain: Drive Chain Dual Differential Direct Drive Did it work? No yes Summary and observations: The Dual Differential system was unable to put forth enough power using the one motor to move the robot forward this is probably due to the weight of the RCX, the 6 batteries, the webcam and the gearing mechanism for the arms. Capturing the ball: 19

25 Run Was the ball captured? 1 Yes 2 Yes 3 Yes 4 No 5 Yes 6 Yes Summary and observations: 5 out of the 6 attempts were successful. The time that the arms did not close about the ball the ball had hit the side of the arms and had bounced away. This displays an 80% success rate. Even with such a small test set I believed it to be sufficient to be considered complete. Tracking the Ball: Lighting Bright Half Light Dark Pass or Fail? Fail Fail Pass Summary and observations: When testing in the full sunlight the surrounding surface and objects would drown out the ball due to the effects described in chapter 2.6. The robot would only track the ball successfully in the dark where the lights were distinct enough to be seen. Tests in a game situation: Lighting Run 1 Run 2 Run 3 Bright Fail Fail Fail Half Light Fail Fail Fail Dark Pass Pass Pass Summary and observations: When preforming the final test the robot was able to complete the task successfully. The system was able to complete all the tasks under optimum conditions. Though, again in brighter conditions the robot 20

26 was unable to distinguish the ball from the background. 3.2 Evaluation Comparing system to initial objectives It is clear to see from the results that the system is capable of fulfilling the minimum requirements set out at the beginning of the report. However the system does more this. Instead of using the default sensors to locate the ball, a computer vision solution was developed that provided more functionality and even opened the door to further uses of both the software system and the robot itself. It is also obviously clear that light effects the system drastically, I suspect that the ball used is actually much less vivid than intended, either that or the camera is unable to pick out the IR light as well as others. The system could be altered very simply in order to react instead to a coloured ball which would effectively invert the effects of lighting to the system. The effectiveness of this would be maximised by using contrasting colours for the border of the pitch and the ball Problems faced along the way It was clear from very early on in the development cycle that some of the conceptual ideas for the system were not going to work. These included the idea of using the birds eye view for the camera and having the fixed arms to control the ball. The biggest problem to arise was an administrative problem mid way through the second semester as the work was really getting going on the software it became apparent that the laptop used for the development was not in the best condition. It was then that a room with a computer was requested to continue the development. However, due to problems with the support staff system, none of their s were getting through and a small issue halted development. This resulted in the truncation of the allotted development time. 21

27 3.2.3 Overall The Schedule devised at the start of the project was heavily weighted towards the second semester (as documented in the mid project report) this schedule, though necessary due to the distribution of credits, meant that any problems that arose would have less time to be solved. The problems were overcome and the both the minimum requirments and my personal goals were achieved and so the results of the project in its entirety have been very pleasing. 22

28 Chapter 4 Possible Extenstions 4.1 Full Wireless Functionality The addition of a wireless camera would have given the robot more freedom, however, this equipment was not available. For the total removal of the laptop as an external processor, some technical limitations would need to be addressed; there would have to be a new communication port available on the RCX since the brick connections would probably be inappropriate, there would probably have to be an improvement to the overall processing and memory capacity available in order to carry out even the most basic of image processing tasks at the rate that would be necessary. 4.2 Self Localisation Adding a method of self localisation would allow the robot to perform more complex navigation within the environment this would most likely come about by using some visual queues on the actual playing field. There is the traditional RoboCup method of using the brightness of the mat as the reference. There is another way though, using an edge detection algorithm that looks for changes in the image as the marker of an edge could in theory be limited to looking for only a very slow change over the whole 23

29 image. This could be used along with converting the 2D points on the image to 3D points in the world to find the relative angle of the gradient to the point of view. This system would require significant development time. 4.3 Adding more Players This extensions would require more than just the construction of more robots. Extra functionality would need to be added to abilities to locate all the other players navigate around each other. This would call for the addition of extra collision detectors to warn of player/player collisions. Extra routines would be necessary to try and go around players, this could be achieved by using the vision system to judge whether the opposing player was between it and the goal/ball and then taking appropriate steps to fix it. The amount the opposition overlapped the goal could be used as a indication of how far the player would have to move in order to circumnavigate them. 4.4 Other Uses The produced software and robot provide a basis for solutions to other more significant problems and could be adapted in a number of ways. For example, the robot could have its arms replaced with whiskers that use the touch sensors to navigate through corridor type environments and could be be used in a RoboCup rescue competition. The vision system could be adapted to be used in more advanced robotic systems, that would allow the image processing to be integrated with the robot, moving away from the Mindstorms platform. 24

30 Chapter 5 Conclusion The aim of the project was to produce a Lego Mindstorms robot that could play a simplified version of the RoboCup soccer game. In order to carry this out the problem was divided into several sub-problems: make the robot move locate the ball on the field move to the ball hold the ball turn with the ball move to the goal with the ball direct the ball into the goal The minimum requirements of the problem encompassed the key areas of this; being able to move to the ball implies being able to move. I believe that the project has been a success, problems that occurred due to hardware issues with my laptop and the communication software with the support staff were 25

31 eventually overcome and the software was developed. Though the robot was designed with the RoboCup soccer in mind I actually think, with some small modifications, it could be used for a wide range of tasks. If you were to switch the arms for collision detectors it could quite easily be a RoboCup Rescue bot. The changes to the software would be minimal, with the addition of a right hand wall following function running instead of the default ball search. Most of the previous projects in this area seem to have limited themselves to the Mindstorms Kit, last year was an exception however. There were two projects of interest Mark Jagan [11] and David Ashmore [1]. Jagen s system was fairly simple using motion detection while not really identifying the object. Having looked back over Ashmore s work, it seems that we had both followed similar paths when considering the options available. After taking a look at the inner workings of the solution he had produced, there are a few areas that were improved upon, such as the way the image was analysed. In the project after observing the position of the camera it was apparent that the top half of the image wasn t used and so I stopped analysing it, reducing the system load. Using the RoboCup ball clear lighting level limitations must be set on the system. The use of the IR emitting ball is clearly aimed at the light sensors provided by the Mindstorms kit. For a system that is based on image analysis an emitter in the visible light spectrum would have been more appropriate, or just a coloured ball. The function used could easily be adapted by changing which values are measured. Doing this would vastly improve the performance of the robot in almost all lighting conditions. Overall I believe the project to have been success. The Lego Mindstorms kit is a an excellent introduction into the world of both robotics as well as key AI concepts and ideas. 26

32 Bibliography [1] David Ashmore. Creating A Robotic Footballer. University of Leeds, [2] BrickOS. Brickos homepage. April [3] The RoboCup Federation. Official robocup site. April [4] Giulio Ferrari, Andy Gombos, Soren Hilmer, Jurgen Stuber, Mick Porter, Jamie Waldinger, and Dario Laverde. Programming Lego Mindstorms with Java. Syngress, [5] Mario Ferrari, Giulio Ferrari, and Raplh Hempel. Building Robots With Lego Mindstorms: The Ultimate Tool for Mindstorms Mainiacs. Syngress, Chapter 2: Playing with Gears. [6] David A. Forsyth and Jean Ponce. Computer Vision: A Modern Approach. Prentice Hall, [7] M Gasperi. Michael gasperi s lego mindstorms nxt/rcx sensor input page. April [8] The Lego Group. Lego.com lego mindstorms. April [9] The Lego Group. Lego.com mindstorms overview. April [10] Hobbytron.com. Lego mindstorms robotic invention system April [11] Mark Jagen. Motion Sensitive Security Camera Robot. University of Leeds, [12] Johannes Klotz, Brian Thomas, Leonard Vong, and Eduardo Pinto. Robocup soccer rules. April

33 [13] J. Knudsen. The Unofficial Guide to LEGO MINDSTORMS Robots. O Reilly, Chapter 11: Make Your Own Sensors. [14] Legolog. Legolog homepage. April [15] LeJOS. Lejos homepage. April [16] Derek Magee. Ai31: Computer vision. March [17] Martha. N. ROBOLAB - Getting Started, Teachers Guide for ROBOLAB software. LEGO DACTA, [18] NQC. not quite c homepage. April [19] Hacienda Robotics Program. Compact dual differential. April [20] John Stell. Ai21: Speech, audio and image processing. March [21] Wikimedia. Main page - wikipedia, the free encyclopedia. April

34 Appendix A I found this project to be extremely enjoyable, taking two things that I enjoy - building things and computing - and combining them with robotics, a field that has always interested me. During this year I have learnt much about the processes involved in undertaking a project such as this, such as the developmental issues and the administrative complications that can come about when working on a full project. There are also the rewards associated with building something that interests other people, and the continual development of ideas into something deeper and more complex. I also feel that I have learnt a lot about computing and AI in general. Much of this came from reading books such as the Mindstorms programming with Java book [4] and wikipedia [21] which has become an incredible place to find appropriate books and sources of information. If I could have changed anything about the way this project had gone, I would have ensured a stable system to work on earlier. Due to communication issues and the software used by the local support team there were some problems; these pushed the software development back and it has impacted the time available for completing the project as a whole. One of the first things you should do is make sure that everything is ready to work on, even if you do not plan on attempting the practical side for weeks. Overall though I have to say that I am generally pleased with how it all went. 29

35 Appendix B Figure 5.1: The gearing system for the arms Figure 5.2: The drive system used 30

36 Figure 5.3: The Docked USB tower Figure 5.4: The final incarnation of the robot 31