Representation Learning for Mobile Robots in Dynamic Environments

Transcription

1 Representation Learning for Mobile Robots in Dynamic Environments Olivia Michael Supervised by A/Prof. Oliver Obst Western Sydney University Vacation Research Scholarships are funded jointly by the Department of Education and Training and the Australian Mathematical Sciences Institute.

2 1 Abstract This project will be using Neural Network and exploring different architecture of networks using data from an abstract robot simulation, RoboCup simulation league. This is where two teams of 11 robots play soccer against each other. In this simulation, robots are part of a dynamic environment where they see a part of the soccer field, with a number of landmarks, and possibly the ball and other robots. For close-by robots, this also includes information about team and uniform number; for robots that are further away, this information is missing. We will investigate different neural network architectures, with the goal to find out how to best learn a representation that allows for an accurate reconstruction of the current state of the environment. 2

3 Contents 1 Abstract 2 2 Introduction 4 3 Neural Network 4 4 RoboCup 6 5 Architecture of the Neural Networks 6 6 Results 7 7 Conclusion and Future Work 8 8 Acknowledgments 8 3

4 2 Introduction Mobile robots that move around in a known environment together with a team of other robots have two very fundamental problems to solve: (1) Where am I?, and (2) Where is everyone else? The problem of a robot estimating its own location and orientation, using data from sensors like cameras, laser range finders, or sonars, is called self-localisation. Known positions of fixed objects that are detected by the robot s sensors can provide the necessary information to calculate the current location and orientation, provided enough of these landmarks are in the robot s current field of view, and the sensor data are accurate enough. In situations with limited information, however, it may not be possible for the robot to correctly localise just from its current view of the world. In this case, statistical approaches can be used, to estimate and filter out the most likely current position in the environment. Robots that cooperate in a team face the additional problem of keeping track of everyone else in the team, in particular with only partial perception of the environment and unreliable information. An accurate model of the current state of the world is beneficial to both plan actions, as well as to predict what the state of the world might look like in the near future. Approaches for self-localisation and maintaining a world model usually require a choice of representation for objects in the environment, a sensor model, or also a selection of features that are deemed relevant for the problem, and as a result, are dependent on the quality of representation, sensor model and feature selection. In this project, we will investigate to instead learn the best representation of current and past sensor data in order to maintain an up-to-date world model, using a neural network ( deep learning ) based approach [7]. This project will be using data from an abstract robot simulation, RoboCup simulation league [3], where two teams of 11 robots play soccer against each other (see also Fig. 4). In this simulation, robots are part of a dynamic environment where they see a part of the soccer field, with a number of landmarks, and possibly the ball and other robots. For close-by robots, this also includes information about team and uniform number; for robots that are further away, this information is missing. We will investigate different neural network architectures, with the goal to find out how to best learn a representation that allows for an accurate reconstruction of the current state of the environment. 3 Neural Network Humans store learned information in a highly interconnected neural network in the brain. When given a set of inputs and outputs, our neurons learn the relationship between the inputs and outputs through weights assigned to each input and net of synapse firing thresholds [1]. Artificial Neural Networks are inspired by the operations of brain. In our application, it s is a two-stage regression or classification model (see Fig. 1) [5], but Neural Networks can also be used for classification tasks. Z m = σ(α 0m + α m T X), m = 1...M (1) Y k = (β 0k + β k T Z), k = 1...K (2) where Z = (Z 1, Z 2,..., Z M ) are the units in the hidden layer, and Y = (Y 1, Y 2,..., Y K ) is the output. Non-linear activation functions are an important component of neural networks, and the sigmoid function, σ(x), is a commonly used example for such a function (see Fig. 2 for graph). 4

5 Multiple linear layers in a neural network can be reduced to one linear layer, so that multiple layers only make sense when there are nonlinearities like σ(x). Sigmoid function equation σ(x) = 1 (1 + e ( x) ) α are the weights in the first layer and the β are the weights in the second layer which are initially chosen at random then adjusted accordingly to minimise the error using the back-propagation method [4]. The root mean square error is used to compute the error by subtracting the computed output Y k from the correct output Y i RMSE = n 1/n (Y i Y k ) 2 (4) i=1 (3) Figure 1: Training a Neural Network [2] The number of hidden layers, and the number of nodes in each hidden layer, must be specified in advance. In theory the more hidden layers and nodes the more accurate the neural network but this will increase the training time as well as risk overfitting, as the neural network will memorise the output rather than compute it which would result in low training error but a high testing error. 5

6 Figure 2: Sigmoid Function [8] 4 RoboCup Robocup is an international competition, it s goal is by the year 2050, develop a team of fully autonomous humanoid robots that can win against the human world soccer champion team. There are many leagues in Robocup including the 2D simulation which is the most practical when trying to evaluate various strategies, theories or algorithms [9]. In the simulation, there are 2 teams of 11 virtual players versing each other in soccer. Each player is a seperate computer program with it s own environmental information that it can see. There are stationary landmarks on the field which are 55 flags and 4 lines (Fig. 3). Each player has a camera sensor that relays what is in it s field of view once every step cycle, once every 150 milliseconds which it can use to know its location. Fig. 4 is an example of the information that s being relayed to the player. It contains the Flag name, distance and angle of the flag from player itself. We created code and different tools to extract this data for the flags as well as the players exact correct locations from the simulator and convert it into the correct format for the Neural Networks [3]. 5 Architecture of the Neural Networks After trying many different Architectures for the NN, the best results were achieved when using a NN with 4 inputs; X and Y coordinates of the flag location, the angle and distance of the flag from the player. Two independent Neural networks were created for the X- and Y- coordinates of the player (see Fig. 5 and Fig. 6 respectively). With this approach, all weights can be trained specifically for the purpose of estimating X or Y. Using these Networks, the player is now aware of it s location so new Neural Networks were created using these estimates to take advantage of the 6

7 Figure 3: Landmarks on the soccer field neural networks ability to make predictions. The new networks have 2 inputs each, the previous X coordinate of the player as well as the current one to predict its next X location (Fig. 7) and a similar network for the Y location (Fig. 8). 6 Results Fig. 9 shows an example of the results, the yellow circle represents the actual location while the other circles represent the predictions according to individual flags, the purple being it s worst prediction. So by averaging the results we were able to get a much more accurate results as in Fig. 10. The neural networks had a large RMSE for the X and Y location, as in the results table below, when averaged it had a smaller error but it s still large error compared to the results reported in [6] using the triangulation method which had an average error of The prediction error using the estimated locations was 0.31 (Fig. 11). To estimate a lower bound for the prediction error of our method, we also used ground truth of previous player locations as an input (Fig. 13). This resulted in a very low error of 0.10 for predicting the next position of the player (Fig. 12). This error is lower than the triangulation error for estimating the current position. It s also lower than the error of assuming that the player remains stationary to ensure that the neural network is successful and useful. 7

8 Figure 4: Data received according to field of view Results Error XNN 2.37 YNN 2.27 Average 0.85 Prediction Est NN 0.31 Prediction Act NN 0.10 Triangulation [6] Conclusion and Future Work Even though the neural networks were not able to determine the current location of the player more accurately than triangulation, we were able to predict the next position of the robot with a small error rate. There is a large scope for future work within RoboCup, or for applications in robotics in general: as an example we can explore different inputs for the Neural network that can better the localisation such as speed and energy. We can also predict the location of other players, the ball or we can use the neural network to predict more than one future move (Fig. 14) and integrate this information into the player s game strategy. Outside RoboCup, Neural networks are becoming very popular because of the ability to make predictions and they are being integrated into many areas today. In this project the neural networks were able to predict the next position for the robot. This method can be used in other areas, e.g., to predict the location of planes [1] or in an autonomous car. 8 Acknowledgments I would like to thank the AMSI for this opportunity and the funding as well as my supervisor Oliver Obst for his guidance, time and effort. 8

9 Figure 5: X Neural Network Figure 6: Y Neural Network 9

10 Figure 7: X Prediction Neural Network Figure 8: Y Prediction Neural Network 10

11 Figure 9: Prediction vs. Actual Figure 10: Prediction and Average vs. Actual 11

12 Figure 11: Prediction A vs. Actual Figure 12: Prediction E vs. Actual 12

13 Figure 13: Prediction E vs. Prediction A vs. Actual References [1] Doshi A. Aircraft Position Prediction Using Neural Networks. Massachusetts Institute of Technology, [2] Oustimov A. and Vu V. Artificial neural networks in the cancer genomics frontier. Translational Cancer Research, 3, [3] Heintz F. Huang Z. Kapetanakis S. Kostiadis K. Kummeneje J. Noda I. Obst O. Riley P. Steffens T. Wang Y. Chen M., Foroughi E. and Yin X. RoboCup Soccer Server, July [4] Bengio Y. Goodfelloe I. and Courville A. Deep Learning. MIT Press, [5] Tibshirani R. Hastie T. and Friedman J. The Elements of Statistical Learning [6] Bach J. and Gollin M. Self-loalisation revisited. RoboCup 2001: Robot Soccer World Cup V, pages , [7] Schmidhuber J. Deep learning in neural networks: An overview. CoRR, [8] V. Morello, E. D. Barr, M. Bailes, C. M. Flynn, E. F. Keane, and W. van Straten. Spinn: a straightforward machine learning solution to the pulsar candidate selection problem. Mon. Not. Roy. Astron. Soc., 443(2),

14 Figure 14: Predicting the next locations [9] Ribeiro F. Stone P. Stryk O. Nardi D., Noda I. and Veloso M. Robocup soccer leagues. 14