Self Organising Neural Place Codes for Vision Based Robot Navigation

Transcription

1 Self Organising Neural Place Codes for Vision Based Robot Navigation Kaustubh Chokshi, Stefan Wermter, Christo Panchev, Kevin Burn Centre for Hybrid Intelligent Systems, The Informatics Centre University of Sunderland Sunderland, SR6 DD United Kingdom URL: Abstract Autonomous robots must be able to navigate independently within an environment. In the animal brain so-called place cells respond to the environment an animal is in. In this paper we present a model of place cells based on Self Organising Maps. The aim of this paper is to show how image invariance can improve the performance of the neural place codes and make the model more robust to noise. The paper also demonstrates that localisation can be learned without having a pre-defined map given to the robot by humans and that after training, a robot can localise itself within a learned environment. I. INTRODUCTION Ideally, an autonomous robot should have a self-contained system which allows it to adapt and modify its behaviour to all possible situations it might face. The classical method is to pre-define the internal model of the robot relating it to the external world. With a pre-defined program the robot can navigate only within a highly controlled environment [1]. However, the external world is very complex thus making it often unpredictable [2]. There are various ways to address the problem of localisation. The most common approach is to ignore the errors of localisation [3]. This has the advantage of being simple but a major disadvantage is that it cannot be used as a global planning method. To overcome this problem, another technique can updates the robot location by bar codes and laser sensors in the environment, thus giving it an explicit location. This method is motivated purely by the go until you get there philosophy [3]. Another approach is to use topological maps, where symbolic information for localisation at certain points, for instance gateways, is used [4], [], [6]. With gateways as a navigational strategy, the robot can change its direction, for instance at intersections of hallways. Therefore the gateways are critical for localisation, path planning and map making. The main disadvantage here is that it is very hard to have unique gateways. Another technique of localisation is to match raw sensor data to an a priori map [3]. Usually the sensors used are distance sensors such as sonar, infrared, laser etc. The problem here is that the sensor data rarely comes in without noise but it can be used in highly restricted environments. It is used in certain industrial robots which operate in highly controlled environments but they fail to work in dynamic environments. This problem is overcome by generating small local maps and then integrating them with larger maps. The use of small local maps motivates the need of good map building [3]. In this paper we present a model of place cells that learns to form landmarks for helping the robot to navigate. In order to generate an internal representation, we use vision to identify landmarks. This paper focuses on a type of landmark localisation which depends on extracting novel features from the environment in order to localise itself. Landmarks can be defined as distinguishable input patterns. In other words each category of input activates different output neurons. In our approach we have developed Self Organising Maps (SOMs) which classify the input images for localisation. One of the major problems that is solved by the visual system in the cerebral cortex is the building of a representation of visual information which allows recognition to occur relatively independent of size, contrast, spatial frequency, position on the retina, angle of view etc. Although neural networks do generalise well, the type of generalisation they show naturally is to those vectors, which have a high correlation with what they have already learned [7]. If we consider a shift in the image coming into the retina by just by a few pixels, the same neuron in the SOM s map will not respond even though it is the same pattern but the neighbouring neuron would respond for this pattern. Therefore we clustered neurons for the same landmark on the SOM. The self organising map does not grow with the number of landmarks. In order to represent more landmarks, it is necessary to reduce the number of clusters that are being formed on the map. This creates a need to compute invariant pattern representations for the map. Invariant pattern representation is a form of preprocessing performed in V1 of the visual cortex and striate cortex of primates [8]. As our aim is to have smaller clusters of place codes on the self organising map, our focus is not on size-invariant landmarks, because landmarks from different distances would have different sizes in the visual field. We focus on transform invariant processing. Our model is based on associative memory for template matching and template alignment. We demonstrate that neural localisation based on self organising place codes

2 holds a lot of potential for learning robot navigation. II. ARCHITECTURE Despite the progress made in the field of AI and Robotics, robots today remain inferior to humans or animals in terms of performance [9]. One main reason for this may be that robots do not possess the neural capabilities of the brain. Human and animal brains adapt well to diverse environments, whereas artificial neural networks are usually limited to a controlled environment, and also lack the advantage of having millions of neurons working in true parallelism. Our approach is to try to emulate the neural visual navigation functions present in human and animal brains. The main emphasis of this research is to build a robot with a goal-based navigation, with the objective that it can learn and move autonomously. The central focus is on natural vision for navigation based on neural place codes. This section summarises the primary algorithm and the principle underlying our approach. A. Overall Architecture of our Model In our architecture there are various neural networks responsible for various aspects of navigation. Navigation is often based on sensor fusion; navigation cannot just depend on one sensor input [1], [11] and humans and animals use various sensors to navigate [3], [12]. We are primarily using vision as global navigation strategy and for our reactive and local navigation behaviour we have employed the use of sonar and infra red. Therefore our model of the navigation system itself consists of several neural network units. We used SOMs for the visual landmarks, which enables the robot to generate its landmarks of the most salient features in a visual field. A primitive visual landmark allows us to implement simple visually-based behaviour. Our approach is based on functional units and each of these functional units uses a neural network. An overview of the different functional units can be seen in figure 1. In figure 1, visual information derivation is a module which is responsible for getting the images from the robot s camera. The image preprocessing in the localisation module is responsible for normalising the images for the network. Our invariance processing (shown in more detail in figure 2), part of localisation module, makes use of associative memory for transform invariance and pattern completion for noise reduction. The invariance and pattern completion modules are based on Multi-Layered Perceptron (MLP) (figure 2), the output of which forms the input to the SOM. Self localisation and target representation are based on SOMs. This makes up the localisation module which is responsible for the localisation of the robot in the environment. Goals or targets for navigation can be given by language, action, desire etc. In our modal we assume that the goal is triggered by language instructions. Once the language instruction is received the recall of the goal is triggered. An internal representation of this goal is then generated in the prefrontal cortex, or the working memory, of the brain. As this whole process is not within the scope of the this paper goal is given as the image of where the robot is desired to be. This image is then the target representation of the architecture. Effective navigation depends on the representation of the world the robot is using [3]. In our architecture the world representation is part of spatial representation module. This module will provide the path planning module with necessary information from the localisation module and visual target module. It will map both the current location and the location of the target into the same map. This enables the path planning module to choose the best pathway. There are various ways in which the robot can be instructed as to where its target for navigation is. We are currently experimenting how to translate the place code output and target representation into a spatial representation. The path planning module provides output to the motors. This forms the global navigation strategy. We have implemented the local navigation strategy using reactive behaviour. Both the global and local navigation strategy are combined in the navigation strategy module which is mostly responsible for choosing motor commands either from local or global behaviours. Accordingly it chooses the output from either the global navigation strategy or the local navigation strategy and transforms it into motor control commands. B. Place Code Localisation based on Invariant Images Hippocampal pyramidal cells called place cells have been discovered that fire when an animal is at a certain location in its environment. In our model, place cells can provide candidate locations to the path integrator and place cells could localise the robot in a familiar environment. Self-localisation in animals or humans often refers to the internal model of the world outside. As seen in a white water maze experiment [13] a rodent which was not given any landmarks could still reach its goal by forming its own internal representation of landmarks of the world outside. Humans and animals can create their own landmarks, depending on the firing of place cells. These cells change their firing patters in an environment when prominent landmarks are removed from the environment. With this evidence from computational neuroscience [14], [1], [16] it is reasonable to assume that place cells could prove to be an efficient way of localisation using vision. Self Organising Maps [17] networks learn to categorise input patterns and to associate them to different output neurons or a set of output neurons. Each neuron, j, is connected to the input through a synaptic weight vector w j = [w j1...w jm ] T. At each iteration, the SOM [17] finds a winning neuron v by minimising the following: v(x) = arg min j x(t) w j, j = 1, 2,...n (1) x belongs to an m-dimensional input space,. is the Euclidean distance, while the update of the synaptic weight vector follows: w j (t+1) = w j (t)+α(t)h j,v(x) (t)[x(t) w j (t)], j = 1, 2,...n (2)

3 Visual Target Visual Information Derivation Localisation Module Spatial Representation Path Planning Local Navigation Navigation Strategy Motor Control Fig. 1. Overall architecture of the visual navigation strategy of the robot. This diagram shows the flow of the model. Image Pre Processing AutoAssociative Memory Retina Goal Place Codes Based on SOMs Visual Target Spatial Representation Fig. 2. An overview of the localisation module in our model. The image pre-processing is responsible for getting the images from the robot camera and resizing it. The associative memory is responsible for image-invariant processing. Localisation of the robot is done by place codes. Visual Target represents the main goal for the navigation. Spatial representation will now take activations from both the neural network region and represent it in the environmental space. This classification is based on features extracted from the environment by the network. Feature detectors are neurons that respond to correlated combinations of their inputs. These are the neurons that give us symbolic representations of the world outside. In our experiments once we get symbolic representations of the features in the environment we use these to localise the robot in the environment. The sparsification performed by the competitive networks is very useful for preparing signals for presentation to pattern associators and auto associators, since this representation increases the number of patterns that can be associated or stored in such networks [8], [18]. Although the algorithm is simple, its convergence and accuracy depend on the selection of the neighbourhood function, the topology of the output space, a scheme for decreasing the learning rate parameter, and the total number of neuronal units [19]. An important property of self organising map is feature discovery. Each neuron in a self-organising map becomes activated by a set of consistently coactive input stimuli and gradually learns to respond to that cluster of coactive inputs. We can think of self organising maps as feature discovery in the input space. The features in the input stimuli can thus be defined as consistently coactive inputs. Self organising maps thus show that how feature analysers can be built without any external teachers [8]. This is a very important aspect of place cells, as the place cells have to respond to unique feature or landmarks in the input space in order to localise the robot. The removal of redundancy is thought to be a key aspect of how the visual system operates [8], [18]. Our hybrid model of neural networks also reduces the dimensions of the input vector as a set of input patterns, in our case pixels of the input image vector. Removal of redundancy is done by an invariance layer. The representation of a image is done by activation of neurons, which forms as input to Place Codes based on Self Organising Map. A. Experimental Setup III. EXPERIMENTS AND RESULTS The experiments were conducted on the Khepera robot (figure 3). The robot was in a cage, which was divided into four parts, north, south, east and west. The cage was further divided into 1 cm x 1 cm grids. These grids were only

4 trained. Invariance and the place codes networks were trained in a sequential manner. First the invariance layer was trained and then the neural place code layer was trained. Dimensions No of Connections Input Layer 17 x 27 x 3 - Invariant Layer 1 17 x 27 x 3 17 x 27 x 3 Invariant Layer 2 7 full connections Invariant Layer 3 12 full connections Invariant Layer 4 17 x 27 x 3 full connections SOM Place Code Layer x 17 x 27 x 3 Fig. 3. This is a Khepera robot that we use for the experimentation. used for the purpose of calculating the error by the place cells. All the landmarks were placed against the wall of the cage. There were cubes and pyramids of different colour codes spread across the walls of the cage randomly. The cage was divided in small squares of 1cm. Each square represents a place code. Each cell was given a name, depending on where the cells were located, for example a cell in the southern hemisphere within the eastern block would have a name se1. The naming convention was simple, the first letter representing which hemisphere, the second letter which block and the numbers the x and y co-ordinates. This information was purely for our use to test the results and to setup the experiments. This information was not provided to the robot. For training purposes there 4 images were taken from each place code. For testing, there were 1 images from each place code in the environment. N o r t h S o u t h East West Movement of Khepera Robot Fig. 4. This is the cage where the robot was placed. The robot was allowed to move in a limited space, as we did not want the robot to be too close to the landmarks nor too far away from the landmarks. The area of movement of robot was divided into smaller grids of 1cm x 1cm giving us an approximate position of the robot. The stimuli used for training and testing our model in this paper are specially constructed to investigate the performance of the localisation problem using the self organising maps. To train the network, a sequence of 2 images were presented to represent over 3 locations in the cage. At each presentation the winning neuron was selected and the weight vector of the wining neuron was update along with the distance vector. The presentation of all the stimuli across all the landmarks consists of one epoch of training. In this manner the networks were TABLE I DIMENSIONS FOR OUR HYBRID NEURAL MODEL FOR LOCALISATION B. Clustering of Place Codes BASED ON MLP AND SOM The basic property of a SOM network is to form clusters of information relating to each other, in our case landmarks. A cluster is a collection of neurons which are next to each other representing the same landmark. Figure ((a)(b)) shows that when the robot was approaching the desired landmark, there were activations in the neighbouring neurons. This is due to clustering of similar images around the landmark. There are multiple similar images that are being represented by a single neuron, making the cluster a smaller and richer in information. This is achieve with the invariance module. On the other hand figure ((d)) shows the landmarks which were at a distance to the location represented in figure ((a) and (d)). Two landmarks (nw1 and ne2) that were given to the robot at a distance are mapped not only in different clusters but also distant from each other. By their very definition, landmarks are features in the environment. This was the reason behind a formation of these clusters by SOM s. The landmarks that were chosen by the SOM were quite significant in the image and distinguished features from the rest of the environment and other landmarks. C. Reduction in Cluster Size It was observed in [7], that the main cause for large clusters of place codes was due to the Self Organising Map trying to handle transform invariance by having the neighbouring neurons responding to the invariance. With the use of associative memory for transform invariance the size of the clusters is reduced. Now, the Self Organising Map, does not represent the invariance, rather it represents the place codes. Images were collected at every 1th frame i.e. there was approximately half a second difference between the images. This causes a large amount of overlap and large amount of transform invariance. The associative memory clustered the similar images and reduced the transform invariance. The number of neurons per location could be reduced (figure 6). There would be fewer neurons required to represent the same location if there was a shift in the images. This has also has additional benefits, mainly now the Self Organising Map can represent more place codes without actually growing or increasing the size of the map.

5 1 Activation of Neurons for region ne2 1 Activation of Neurons for region ne (a) 1 1 (b) 1 Activation of Neurons for region ne2 andne3 1 Activation of Neurons for region nw (c) 1 1 (d) Fig.. (a) Activation in region of NE2 (b) Activation in the region of NE3 (c) Activation of both the regions of NE2 and NE3 (d) Activation of NW1 Neighbouring regions are place code neighbours to each other. There is also a clear overlap of a neurons (a) and (b) regions. The reason for the overlap is the robot between both the locations, it could see both the prominent landmarks. D. Performance of network To test the performance of the network, we tested our network with white noise with mean deviation ranging from. to.8 and variance of.1 to the image. The effects of the noise on the images can be seen in figure 7. The aim of the neural network is to localise itself in an environment. As the place cells are based on SOM, there is a cluster of neurons responsible for a place code in the environment. The neuron responding to that place code would be more precise in coordinates than the neighbouring neurons. There are various reasons why we would need a robust localisation. With the increase of noise, it was possible for the robot to be lost easily [7]. During these times, the approximate coordinate would help the robot to localise itself. We have considered two parts for localisation with noise handling, the first being that neuron responsible for the robot responding and the second being, another neuron in the cluster of neurons responding to the same place. In the latter case, localisation is not very accurate. As seen in figure 8, the clusters are more robust for noise handling as compared to having one neuron. As the amount of noise increases the neurons or the cluster response to localisation becomes blurred. However, the network performance is quite impressive, since until. mean deviation of noise, the error for localisation is below 3%. The cluster for place codes still performs much better and is still below 2%. The noise handling by the neural networks was also improved by adding an additional layer of associative memory below the place codes. The associative memory would reduce the noise before the visual inputs are given to the neural place codes layer in the architecture. It is seen in the figure 8 that associative memory helps place cells to perform better, giving less error per neuron. As the noise level increased above 7% the performance of the network with associative memory was reduced. This was mainly observed for higher levels of noise. With noise of 7% the performance did not make a lot of difference with or without associative memory. At higher levels of noise with more than 8% the noise handling at associative memory failed, reducing the place codes performance then without the layer. At the same time, with 8% noise the performance of SOM s is completely random as can be expected. For our experimentation we are not expecting more than 3% of noise levels. This level would mostly be caused by interference in wireless signals. Even if the noise levels for a few frames are higher than 3%, it would be possible for the network to localise itself with the following input frames coming from the robot retina. With the use of the associative memory, the performance has improved up to %.

6 Cluster of Neurons Noise Handling By Network Neurons Per Cluster Percentage of Error Mean Deviation of Noise added 1 NE1 NE2 NE3 NE4 NE4 NE NE6 NW6 NW7 NW8 NW9 NW1 NW11 NW12 NW13 SE1 SE2 SE3 SE4 SE4 SE SE6 SW6 SW7 SW8 Clusters Original Cluster Size After Autoassociative Memory SW9 SW1 SW11 SW12 SW13 Fig. 8. Error Per Neuron Error Per Cluster Error Per Neuron (Associative Memory) Noise handling by the neural networks Fig. 6. The cluster sizes i.e. number of neurons per cluster representing a particular location of the robot. There is a drastic reduction in the size of the clusters after the use of associative memory before the place code map. This makes it possible for us to have more place code on the a map of the same size. (a) Fig. 7. This figure above shows different noise levels added to the image. (a) Is is image without any noise. (b) Image with. Mean deviation (b) IV. CONCLUSIONS AND FUTURE WORK In this paper we have described a place cell model based on a SOM for navigation. The model was successful in learning the locations of the landmarks even when tested with distorted images. Visual landmarks were associated with the locations in a controlled environment. This model clusters neighbouring landmarks next to each other. The landmarks that are away from each other in the environment are also relatively away from each in the map. Rather than predefined localisation algorithms as internal modules, our SOM s architecture demonstrates that localisation can be learnt in a robust model based on external hints from the environment. In future model is further extended to be implemented on Peoplebot robot for the robot to localise itself within a corridor. ACKNOWLEDGMENT The authors would like to thanks Dr. Harry Erwin for his help and insight into navigation and place cells. Authors would like to thanks Chris Rowan for his help in collecting data for training purposes. REFERENCES [1] U. Nehmzow, Mobile Robotics: A Practical Introduction. London: Springer Verlag, 2. [2] A. Arleo and W. Gerstner, Spatial cognition and neuro-mimetic navigation: A model of hippocampal place cell activity, Biological Cybernetics, Special Issue on Navigation in Biological and Artificial Systems, vol. 83, pp , 2. [3] R. R. Murphy, Introduction to AI Robotics. London, England: The MIT Press, 2. [4] P. Bonasso and R. Murphy, eds., Artificial Intelligence and Mobile Robotics: Case Studies of Successful Robot Systems. London: The MIT Press / AAAI Press, [] S. Thrun, M. Beetz, M. Bennewitz, W. Burgard, A. Cremers, F. Dellaert, D. Fox, D. Hähnel, C. Rosenberg, N. Roy, J. Schulte, and D. Schulz, Probabilistic algorithms and the interactive museum tour-guide robot minerva, International Journal of Robotics Research, vol. 19, no. 11, pp , 2. [6] W. Burgard, A. Cremers, D. Fox, D. Hahnel, G. Lakemeyer, D. Schulz, W. Steiner, and S. Thrun, Experiences with an interactive museum tourguide robot, Artificial Intelligence, vol. 114, no. 1-2, pp. 3, [7] K. Chokshi, S. Wermter, and C. Weber, Learning localisation based on landmarks using self organisation, in ICANN 3, 23. [8] E. Rolls and G. Deco, Computational Neuroscience of Vision. New York: Oxford University Press, 22. [9] J. Ng, R. Hirata, N. Mundhenk, E. Pichon, A. Tsui, T. Ventrice, P. Williams, and L. Itti, Towards visually-guided neuromorphic robots: Beobots, in Proc. 9th Joint Symposium on Neural Computation (JSNC 2), Pasadena, California, 22. [1] J. A. Castellanos, J. Neira, and J. Tardoes, Multisensor fusion for simultaneous localisation and map building, IEEE Transactions on Robotics and Automation, vol. 17, pp , 21. [11] S. Martens, C. A. Gail, and P. Gaudiano, Neural sensor fusion for spatial visualisation on a mobile robot, in Proceedings of SPIE, Sensor Fusion and Decentralised Control in Robotic Systems (S. P.S. and M. G.T., eds.), [12] R. A. Brooks, Cambrian Intelligence: The Early History of the New AI. Cambridge, Massachusetts: The MIT Press, [13] A. D. Redish, Beyond Cognitive Map from Place Cells to Episodic Memory. London: The MIT Press, [14] A. Arleo, Spatial Learning and Navigation in Neuro-Mimetic Systems, Modeling the Rat Hippocampus. PhD thesis, Swiss Federal Institute of Technology, Lausanne, EPFL, Switzerland, 2. [1] A. D. Redish and D. S. Touretzky, Navigating with landmarks: computing goal locations from place codes., [16] E. Rolls and A. Treves, Neural Network and Brain Function. New York: Oxford Press, [17] T. Kohonen, Self-organizing Maps. Springer Series in Information Sciences, Berlin, Germany: Springer-Verlag, 3rd ed ed., 21. [18] S. Wermter, J. Austin, and D. Willshaw, Emergent Computational Neural Architectures based on Neuroscience. Heidelberg: Springer, 21. [19] M. Haritoppulos, H. Yin, and N. M. Allinson, Image denoising using self-organisising map-based non-linear independent component analysis, Neural Networks: Special Issue, New Developments in Self Organising Maps, vol. 1, pp , October - November 22.