Teaching robots: embodied machine learning strategies for networked robotic applications

Transcription

1 Teaching robots: embodied machine learning strategies for networked robotic applications Artur Arsenio Departamento de Engenharia Informática, Instituto Superior técnico / Universidade Técnica de Lisboa Institute for Human Studies and Intelligent Sciences artur.arsenio}@ist.utl.pt Conference Topic CT 13 Abstract A plethora of network applications exists nowadays for users to download to their mobile devices, game consoles, or personal computers. Interactivity is increasingly a must on such applications. This paper presents several interactive applications to be downloaded from an application server and deployed at robotic platforms, which are especially suited for human-robot interactions. We introduce learning on robot applications, aiming at increasingly and pervasively including robots into our society, at our homes or on the move. We argue for establishing social interactions, treating robots as humans, using theories from children development as a basis for developmental learning on robots. Hence, by downloading interactive applications, and equipped with a series of children toys, a robot can socially interact with humans, having an increasingly satisfactory experience as the robot learns new knowledge. Key Words: Machine Learning, Robotics, Distributed Systems, Networked Applications, Developmental Learning 1. Introduction Children love toys. Human caregivers often employ learning aids, such as books, educational videos, drawing boards, musical or textured toys, to teach a child. These social interactions should be extrapolated to robots as well, for them to learn and to interact with humans [1]. With the current advent of network applications, distributed over the internet (with components running on sensors or robots, while others running eventually on network servers [2]), there is the need to download applications into the robots. These applications must be automatically installed on robots, enabling them to interact with humans locally or even remotely through the internet. 1.1 Development of Biologically Inspired Interacting Applications that Learn We, humans, can be seen as a biological machine. However, we do not treat children as machines, i.e., automatons. But this view is still widely employed in industry to build robots. Building robots involves indeed the hardware setup of sensors, actuators, metal parts, cables, processing boards, as well as the software development. Such engineering might be viewed as the robot genotype. But equally important in a child is the phenotype, the developmental acquisition of information in a social and cultural context. Inspired in infant development, we aim at developing a robot's perceptual system through the use of learning aids, so that a robot learns about the world according to a child s developmental phases, by socially interacting with humans. The human caregiver plays a very important role on a robot s learning process (as it is so with children), performing educational and play activities with the robot (such as drawing, painting or playing with a toy train on a railway), facilitating robot s perception and learning. 1.2 Networked and Interactive Applications We aim at enculturating robots - introducing robots into our society and treating them as us - using child development as a metaphor for developmental learning on a robot. And by doing so, we developed a plethora of interactive applications that, whenever downloaded from a server platform to a robot, and installed on it, can provide humans with new levels of interactivity while communicating with the robot.

2 2. Machine Learning For an autonomous robot to be capable of developing and adapting to its environment, it needs to be able to learn. The field of machine learning offers many powerful algorithms, but these require training data to operate. Infant development research suggests ways to acquire such training data from simple contexts, and use this experience to bootstrap to more complex contexts. We need to identify situations that enable the robot to temporarily reach beyond its current perceptual abilities, giving the opportunity for development to occur [1]. This led us to create children-like learning scenarios for teaching a robot. These learning experiments are used for transmitting information to the humanoid robot Cog (see Figure 1) by interactive applications, to learn about object's multiple visual and auditory representations from books, other learning aids, musical instruments and education activities such as drawing and painting. Our strategy for the development of these applications relies heavily in human-robot interactions. It is essential to have a human in the loop to introduce objects from a book to the robot (as a human caregiver does to a child). A more rich, complete Figure 1 Cog going through many different social interactions with a human, in order to extract information and perform different tasks according to different interactive applications. through the caregiver s eyes. 2.1 Robot Skill Augmentation through Cognitive Artifacts human-robot communication interface results from adding other aiding tools to the robot's portfolio (which facilitate as well the children' learning process). This is achieved by having an application that selectively attends to the human actuator (Hand or Finger). Indeed, primates have specific brain areas to process the hand visual appearance [3]. Inspired by human development studies, emphasis will be placed on facilitating perception through the action of a human instructor. Multi-modal object properties are learned using these children educational tools and inserted into several recognition applications, which are then applied to developmentally acquire new object representations. The goal is for a robot, employing new interactive applications, to see the world A human caregiver can introduce a robot to a rich world of visual information concerning objects' visual appearance and shape. But cognitive artifacts can also be applied to improve perception over other perceptual modalities, such as auditory processing. We exploit repetition (rhythmic motion, repeated sounds) to achieve segmentation and recognition across multiple senses. Hence, we aim at detecting conditions that repeat with some roughly constant rate, where that rate is consistent with what a human can easily produce and perceive. This is not a very well defined range, but we will consider anything above 10Hz to be too fast, and anything below 0.1Hz to be too slow. Repetitive signals in this range are considered to be events in our system: waving a flag is an event, but the vibration of a violin string is not an event (too fast), and neither is the daily rise and fall of the sun (too slow). Abrupt motions, such as a poking movement, which involve large variations of movement, are

3 also used to extract percepts [1,4]. Such restrictions are related to the idea of natural kinds, where perception is based on the physical dimensions and practical interests of the observer. 3. Teaching Applications using Books and Toys Learning aids are often used by human caregivers to introduce the child to a diverse set of (in)animate objects, exposing the latter to an outside world of colors, forms, shapes and contrasts, that otherwise might not be available to a child (such as images of whales and cows). Since these learning aids help to expand the child's knowledge of the world, they are a potentially useful tool for introducing new informative percepts to a robot. Children's learning is often aided by the use of audiovisuals, and especially books, from social interactions with their mother or caregiver. Indeed, humans often paint, draw or just read books to children during their childhood. Books are indeed a useful tool to teach robots different object representations and to communicate properties of unknown objects to them. 3.1 Learning Book Images A human aided perceptual grouping application acquires informative percepts from picture Figure 2 - Object templates extracted from books. books (made of different materials: fabric, cardboard or foam books experimental results for regular books shown on Figure 2) by tracking a periodically moving human actuator (tapping finger), to extract the visual appearance of objects from background pages, as shown in Figure 3, and applied as follows [5]: Stationary image 1. Color segmentation of stationary Object mask image (over a sequence of consecutive frames). 2. Human actor waves on top of the 1 4 object to be segmented. Motion of skin-tone pixels tracked over a time interval, Energy per frequency Color Segmentation 3 content is determined for each trajectory point 3. Periodic, skin-tone points are grouped together into the finger 2 Periodicity detection mask. Figure 3 Human-aided perceptual grouping algorithm.

4 4. Target object's template given by union of all color regions of the stationary image which intersect the finger s trajectory. Teaching robots from books may become an interesting market application. The software required is computationally inexpensive, and it could easily be incorporated on a micro-chip. Integration of such technology with cross-modal association (which is also computationally inexpensive) is also possible for introducing further applications. Just imagine a child showing a tree from a book to a baby robot (or a Sony AIBO robot), while saying tree. The posteriori visual perception of the tree would enact the production of the sound tree by the robot, or listening to the repetitive sound tree would trigger visual search behaviors for such object. 3.2 Matching Geometric Patterns: Drawings, Paintings, Pictures... Object descriptions may came in different formats - drawings, paintings, photos, etc. Hence, the link between an object representation in a book and real objects recognized from the surrounding world is established through object recognition. Objects are recognized using geometric hashing, a widely used recognition technique. The algorithm operates on three different set of features: chrominance and luminance topological regions, and shape [1] (determined by an object s edges). Except for a description contained in a book, which was previously segmented, the robot had no other knowledge concerning the visual appearance or shape of such object. Additional possibilities include linking different object descriptions in a book, such as a drawing, as demonstrated also by results presented in Figure 4. A sketch of an object contains salient features concerning its shape, and therefore there are advantages to learning, and linking, these different representations. This framework is also a useful tool for linking other object descriptions in a book, such as a photo, a painting, or a printing [1]. 3.3 Toys Applications A plethora of educational tools are widely used by educators to teach children, helping them to develop. Examples of such tools are toys (such as drawing boards), educational TV programs or educational videos. The Baby Einstein collection includes videos to introduce infants and toddlers to colors, music, literature and art. Famous painters and their artistic creations are displayed to children on the Baby Van Gogh video, from the mentioned collection. This inspired the design of learning experiments in which a robot is introduced Figure 4 - Matching objects from books to real world objects and drawings. Figure 5 - The image of a painting by Vincent Van Gogh, Road with Cypress and Star, 1890 is displayed on a computer screen. Paintings are contextually different than pictures or photos, since the painter style changes the elements on the figure considerably. Van Gogh, a post-impressionist, painted with an aggressive use of brush strokes. But individual painting elements can still be grouped together by having a human actor tapping on their representation in the computer screen to group them together.

5 to art using an artificial display (a computer monitor) [1]. The image of a painting by Vincent Van Gogh, Road with Cypress and Star, 1890 is displayed on a computer screen. Paintings are contextually different than pictures or photos, since the painter style changes the elements on the figure considerably. Van Gogh, a post-impressionist, painted with an aggressive use of brush strokes. But individual painting elements can still be grouped together by having a human actor tapping on their representation in the computer screen to group them together. Other videos, such as the humanoid robot Cog sawing a piece of wood, were also shown to the robot, from which visual templates of objects were extracted. Drawing boards are also very useful to design geometric shapes while interacting with a child. 4. Educational, Learning Applications A common pattern of early human-child interactive communication is through activities that stimulate the child's brain, such as drawing or painting. Children are able to extract information from such activities while they are being performed on-line. This capability motivated the implementation of three parallel processes which receive input data from three different sources: from an attentional tracker, which tracks the attentional focus and is attracted to a new salient stimulus; from a multi-target tracking algorithm implemented to track simultaneously multiple targets; and from an algorithm that selectively attends to the human actuator. 4.1 Learning Hand Gestures Standard hand gesture recognition algorithms require an annotated database of hand gestures, built off-line. Common approaches, such as Space-Time Gestures [6], rely on dynamic programming. Others [7] developed systems for children to interact with lifelike characters and play virtual instruments by classifying optical flow measurements. Other classification techniques include state machines, dynamic time warping or Hidden Markov Models. We follow a fundamentally different approach, being periodic hand trajectories mapped into geometric descriptions of these objects Figure 6 reports an experiment in which a human draws repetitively a geometric shape on a sheet of paper with a pen. The robot learns what was drawn by matching one period of the hand gesture to the previously learned shape (the hand gesture is recognized as circular). Hence, the geometry of periodic hand trajectories are on-line recognized to the geometry of objects in an object database, instead of being mapped to a database of annotated gestures. 4.2 Object Recognition from Hand Gestures The problem of recognizing objects in a scene can be framed as the dual version of the hand gestures recognition problem. Instead of using previously learned object geometries to recognize hand gestures, hand gestures' trajectories are now applied to recover the geometric shape (defined by a set of lines) and appearance (given by an image template enclosing such lines) of a scene object (as seen by the robot). Visual geometries in a scene (such as circles) are recognized as such from hand gestures having the same geometry (as is the case of circular gestures). Figure 6 shows results for a set of tasks. The robot learns what was painted by matching the hand gesture to the shape defined by the ink on the paper. This algorithm is useful to identify shapes from drawing, painting or other educational activities. 4.3 Shape from Human Cues This same framework is applied to extract object boundaries from human cues. Indeed, human manipulation provides the robot with extra perceptual information concerning objects, by actively describing (using human arm/ hand/finger trajectories) object contours or the hollow parts of objects, such as a cup (Figure 6). Tactile perception of objects from the robot grasping activities has been actively pursued [8]. Although more precise, these techniques require hybrid position/ force control of the robot's manipulator end-effector so as not to damage or break objects.

6 Figure 6 - Sample of experiments for object and shape recognition from hand gestures. 4.4 Functional Constraints Not only hand gestures can be used to detect interesting geometric shapes in the world as seen by the robot. For instance, certain toys, such as trains, move periodically on rail tracks, with a functional constraint fixed both in time and space. Therefore, one might obtain information concerning the rail tracks by observing the train's visual trajectory. To accomplish such goal, objects are visually tracked by an attentional tracker which is modulated by an attentional system [1]. The algorithm starts by masking the input world image to regions inside the moving object's visual trajectory (or outside but near the boundary). Lines modelling the object's trajectory are then mapped into lines fitting the scene edges. The output is the geometry of the stationary object which is imposing the functional constraint on the moving object. Figure 6 shows experimental results for the specific case of extracting templates for train rail tracks from the train's motion (which is constrained by the railway circular geometry). 4.5 Language Learning Applications Auditory processing is also integrated with visual processing to extract the name and properties of objects. However, hand visual trajectory properties and sound properties might be independent - while tapping on books, it is not the interacting human caregiver hand that generates sound, but the caregiver vocal system pronouncing sounds such as the object s name. Therefore, cross-modal events are associated together under a weak requirement: visual segmentations from periodic signals and sound segmentations are bound together if occurring temporally close [4]. This strategy is also well suited for sound patterns correlated with the hand visual trajectory (such as playing musical tones by shaking a rattle).

7 5. Cross-Modal Sensing Applications Different objects have distinct acoustic-visual patterns which are a rich source of information for object recognition, if we can recover them. The relationship between object motion and the sound generated varies in an object-specific way. A hammer causes sound after striking an object. A toy truck causes sound while moving rapidly with wheels spinning; it is quiet when changing direction (see Figure 7). These statements are truly cross-modal in nature. Features extracted from the visual and acoustic segmentations are what is needed to build an object recognition system [4]. The feature space for recognition consists of: Sound/Visual period ratios Figure 7 - The car and the cube, both moving, both making noise. The line overlaid on the spectrogram (bottom) shows the cutoff the sound energy of a hammer determined automatically between the high-pitched bell in the peaks once per visual period, cube and the low-patched rolling sound of the car. The frequencies while the sound energy of a of both visual signals are half those of the audio signals. car peaks twice. Features: sound/vision peak energy and period ratios Visual/Sound peak energy ratios 4 the hammer upon impact creates high peaks of sound energy relative 2 to the amplitude of the visual Car trajectory. Hammer Cube rattle Dynamic programming is applied to 0 Snake rattle match the sound energy to the visual trajectory signal. Formally, let -2 = ( S ) and V ( V ) S 1 S n = 1 V m be sequences of sound and visual trajectory energies segmented from n and m periods of the sound and visual trajectory signals, respectively. Due to noise, n may be different to m. If the estimated sound period is half the visual one, then V corresponds to energies segmented with 2m half periods (given by the distance between maximum and minimum peaks). = 1 P l defines an alignment between S and V, where max( m, n) 1 m + n 1, and P k = ( i, j), a match k between sound cluster j and visual cluster i. The matching constraints are set by: P 1,1 A matching path P ( P ) The boundary conditions: 1 = ( ) and P l = ( m, n). Temporal continuity: P k + 1 i + 1, j + 1, i + 1, j, i, j + 1 Steps are adjacent elements of P. [( ) ( ) ( )] Log(visual peak energy/acoustic peak energy) Log(acoustic period/visual period) Figure 8 - Object recognition from cross-modal clues. The confusion matrix for a four-class recognition experiment is shown. Objects are recognized based on cross-modal features.

8 The function cost c ij is given by the square difference between V i and S j periods. The best matching path W can be found efficiently using dynamic programming, by incrementally building an m n table caching the optimum cost at each table cell, together with the link corresponding to that optimum. The binding W will then result by tracing back through these links, as in the Viterbi algorithm. Figure 8 shows cross-modal features for a set of four objects. The system was evaluated by selecting randomly 10% of the data for validation, and the remaining data for training. This process was randomly repeated 15 times. The recognition rate averaged over all these runs were, by object category: 86.7%for the cube rattle, 100% for both the car and the snake rattle, and 83% for the hammer. The overall recognition rate was 92.1%. Such results demonstrate the potential for object recognition applications using cross-modal cues. 6. Networked Robotic Platform We have presented a collection of interactive applications for human-robot interactions. The architecture to deploy such applications over the network into robots is shown in Figure 9. Notice that applications may be deployed as distributed applications, having interactive components running on the robot, and other components running on foreign servers (or even on other robots or machines, such as on Peer2Peer systems [9]). Indeed, the learning modules are often more suitable to run on more porwerfull network machines, while live object recognition should run on the robot platform. In addition, network bandwidth should be appropriate, so that quality of service for such near-real time applications is assured. Selects Applications Web portal Download User Robot Application Server Applications Database Play game Applications Figure 9 Architecture for networked interactive applications. A user accesses a Web Portal and selects applications to be deployed at a robot. Such applications are stored on an application server (or cluster of network servers), and are transmitted through the internet. Although not shown explicitly, distributed applications are possible, by having application components running on internet servers (application server or others). Also possible is a P2P solution, in which some application components may even run on other robots hardware.

9 7. Other Potential (Networked) Applications This paper covered a collection of interactive applications that were implemented, and that can be deployed remotely from a server into a robot. But a larger number of interactive applications are possible, being presented hereafter an overview of future potential applications. 7.1 Human-Robot interaction through more complex mechanisms Our work could also be extended to account for multi-link object recognition for carrying out the identification of possibly parallel kinematic mechanisms (e.g., two pendulums balancing in a bar) over constrained kinematic mechanisms (e.g., a slide and crank mechanism) mechanisms constrained by pulleys, belts chain of rotating cranes or pendulums the different moving links of an object (e.g., car with wheels) or a combination of them. Such scheme could then be applied developmentally: for instance, by learning the kinematics of two rolling objects, such as two capstans connected by a cable, the robot might use such knowledge to determine a configuration of a capstan rolling around the other. 7.2 Processing of tactile information The addition of tactile sensors to a robot, enables the download of network applications to interact in reacher ways with a robot. On possibility is the cross-modal integration of this sensory modality with the other modalities. 7.3 Integration of parsing structures for language processing This paper describes methods to learn about sounds and first words. A babbling language was previously developed on the robot Cog at MIT CSAIL [10], as well as a grounded language framework to learn about activities [11]. The integration of these components would add flexibility for interacting with a robot. In addition, there are strong correlations between learning the execution of motor tasks and speech, which would be interesting to exploit. 7.4 Caregiven-Children Games This work could be extended by further employing learning by scaffolding from educational activities between a robot and a helping caregiver. Good examples of such future learning include games in which the caregiver position babies facing them in order to play roll the ball or peek-a-boo games [12]. Another interesting example occurs when caregivers help toddlers solving their first puzzles by a priori orienting pieces in the right direction. In each case the children receive very useful insights that help them learn their roles in these social interactions, making future puzzles and games easier to solve. Solving puzzles is therefore an interesting problem involving object recognition, feature integration and object representation, for which this paper framework could be extended. In addition, hand gestures recognition from non-repetitive gestures (but without hand gesture annotation) would be of interest, since a lot of information can be conveyed to a robot by the detection of human gesturing. An interesting possibility would be for the robot to play games with humans, or other robots - which would have as well to download the network application from a server platform (or employing Peer2Peer technology), through the internet, besides the physical human-robot interaction. 8. Conclusions Teaching a robot information concerning its surrounding world is a difficult task, which takes several years for a child, equipped with evolutionary mechanisms stored in its genes, to

10 accomplish. Learning aids are often used by human caregivers to introduce the child to a diverse set of (in)animate objects, exposing the latter to an outside world of colors, forms, shapes and contrasts, that otherwise could not be available to a child (such as the image of a Panda). A learning aid expands the child's knowledge of its surrounding world, and it is therefore a potentially useful tool to introduce new informative percepts to a robot. If in the future robots are to behave like humans, a promising venue to achieve this goal is by treating then as such, and initially as children. Learning aids such as books or educational activities that stimulate a child's brain are important tools that caregivers extensively apply to communicate with children. And we exploited such tools to develop interactive applications. There are already some companies providing remote applications to be downloaded into robots, such as Lego Mindstorms, among others. But such applications still lack complex levels of interactivity. This paper described work on interactive applications being targeted for human-robot interactions. But such applications are not static on their capabilities: by employing developmental learning strategies, as a robot interacts with humans, it learns new knowledge, which enables it to demonstrate richer levels of interactivity later on. References 1. A. Arsenio, Cognitive-Developmental Learning for a Humanoid Robot: A Caregivers' gift, Ph.D. thesis, MIT, May/June A. Arsenio. Intelligent Algorithms and Clever Applications in User Centric Network. Eds. J. Crowcroft, J. Kempf, P. Mendes, R. Sofia Abstracts Collection and Report: User- Centric Networking URL: 3. D. I. Perrett, A. J. Mistlin, M. H. Harries, A. J. Chitty, Understanding the visual appearance and consequence of hand action, in Vision and action: the control of grasping (Ablex, Norwood, NJ, 1990) P. Fitzpatrick, A. Arsenio, Feel the beat: using cross-modal rhythm to integrate robot perception (International Workshop on Epigenetic Robotics, 2004). 5. A. Arsenio, Teaching a humanoid robot from books, in International Symposium on Robotics (2004). 6. T. Darrel, A. Pentland, Space-time gestures, in IEEE Conference on Computer Vision and Pattern Recognition (New York, NY, 1993) R. Cutler, M. Turk, View-based interpretation of real-time optical flow for gesture recognition, in Int. Conference on Automatic Face and Gesture Recognition (1998). 8. K. Rao, G. Medioni, H. Liu, B. G.A., Shape description and grasping for robot hand-eye coordination, IEEE Control Systems Magazine 9 (1989) (2) R. Schollmeier, A Definition of Peer-to-Peer Networking for the Classification of Peer-to Peer Architectures and Applications, Proceedings of the First International Conference on Peer-to-Peer Computing, IEEE, P. Varchavskaia, P. Fitzpatrick, C. Breazeal, Characterizing and Processing Robot-Directed Speech. Proceedings of Second International Conference on Humanoid Robotics, Tokyo, Japan, P. Fitzpatrick, From First Contact to Close Encounters: A Developmentally Deep Perceptual System for a Humanoid Robot, PhD thesis, MIT, R. Hodapp, E. Goldfield, C. Boyatzis. The use and effectiveness of maternal scaffolding in mother-infant games. Child Development, 55: