Knowledge Representation and Cognition in Natural Language Processing Gemignani Guglielmo Sapienza University of Rome January 17 th 2013
The European Projects Surveyed the FP6 and FP7 projects involving robots. Out of such projects, selected those involving relevant speech, natural language processing and dialog aspects: CHRIS ITALK POETICON++ ALIZ- E JAST JAMES Iuro CogX HUMAVIPS
Projects Involving icub CHRIS, ITALK and POETICON ++ are the three european projects that involve the icub robot CHRIS: Playing with a toy ITALK: Recognizing objects POETICON ++: Stearing a cup of coffee
Projects Involving NAOs ALIZ- E and HUMANVIPS involve instead the NAO robots ALIZ- E: Move human- robot interaction from the range of minutes to the range of days. HAMAVIPS: Vision & Dialog (e.g. Multi Party Dialog, Gender Recognition, etc.)
Projects from Munich JAMES: Develop an artificial agent that supports socially appropriate and multi- party interaction. JAST: Human and robot joint- action.
The IURO Project The goal is to enable robots to navigate and emotionally interact in densely populated, unknown human- centered environments.
The COGX Project Cognitive Systems that Self- Understand and Self- Extend George: a system can efficiently acquire conceptual models in a learning process that is not too laborious and is performed in a user- friendly way. Dora: mobile robot that is motivated to fill gaps in her own knowledge; these gaps can be about the extent of space or the categories of rooms.
George Speaking The robot is asked to recognize and describe up to five objects in a table top scene. Initially the tutor drives the learning, but after a while, the robot is able to learn either without verbal feedback, or by asking for clarification when necessary. Tutor driven learning Tutor assisted learning Final Result H: Do you know what this is? G: No. H: This is a red object. G: Let me see. OK. G: Which color is this object? H: It is yellow. G: OK. G: Is this red? H: No. This is yellow. G: OK. H: Do you know what this is? G: It is a blue object. H: What shape is the red object? G: It is elongated. Attributes considered: four colours (red, green, blue, yellow) and two shapes (elongated, compact).
The Robot of the COGX Project It is a P3 mobile robot base with a custom- built superior structure. Components: Microphone and speakers Pan- tilt rig with 2x PointGrey Flea2 cameras Hokuyo laser URG- 04LX 270 degrees, 4 meters
The COGX System Architecture The COGX system architecture is based on the CoSy Architecture Schema, which is structured by sub- architectures (SAs). Different and parallel SAs have been created (e.g. vision, communication, navigation, manipulation, etc.) These SAs can be selectively grouped into a single architecture for a particular task.
A Different Way of Formulating Goals In order to provide the robots with a generic and extensible way to deal with different tasks, the computation and the coordination are treated as a planning problem. The use of planning gives the robot a high degree of autonomy. Complex goal- driven behaviors do not need to be hard- coded into the system, but are planned by the robot itself. However, relying on automated planning means that everything must be encoded as an action that the planner can understand.
Example When the robot is given the command Bring me the AI book the planner realizes that it first needs to know the location of the book. Thus, in the initial phases of the planning process, it will query SAs who can provide informations about the location of the book (e.g. a library). For planning HRIs: If a human is believed to know an object property, the robot will interrogate the human instead of querying SAs.
How Does Dora Work? DORA can be summarized in two basic components: The conceptual map: used probabilistic common sense knowledge to make connections between concepts.
The planning system: combines decision theoretic reasoning with fast classical continual planning. The planner does not create a full plan for every eventuality but computes one plan at a time and monitors its execution, replanning if the robot gets stuck. This system can also take probabilities into account by making assumptions. These assumptions will lead to higher costs of the resulting plan, thus making the system prefer plans that tend to rely on more likely facts.
Last Features Added Probabilistic knowledge under uncertain sensing Dora can search for objects (a box of cornflakes that was placed among many other objects belonging to the nine categories that the robot has been trained to detect). Recognition of context dependent spatial regions (e.g. the front of the classroom) CDSR can be defined using qualitative spatial representations corresponding to sensor data. For each object, Dora computes the strengths of 8 spatial relations between that object and each of the objects adjacent to it, using a Voronoi diagram. assumption: similar areas will feature similar CDSRs, and these similarities can be recognized through analogy.
Summarizing Analyzed the most relevant FP projects involving speech, natural language processing and dialog aspects. Out of these projects, identified the most noteworthy work: COGX. Studied the functioning of the robot and the most recent features added to it.