Human-Robot Interaction: A first overview

Transcription

1 Human-Robot Interaction: A first overview Pierre Lison Geert-Jan M. Kruijff Language Technology Lab DFKI GmbH, Saarbrücken

2 Preliminary Infos Schedule: First lecture on February 1st (Monday), 8:30-10:00 First overview of HRI: basic principles, issues, applications Second lecture on February 3rd (Wednesday), 8:30-10:00 Lecturers: Approaches to situated dialogue processing and management Pierre Lison [ plison@dfki.de ] & Geert-Jan M. Kruijff: [ gj@dfki.de ] Affiliation: Talking Robots Group, Language Technology Lab, DFKI Teaching material: The lecture slides (PDF) will be put online on the course website If needed, additional references can of course be provided 2

3 Talking Robots? Our long-term aim: 3

4 Talking Robots? Our long-term aim: Hi! I am C-3PO, human-cyborg relations. 3

5 Talking Robots? 4

6 Talking Robots? Our long-term aim: Hi! I am C-3PO, human-cyborg relations. 5

7 Talking Robots? Our long-term aim: Hi! I am C-3PO, human-cyborg relations. (and he speaks 6 million languages...) 5

8 Talking Robots? Our long-term aim: Hi! I am C-3PO, human-cyborg relations. (and he speaks 6 million languages...) For the time being, we ll have to scale down our expectations... 5

9 Talking Robots? Hi! I am C-3PO, human-cyborg relations. Aim: building service robots endowed with general communicative abilities (for instance, via spoken dialogue) Such robots could assist us in our daily life take care of routine tasks, in homes, offices, schools or hospitals help disabled or mentally impaired persons serve as helpers / social companions for the elderly or simply entertain us! 6

10 Talking Robots? Hi! I am C-3PO, human-cyborg relations. Research goal: building robotic systems which are able to interact with humans via spoken dialogue Understanding speech, deciding what to do next, producing an answer... with noise, uncertainty, and real-time constraints... and taking all the context into account! Question: how can we achieve this, given the current limitations of robotics and NLP technology? 7

11 Outline Introduction Short introduction to (cognitive) robotics Definitions, classifications Perception & action Cognitive architectures Human-Robot Interaction (HRI) Generalities Specificities of HRI Spoken dialogue systems for HRI Approach and challenges How it works Conclusion 8

12 Outline Introduction Short introduction to (cognitive) robotics Definitions, classifications Perception & action Cognitive architectures Human-Robot Interaction (HRI) Generalities Specificities of HRI Spoken dialogue systems for HRI Approach and challenges How it works Conclusion 9

13 What is a robot? A robot is a physical artificial agent It can perceive the world around him, via its sensors (camera, lasers, microphones, etc.) It can also act on its environment, via its actuators (usually motor-driven mechanical devices) Some robots are also able to interact with other agents (human or robotic) in their environment 10

14 What is a robot? Robots are typically developed to accomplish particular goals in their environment Most industrial robots today are either remotely controlled, or preprogrammed to execute only a fixed sequence of actions Others (so-called cognitive robots) integrate reasoning capabilities to cope with new situations 11

15 Types of robots Robots can be distinguished according to many categories and criteria One useful classification is based on the types of embarqued actuators: Static robots (e.g. robotic arms) have manipulation actuators Mobile robots have actuators for motion control And finally, hybrids combine both manipulation and motion (for instance, humanoids robots) Static robots Kuka robotic arm (LWR Leica T-Mac) Hybrid robots Mobile robots Big Dog from Boston Dynamics Underwater snake robot, H. Fukushima Lab Asimo humanoid robot from Honda 12

16 Interacting robots Another crucial factor is the number of interacting agents in the environment Multi-agent (collaborative or competitive) In order to be multi-agent, a robot must recognise others as intentional agents, with whom to cooperate or compete A special case is swarm intelligence, where a large number of agents in interaction create an emerging behaviour Single-agent Nao robots participating to RoboCup Swarm Nao robot (Aldebaran Robotics) Simple swarm robots 13

17 Challenges in robotics The goal: building cognitive robots able to make sense of their environment, and act rationally in it This is quite challenging! Why? We must deal with environments which are :... often large, noisy and complex! Partially observable (sensory information is noisy and incomplete) Stochastic (outcomes of actions are non-deterministic) Dynamic (the state of the world changes over time) Continuous (observation measures are real-valued) Sequential (current decision affects future actions) Multi-agents (the agent has to cope with other agents) 14

18 Robot sensors Sensors allow the robot to perceive its external environment They can be either passive (merely capture information) or active (send out a signal) Many different types: Imaging sensors (cameras for monocular or stereo vision) Sonar, laser (measure the distance to nearby objects) Tactile sensors (whiskers, bumps) Proprioceptive sensors (inform the robot of its own state) Force and torque sensors And many others (GPS, microphones, inertial sensors, etc.) 15

19 Robot perception Well-known examples of perception processes in robotics: localisation & mapping Localisation: find the current spatial position of the robot in the environment space, given the observations accumulated by the sensors (cameras and range finders) The initial pose of the robot is sometimes known (in which case localisation becomes a tracking problem), sometimes it isn t Well-known algorithm for this problem is the extended Kallman Filter Mapping: find the current position of other objects (walls, rooms, other landmarks) in the environment space Most difficult task: Simultaneous Localisation and Mapping in unknown environments (SLAM) 16

20 Robot perception Another example of perceptive process, which is crucial for HRI: speech understanding The task here is to derive the intented communicative act from a perceived spoken utterance Input is usually noisy and ambiguous Word-Error-Rates of 30 % or more are not uncommon Plus, speech understanding systems are often too brittle Pervasiveness of speech recognition errors Very limited coverage (both vocabulary and grammar) Lack of robustness and adaptation 17

21 Robot actuators Actuators allow the robot to act on its external environment to achieve its goal They are the muscles of the robot They typically take the form of mechanical devices that takes energy and converts it into motion Usual source of power: electric motor (but pneumatic or hydraulic actuation are also possible) Actuators are used for: Manipulation Locomotion 18

22 Robotic locomotion Wheeled robots Legged robots More examples of locomotion for mobile robots: PeopleBot from Mobile Robots (running our software) Flying robots Scorpion (DFKI Bremen) Underwater robots Each of them is adapted for a specific terrain or medium Microdrones GmbH Squid robot (Osaka University) 19

23 Cognitive Architectures Intelligent behaviour relies a combination of highly complex cognitive capacities. An intelligent system must be able to actively perceive its environment, reason about it, and achieve goals through plans and actions. Artificial cognitive systems must therefore encompass a large number of distributed and cooperating subsystems: icub humanoid robot RoboCup Consortium computer vision (perception, attention), navigation and manipulation skills (motor control), and various deliberative processes (reasoning, learning, planning). Leonardo: a sociable robot MIT Media Lab 20

24 Robot control Given some sensory information, how does the robot decides which action to perform? Model-based deliberative control: try to build a model of the environment and use it to plan the optimal action Often requires prior knowledge of the model structure Complex computations Allows for rich sequential decision-making Reactive control: derive the action directly, without prior model Direct connection between perceptive inputs and actions Faster, often more robust (sensor-driven) No planning or prediction involved 21

25 Robot control Each approach has its strengths and weaknesses Reactive control more appropriate for low-level decision-making in real-time, bottom-up perception Deliberative control is more appropriate for high-level decision-making to achieve long-term goals, top-down attention and control Development of hybrid cognitive architectures integrating both reactive & deliberative control Rely on sense-think-act models on top of a behaviour-based substrate. Low-level reactivity is therefore separated from higher level reasoning such as planning, reasoning, and learning Examples: Brooks subsumption architecture, three layered architecture Use the world itself as its own model [Brooks] 22

26 Information fusion Most robotic platform include several sensory modalities How can we merge these fragmented pieces of information into a single unified representation? This is the classical problem of information fusion, or binding A robotic platform must also be able to combine information arising from different knowledge sources (or modalities) and fuse them Belief A Vision label(a) = mug? Multimodal belief C Belief B P = 0.77 label(a) = mug? shape(a) = cylinder Haptic shape(a) = cylinder? P = 0.86 P = x? 23

27 Outline Introduction Short introduction to (cognitive) robotics Definitions, classifications Perception & action Cognitive architectures Human-Robot Interaction (HRI) Generalities Specificities of HRI Spoken dialogue systems for HRI Approach and challenges How it works Conclusion 24

28 What is human-robot interaction? Human-robot interaction is the field of study dedicated to understanding, designing, and evaluating robotic systems for use by or with humans [Goodrich] Recent but very fast-growing research area! The objective of HRI is to develop principles, techniques and algorithms to allow for natural and effective communication between humans and robots. HRI is a highly multidisciplinary research area, drawing from a wide range of fields such as artificial intelligence, engineering, robotics, (computational) linguistics, cognitive science, social psychology, human factors, and anthropology. 25

29 What is human-robot interaction? Communication in all its aspects Verbal- and non-verbal behaviors, including gesture, posture, affective display,... at various interaction ranges (proximal, distant), with reference to varying spatio-temporal contexts HRI in this lecture Focus on spoken dialogue, proximal interaction, varying spatial contexts 26

30 Application domains Personal services Performing routine tasks, providing information, ensuring personal security & safety At homes, offices, public places Education Educational companions in schools or at home Health care Helping disabled or mentally impaired persons Rehabilitation and elder care Exploration, search-and-rescue in hazardous or hostile environments Varying levels of autonomy and collaboration (Transportation, industrial and field robotics) 27

31 Symbol grounding Robots are agents with physical bodies In order to acquire an external meaning, linguistic symbols ultimately need to be grounded in other modalities Symbols must connect to perception and bodily experience For instance, the linguistic symbol door must somehow be connected to some prototypical image of a door, as well as to prototypical affordances (what can be typically done with a door, and how) Merleau-Ponty But humans and robots experience reality differently! Concept of embodiment: Agents with different bodies perceive and act in the world in different ways, and will also structure the world very differently Francisco Varela The embodied mind thesis: the nature of human cognition is largely shaped by the form of the human body [Merleau- Ponty, Varela, Lakoff, and many others] Question: since robots and humans view reality so differently, will true communication between them ever be possible? George Lakoff 28

32 Situated Interaction Another important aspect of HRI compared to other kind of human-machine interaction is its situatedness Human-robot interaction is typically situated The robot and the human are physically situated in a shared environment They can see each other, and jointly attend to each other They can directly refer to entities or events in the shared scene (via pointing, deictics, presuppositions, etc.) Leonardo: a sociable robot MIT Media Lab To understand situated dialogue, it is therefore also necessary to understand the situated reality Exception: remote interaction (for instance, spatial robotics) 29

33 Situated Interaction (2) Teaching the robot about new objects, Playing games on a table top... Showing a robot around the house Spatio-temporal framing Spoken dialogue in our case is often referential to aspects of the environment The environment may be the small-scale space, e.g. a table top, an area we are in, but may also concern large-scale space, going beyond what is currently visible. It can also refer to past, future or conditional events ( if you open the box, you will find... ) Describing what kind of object it should be looking for, in some other location, And trying to ask someone how to get to that location. 30

34 Situated Interaction (3) Appropriate awareness of the situated context and its associated bodily experience are crucial to: Guide spoken dialogue comprehension and production (priming effects) Establish expectations about what is likely to come next (anticipation and prediction) Determine the current focus of attention (saliency, foregrounding/ backgrounding, etc.) Language processing cannot be isolated from its environment and from other cognitive processes Rather, it should be seen as essentially an embodied and situated activity (emphasis on all three) rather than a formal system operating on abstract symbols We should also add: a socially situated activity, since social behaviours drive much of language use 31

35 Non-verbal interaction Non-verbal interaction plays an important role in HRI Talking robots should perceive the body language of their interlocutor, and use their own body for interacting as well Both non-verbal language understanding and production Humans will naturally expect the robot to behave in such a way, and will perceive the interaction as awkward if it doesn t Kismet, MIT Media Lab Dexter, MIT Media Lab 32

36 Taking stock Introduction Short introduction to (cognitive) robotics Definitions, classifications Perception & action Cognitive architectures Human-Robot Interaction (HRI) Generalities Specificities of HRI Spoken dialogue systems for HRI Approach and challenges How it works Conclusion 33

37 Processing modules 34

38 Some problems to tackle The usual for spoken dialogue in HRI: Pervasiveness of speech recognition errors Partial, fragmentary, ungrammatical utterances, Presence of many disfluencies (filled pauses, speech repairs, corrections, etc.) Limited grammar coverage Ambiguity resolution at all processing levels Uncertainty in contextual interpretation of utterances Performance requirements for real-time dialogue The system must be capable of responding quickly to any utterance, even in the presence of noisy, ambiguous, or distorted input 35

39 Disfluencies in Spoken Dialogue Extract from a corpus of task-oriented spoken dialogue: The Apollo Lunar Surface Journal. Parker : That s all we need. Go ahead and park on your 045 <okay>. We ll give you an update when you re done. Cernan : Jack is [it] worth coming right there? Schmitt : err looks like a pretty go/ good location. Cernan : okay. Schmitt : We can sample the rim materials of this crater. (Pause) Bob, I m at the uh south uh let s say east-southeast rim of a, oh, 30-meter crater - err in the light mantle, of course - up on the uh Scarp and maybe (correcting himself) err 200 meters from the uh rim of Lara in (inaudible) northeast direction. 36

40 Disfluencies in Spoken Dialogue Extract from a corpus of task-oriented spoken dialogue: The Apollo Lunar Surface Journal. Parker : That s all we need. Go ahead and park on your 045 <okay>. We ll give you an update when you re done. Cernan : Jack is [it] worth coming right there? Schmitt : err looks like a pretty go/ good location. Cernan : okay. Schmitt : We can sample the rim materials of this crater. (Pause) Bob, I m at the uh south uh let s say east-southeast rim of a, oh, 30-meter crater - err in the light mantle, of course - up on the uh Scarp and maybe (correcting himself) err 200 meters from the uh rim of Lara in (inaudible) northeast direction. 36

41 Insights from psycholinguistics Can we draw inspiration from how humans process dialogue to improve language processing? In visually situated dialogue, there is a close (bidirectional) coupling between how humans understand what they see, and what they hear This coupling is closely time-locked, as evidenced by Empirical analyses of saccadic eye movements in visual scenes [Knoeferle & Crocker, 2006] Neuroscience studies of event-related brain potentials (ERPs) [Van Berkum 2004] 37

42 Insights from psycholinguistics Can we draw inspiration from how humans process dialogue to improve language processing? In visually situated dialogue, there is a close (bidirectional) coupling between how humans understand what they see, and what they hear This coupling is closely time-locked, as evidenced by Empirical analyses of saccadic eye movements in visual scenes [Knoeferle & Crocker, 2006] Neuroscience studies of event-related brain potentials (ERPs) [Van Berkum 2004] At each processing step, exploit the situated context to predict, select, refine, extend, complement the interpretations 37

43 How it works 38

44 How it works 38

45 How it works A robot... 38

46 How it works A robot... 38

47 How it works A robot and a human 38

48 How it works A robot and a human 38

49 How it works A robot and a human 38

50 How it works A robot and a human 38

51 How it works A robot and a human in a shared visual scene 38

52 How it works The robot s observations of the environment are noisy and uncertain A robot and a human in a shared visual scene 38

53 How it works The robot s observations of the environment are noisy and uncertain err... now... robot, take the red cylinder please! A robot and a human in a shared visual scene 38

54 How it works He detects that the human is speaking, and seeks to decode the utterance The robot s observations of the environment are noisy and uncertain err... now... robot, take the red cylinder please! A robot and a human in a shared visual scene 38

55 How it works He detects that the human is speaking, and seeks to decode the utterance (which is also noisy and uncertain) The robot s observations of the environment are noisy and uncertain err... now... robot, take the red cylinder please! A robot and a human in a shared visual scene 38

56 How it works He detects that the human is speaking, and seeks to decode the utterance (which is also noisy and uncertain) The robot s observations of the environment are noisy and uncertain err... now... robot, take the red cylinder please! A robot and a human in a shared visual scene 38

57 How it works The goal is to find the intention behind the utterance He detects that the human is speaking, and seeks to decode the utterance (which is also noisy and uncertain) The robot s observations of the environment are noisy and uncertain err... now... robot, take the red cylinder please! A robot and a human in a shared visual scene 38

58 How it works The goal is to find the intention behind the utterance (including references to the external environment) He detects that the human is speaking, and seeks to decode the utterance (which is also noisy and uncertain) The robot s observations of the environment are noisy and uncertain err... now... robot, take the red cylinder please! A robot and a human in a shared visual scene 38

59 How it works The goal is to find the intention behind the utterance (including references to the external environment) He detects that the human is speaking, and seeks to decode the utterance (which is also noisy and uncertain) The robot s observations of the environment are noisy and uncertain err... now... robot, take the red cylinder please! A robot and a human in a shared visual scene 38

60 How it works (2) Yes sure, will do! err... now... robot, take the red cylinder please! A robot and a human in a shared visual scene 39

61 How it works (2) Yes sure, will do! Once the most probable intention has been computed, the robot must respond to it err... now... robot, take the red cylinder please! A robot and a human in a shared visual scene 39

62 How it works (2) Yes sure, will do! Once the most probable intention has been computed, the robot must respond to it err... now... robot, take the red cylinder please! A robot and a human in a shared visual scene 39

63 How it works (2) It searches for the best action(s) to perform at this point Yes sure, will do! Once the most probable intention has been computed, the robot must respond to it err... now... robot, take the red cylinder please! A robot and a human in a shared visual scene 39

64 How it works (2) It searches for the best action(s) to perform at this point Yes sure, will do! Once the most probable intention has been computed, the robot must respond to it err... now... robot, take the red cylinder please! A robot and a human in a shared visual scene 39

65 How it works (2) It searches for the best action(s) to perform at this point For instance, a communicative action, which generates an utterance Yes sure, will do! Once the most probable intention has been computed, the robot must respond to it err... now... robot, take the red cylinder please! A robot and a human in a shared visual scene 39

66 How it works (2) It searches for the best action(s) to perform at this point For instance, a communicative action, which generates an utterance Yes sure, will do! Once the most probable intention has been computed, the robot must respond to it err... now... robot, take the red cylinder please! A robot and a human Or a physical action which changes the environment in a shared visual scene 39

67 How it works (2) It searches for the best action(s) to perform at this point For instance, a communicative action, which generates an utterance Yes sure, will do! Once the most probable intention has been computed, the robot must respond to it err... now... robot, take the red cylinder please! A robot and a human Or a physical action which changes the environment in a shared visual scene 39

68 How it works (3) Yes sure, will do! err... now... robot, take the red cylinder please! in a shared visual scene A robot and a human 40

69 How it works (3) The robot and the human are both involved in a collaborative activity Yes sure, will do! err... now... robot, take the red cylinder please! in a shared visual scene A robot and a human 40

70 How it works (3) The robot and the human are both involved in a collaborative activity Yes sure, will do! err... now... robot, take the red cylinder please! in a shared visual scene A robot and a human 40

71 How it works (3) The robot and the human are both involved in a collaborative activity The dialogue is part of this activity Yes sure, will do! err... now... robot, take the red cylinder please! in a shared visual scene A robot and a human 40

72 How it works (3) The robot and the human are both involved in a collaborative activity The dialogue is part of this activity Yes sure, will do! err... now... robot, take the red cylinder please! in a shared visual scene A robot and a human 40

73 How it works (3) The robot and the human are both involved in a collaborative activity The dialogue is part of this activity For the interaction to be successful, the participants must establish a common ground Yes sure, will do! err... now... robot, take the red cylinder please! in a shared visual scene A robot and a human 40

74 How it works (3) The robot and the human are both involved in a collaborative activity The dialogue is part of this activity For the interaction to be successful, the participants must establish a common ground Yes sure, will do! err... now... robot, take the red cylinder please! in a shared visual scene A robot and a human 40

75 How it works (3) The robot and the human are both involved in a collaborative activity The dialogue is part of this activity Overall, the robot s actions must reflect: - the goals of the activity (what is to be done), - its history (what has been done) - the environment state (what s my reality) - models of the other agents (what s their reality) - the attentional state (what is the current focus) For the interaction to be successful, the participants must establish a common ground Yes sure, will do! err... now... robot, take the red cylinder please! in a shared visual scene A robot and a human 40

76 Overview of our system the interpretation 41

77 Overview of our system (2) the interpretation Speech recognition with statistical models Incremental parsing with Combinatory Categorial Grammar Dialogue interpretation tasks: reference resolution, dialogue move recognition, etc. 42

78 Overview of our system (2) ASR Parsing Dialogue int. the interpretation Speech recognition with statistical models Incremental parsing with Combinatory Categorial Grammar Dialogue interpretation tasks: reference resolution, dialogue move recognition, etc. 42

79 Overview of our system (3) ASR Parsing Dialogue int. Speech recognition outputs a word lattice the interpretation Word lattice = set of alternative recognition hypotheses compacted in a directed graph The CCG parser takes a word lattice as input and outputs partial semantic representations (logical forms) Logical forms are expressed as ontologically richly sorted, relational structures Dialogue interpretation based on dialogue structure 43

80 Overview of our system (3) ASR Parsing Dialogue int. word lattice parses packed LFs the interpretation Speech recognition outputs a word lattice Word lattice = set of alternative recognition hypotheses compacted in a directed graph The CCG parser takes a word lattice as input and outputs partial semantic representations (logical forms) Logical forms are expressed as ontologically richly sorted, relational structures Dialogue interpretation based on dialogue structure 43

81 Overview of our system (4) ASR Parsing Dialogue int. word lattice parses packed LFs the interpretation Use of contextual information at every processing step: To prime the speech recognition process [Lison & Kruijff, 2008] To select most likely syntactic analyses And to bind linguistic meaning with extra-linguistic knowledge about the environment, in a cross-modal way 44

82 Overview of our system (4) Context: possibility to link to visual objects ASR Parsing Dialogue int. word lattice parses packed LFs the interpretation Use of contextual information at every processing step: To prime the speech recognition process [Lison & Kruijff, 2008] To select most likely syntactic analyses And to bind linguistic meaning with extra-linguistic knowledge about the environment, in a cross-modal way 44

83 Overview of our system (4) Context: salient visual, discourse, and planning content Context: possibility to link to visual objects ASR Parsing Dialogue int. word lattice parses packed LFs the interpretation Use of contextual information at every processing step: To prime the speech recognition process [Lison & Kruijff, 2008] To select most likely syntactic analyses And to bind linguistic meaning with extra-linguistic knowledge about the environment, in a cross-modal way 44

84 Overview of our system (4) Context: salient visual, discourse, and planning content Context: possibility to link to visual objects ASR Parsing Dialogue int. LM priming word lattice parses packed LFs the interpretation Use of contextual information at every processing step: To prime the speech recognition process [Lison & Kruijff, 2008] To select most likely syntactic analyses And to bind linguistic meaning with extra-linguistic knowledge about the environment, in a cross-modal way 44

85 Overview of our system (4) Context: salient visual, discourse, and planning content Context: salient visual, discourse, and planning content Context: possibility to link to visual objects ASR Parsing Dialogue int. LM priming word lattice parses packed LFs the interpretation Use of contextual information at every processing step: To prime the speech recognition process [Lison & Kruijff, 2008] To select most likely syntactic analyses And to bind linguistic meaning with extra-linguistic knowledge about the environment, in a cross-modal way 44

86 Overview of our system (4) Context: salient visual, discourse, and planning content Context: salient visual, discourse, and planning content Context: possibility to link to visual objects ASR Parsing Dialogue int. LM priming context-sensitive parse selection word lattice parses packed LFs the interpretation Use of contextual information at every processing step: To prime the speech recognition process [Lison & Kruijff, 2008] To select most likely syntactic analyses And to bind linguistic meaning with extra-linguistic knowledge about the environment, in a cross-modal way 44

87 Overview of our system (4) Context: salient visual, discourse, and planning content Context: salient visual, discourse, and planning content Context: possibility to link to visual objects ASR Parsing Dialogue int. LM priming context-sensitive parse selection word lattice parses packed LFs the interpretation Use of contextual information at every processing step: To prime the speech recognition process [Lison & Kruijff, 2008] To select most likely syntactic analyses And to bind linguistic meaning with extra-linguistic knowledge about the environment, in a cross-modal way 44

88 Overview of our system (5) Context: salient visual, discourse, and planning content Context: salient visual, discourse, and planning content Context: possibility to link to visual objects ASR Parsing Dialogue int. LM priming context-sensitive parse selection In three keywords: word lattice parses packed LFs the interpretation Hybrid: Integration of fine-grained linguistic ressources with statistical models to deliver both deep and robust dialogue processing Integrated: goes all the way from the speech signal up to the semantic and pragmatic interpretation Context-sensitive: context is used at every processing step to drive the comprehension process (anticipation and discrimination mechanisms) 45

89 Architectural viewpoint 46

90 Taking stock Introduction Short introduction to (cognitive) robotics Definitions, classifications Perception & action Cognitive architectures Human-Robot Interaction (HRI) Generalities Specificities of HRI Spoken dialogue systems for HRI Approach and challenges How it works Conclusion 47

91 Conclusions So what have we seen today? 1. Robots are physical agents which can perceive their environment and act on it to achieve their goals 2. Research in HRI seeks to develop robots endowed with communicative abilities to interact with humans 3. To this end, robots need to be able to: Be aware of their surrounding context Ground the linguistic symbols in the external reality 4. Natural spoken dialogue can be particularly noisy and difficult to process (partial, fragmentary or distorted inputs) 5. Spoken dialogue systems for HRI must incorporate a complex pipeline including speech recognition, speech understanding, dialogue management, output production and text-to-speech 48

92 There are many issues we haven t (yet) touched upon For instance, how to develop robotic systems which can learn particular behaviours from experience Or what are precisely the frameworks and algorithms which can be used to meet the general requirements we outlined 49

93 That s all for today! Thanks for your attention! Questions anyone? For more information on our research, you can of course check our website: 50