Planning for Human-Robot Teaming Challenges & Opportunities

Transcription

1 for Human-Robot Teaming Challenges & Opportunities Subbarao Kambhampati Arizona State University Thanks Matthias HRI Lab [Funding from ONR, ARO J ] 1 [None (yet?) from NSF L ]

2 2

3 Two Great Societies Separated by a Single Problem 3

4 Two Great Societies Separated by a Single Problem 4

5 Robots as Remote Sensors/ Effectors Most applications of Robots view them as glorified remote sensors/effectors The role of planning here is mostly limited to path and manipulator planning Motivation 5

6 ROBOT Path/Motion Manipulator

7 My very first planning paper was a Path paper..

8 Motivation Robots as Remote Sensors/ Effectors Most applications of Robots view them as glorified remote sensors/effectors The role of planning here is mostly limited to path and manipulator planning Not that there is anything wrong with that.. Robots as full-fledged Teammembers Increasing number of applications want the robots to be full-fledged team members Teaming significantly broadens the roles for planning Need to take high-level goals from team members and plan for them 8

9 Case Study Urban Search and Rescue Human-Robot Team in Urban Setting Find and report location of critical assets Human: Domain expert; removed from the scene SEARCH AND REPORT Deliver medical supplies Bonus Goal: Find and report injured humans Requirements Updates to knowledge base Goal changes [Talamadupula et. al., AAAI 2010] RECONNAISSANCE Gather information High risk to humans E.g. Bomb defusal Requirements Support model changes New capabilities E.g.: Zoom camera

10 Human-Robot Teaming Scenarios Ø Search and report (rescue) Ø Goals incoming on the go Ø World is evolving Ø Model is changing Ø Infer instructions from Natural Language Ø Determine goal formulation through clarifications and questions [NIPS 2013; HRI 2012 AAAI 2010 ] 10

11 Path/Motion Manipulator HUMAN ROBOT

12 Belief Modeling Intent Recognition Activity Recognition Path/Motion Manipulator Task PDDL Instructions Goals Δ-PDDL Model Updates OWQG Dialog Trajectory Constraints Hypotheticals LTL HUMAN Questions Negotiation Affect ROBOT Reports Excuses Active Model Elicitation Replanning Open World Excuse Generation 12

13 Position Statement Human-Robot teaming scenarios significantly broaden the roles of planning beyond path and motion planning Task Belief Modeling Dialog planning We advocate investigating the challenges posed by all these planning roles Humans?

14 Belief Modeling Intent Recognition Activity Recognition Path/Motion Manipulator Task PDDL Instructions Goals Δ-PDDL Model Updates OWQG Dialog Trajectory Constraints Hypotheticals LTL HUMAN Questions Negotiation Affect ROBOT Reports Excuses Active Model Elicitation Replanning Open World Excuse Generation 14

15 : Traditional View A fully specified problem --Initial state --Goals (each non-negotiable) --Complete Action Model The Plan Hard problem à Tremendous progress has been made in taming the combinatorics 15

16 : Traditional View A fully specified problem --Initial state --Goals (each non-negotiable) --Complete Action Model The Plan 16

17 : Traditional View Partial models and partial specfications Changing goals Open goals Replanning A fully specified problem --Initial state --Goals (each non-negotiable) --Complete Action Model The Plan Explanation of Failures 17

18 for Human-Robot Teaming Planner is an intermediary between Human and Robot Two main tasks Process Information Changes to the world / state: Replanning Changes to the goals: Open World Quantified Goals Changes to the model: Run-time Model Updates Elicit Information Ask for advice / clarification Explain plans and make excuses / hypotheticals 18

19 HRT System Schematic Instructions Dialog HUMAN ROBOT 19

20 HRT System Schematic Task Instructions Dialog Goals Model Updates Trajectory Constraints Hypotheticals PDDL Δ-PDDL OWQG LTL Reports Excuses HUMAN ROBOT Active Model Elicitation 20

21 Goal Management Human-Robot Teaming Utility stems from delegation of goals Support different types of goals Temporal Goals: Deadlines Priorities: Rewards and Penalties Bonus Goals: Partial Satisfaction Trajectory Goals Conditional Goals Changes to goals on the fly Open World Quantified Goals [Talamadupula et al., AAAI 2010]

22 Model Management One true model of the world Robot High + Low Level models Human User Symbolic model + Additional knowledge Planner must take this gap into account Model Maintenance v. Model Revision Usability v. Consistency issues Use the human user s deep knowledge Distinct Models Using two (or more) models Higher level: Task-oriented model Lower level: Robot s capabilities Robot MODEL Human

23 HRT System Schematic Task Instructions Dialog Goals Model Updates Trajectory Constraints Hypotheticals PDDL Δ-PDDL OWQG LTL Reports Excuses HUMAN ROBOT Active Model Elicitation 23

24 HRT System Schematic Task Instructions Dialog Goals Model Updates Trajectory Constraints Hypotheticals PDDL Δ-PDDL OWQG LTL HUMAN ROBOT Reports Excuses Active Model Elicitation Replanning Open World Excuse Generation 24

25 Excuses & Hypotheticals Excuse Generation Make excuses if task unsolvable Changes to planning task Initial State [Goebelbecker et al. 2010] Goal Specification Operators [Cantrell, Talamadupula et al. 2011] Hypotheticals Goal opportunities Conditional Goals [Talamadupula, Benton et al. 2010] 25

26 Explanations Asking for help Proactively request humans for help Take navigation paths into account Explanations Returning a plan is not enough Human must be informed why the robot is doing something May result in more elaboration /information 26

27 HRT System Schematic Task Instructions Dialog Goals Model Updates Trajectory Constraints Hypotheticals PDDL Δ-PDDL OWQG LTL HUMAN ROBOT Reports Excuses Active Model Elicitation Replanning Open World Excuse Generation 27

28 HRT System Schematic Task PDDL Instructions Goals Δ-PDDL Model Updates OWQG Dialog Trajectory Constraints Hypotheticals LTL HUMAN Questions Negotiation Affect ROBOT Reports Excuses Active Model Elicitation Replanning Open World Excuse Generation 28

29 Dialog Most natural form of communication between Human and Robot: NL Dialog Human-to-Robot Instructions: Model updates [Cantrell et al. 2011] Objectives: Goal changes Robot-to-Human Questions Negotiation Affect 29

30 HRT System Schematic Task PDDL Instructions Goals Δ-PDDL Model Updates OWQG Dialog Trajectory Constraints Hypotheticals LTL HUMAN Questions Negotiation Affect ROBOT Reports Excuses Active Model Elicitation Replanning Open World Excuse Generation 30

31 HRT System Schematic Belief Modeling Intent Recognition Activity Recognition Task PDDL Instructions Goals Δ-PDDL Model Updates OWQG Dialog Trajectory Constraints Hypotheticals LTL HUMAN Questions Negotiation Affect ROBOT Reports Excuses Active Model Elicitation Replanning Open World Excuse Generation 31

32 Belief Modeling Humans communicate via task-based dialog For team situations, model team members Expect robots to do the same Example: When Commander Y interrupts Cindy the robot with a directive for later, Cindy must model Commander Y s mental state in order to define that goal Belief Updates Take utterances from humans and update 32

33 HRT System Schematic Belief Modeling Intent Recognition Activity Recognition Task PDDL Instructions Goals Δ-PDDL Model Updates OWQG Dialog Trajectory Constraints Hypotheticals LTL HUMAN Questions Negotiation Affect ROBOT Reports Excuses Active Model Elicitation Replanning Open World Excuse Generation 33

34 Belief Modeling Intent Recognition Activity Recognition Path/Motion Manipulator Task PDDL Instructions Goals Δ-PDDL Model Updates OWQG Dialog Trajectory Constraints Hypotheticals LTL HUMAN Questions Negotiation Affect ROBOT Reports Excuses Active Model Elicitation Replanning Open World Excuse Generation 34

35 Position Statement Human-Robot teaming scenarios significantly broaden the roles of planning beyond path and motion planning Task Belief Modeling Dialog planning We advocate investigating the challenges posed by all these planning roles