Robotic Applications Industrial/logistics/medical robots

Transcription

1 Artificial Intelligence & Human-Robot Interaction Luca Iocchi Dept. of Computer Control and Management Eng. Sapienza University of Rome, Italy Robotic Applications Industrial/logistics/medical robots Known environment Minimal interaction with expert users Rescue robots Unknown/partially known environment Minimal interaction with expert users Home service robots Known environment Long-term interaction with a few (trained) users Service robots in public environments Known and dynamic environment Short-term interaction with many naïve users 2

2 AI&HRI Motivations Interacting with people requires more "intelligence" than interacting with the environment Difficulty in modeling and perception (Unpredictability) Difficulty in decision making (Social norms) Some examples In HRI proper decisions about when to start an interaction are required failures have more severe consequences wrong assumptions or guesses may be socially unacceptable ICAPS 2017 Tutorial: AI Planning for Robotics and Human-Robot Interaction AAAI 2017 Fall Symposium: AI for Human-Robot Interaction RSS 2017 Workshops 3 AI & HRI HRI application domain (i.e., problem generator) AI&RO methodologies and techniques to solve complex problems (i.e., solution generator) Robotic basic skills commonly used in HRI Computer Vision fairly used for simple recognition tasks Machine Learning used in subtasks (perception, speech recognition) and in Learning by Demonstrations Natural Language Processing less used in robotics KR & Decision making (autonomous behaviors) are less investigated 4

3 HRI Experimental Methodology Wizard-of-Oz: an operator (hidden from the user evaluating the HRI system) replaces perception and decision making of the robot Issues Over-estimation of actual abilities of the robot Partial evaluation of the HRI system (does not measure "intelligence in interaction") Expectations not met by current technology, preventing or reducing possibilities of deployment of actual systems 5 HRI in public environments Main challenges Interaction with naïve users Follow complex social norms Assumption of having complete information is not realistic Perception involving humans is generally more difficult Guessing missing information may bring to socially unacceptable behaviors Need of very robust perception and decision-making 6

4 Analysis of planning techniques largest worldwide competition on service robots Planning in the 24 teams in 2016 Plan Generation High-level Behavior N/A Total 2 (ASP, MDP) Complex robotic systems integrating: Robotics: navigation, manipulation, Computer Vision: object/face/person detection, recognition and tracking HRI: Speech recognition and natural language processing 7 RoboCup@Home Teams WrightEagle: ASP Markvito: MDP/SPUDD Golem: cognitive architecture. Dialogue models specified in SitLog Homer: state machines Leon: state machine (BICA) Pumas: HTN / High-level Petri Nets Sepanta: SMACH SocRob: POMDP + Discrete Event System / SMACH Tech United: SMACH ToBI : BonSAI + State Chart XML = SMACH Walking Machine: SMACH 8

5 Decision making in HRI Decision making in noisy and partially known real environments is still a challenging open issue in AI HRI researchers not expert in AI & decision making do not find a practical and easy way to apply these techniques TODO Manual behavior programming Extend applicability and Wizard-of-Oz experiments Improve usability are commonly used Improve robustness 9 Automatic planning in HRI Planning improves scalability wrt complexity and flexibility Manual plan writing is error prone and not scalable Each task must be defined explicitly/no reuse of components Verification/validation is not possible Explanations are very difficult Advantages Compact representations Less effort in generating many plans High-level domain specification language for non-expert designers 10

6 Example Service robot has to make sure user needs are satisfied. User needs are not known in advance. 11 Example Classical planning (complete knowledge about initial state) Guess a user need Plan with this guess Execute the plan If guess is wrong, adjust conditions and replan When guess is wrong, behavior is not socially acceptable. The robot does not move, but the user needs something. The robot prepares food or drink that is not requested by the user. 12

7 Example Plan with explicit sensing action Go to person Ask if s/he needs something // Sensing action if (need_food AND need_drink) Go to the kitchen Prepare food and drink Serve food and drink to person if (need_food) else Pro-active behavior in which the robot does not just wait for orders. Do nothing 13 Conditional Planning for service robots Advantages Does not require complete knowledge about the initial state at planning time (can model situations where some user needs are not known) Allows for minimal execution of sensing (reduces wrong behaviors due to incorrect perception) Issues Plan generation more complex Conditional Planners less developed Writing domains is still difficult Loop generation (e.g., while conditions) not available 14

8 Not-only planning Planning expert Domain expert Robot planning Robot planning Robot/plan expert user Naïve user 15 Robust plans Domain expert Robot planning Naïve user Plans generated by planners are usually no robust to unmodelled events Interaction with naïve users requires increased robustness of plans 16

9 Generation of robust plans UI Learning π Domain Goal??? π* Execution Planner??? Planning and Execution Component 17 Proposed solution [Sanelli et al., ICAPS 2017] Execution Rules π PNP PNP-ROS Goal ROSPlan Contingent-FF Robustification Domain Planning and Execution Component 18

10 ROSPlan and Contingent-FF ROSPlan, [Cashmore et al., ICAPS 2015] Contingent-FF [Hoffmann and Brafman, ICAPS 2005] 19 Petri Net Plans Formalism for high-level plan representation based on Petri Nets Ordinary and sensing actions Conditions and loops Interrupts Parallel execution (fork and join operators) Multi-robot support [Ziparo et al., JAAMAS 2011] pnp.dis.uniroma1.it PNPGen generates PNP from several planners (MDP solver, ROSPlan, HATP, ) PNP-ROS uses ROS actionlib to run plans including ROS actions 20

11 Execution rules Adding to the conditional plan interrupt (special conditions that activate recovery paths) recovery paths (how to recovery from unexpected events) social norms parallel execution (multi-modalities) Main feature Easy definition Execution variables are generally different from the ones in the planning domain (thus not affecting complexity of planning) 21 Execution rules Examples if personhere and closetotarget during goto do skip_action if personhere and not closetotarget during goto do say_hello; waitfor_not_personhere; restart_action if lowbattery during * do recharge; fail_plan after receivedhelp do say_thanks after endinteraction do say_goodbye when say do display 22

12 Petri Net Plans generation PNP Generation 1. Translation of conditional plan to PNP 2. Introduction of execution rules Robust plan with sensing and loops Algorithm is linear in the size of the plan and of the execution rules (average case) 23 Robot Office Assistant Output: PNP with 17 actions, 45 places, 52 transitions, 104 edges 24

13 Robot Office Assistant 25 Other demos COACHES project - Advertising in many shops (MDP) Human-Robot Collaboration (HATP) [Iocchi et al., ICAPS 2016] [Sebastiani et al., ICAPS 2017] 26

14 Domain description Domain expert Robot planning Naïve user In previous examples planning domains written by planner experts HRI domain experts (not expert in planning) need high-level framework for interaction design 27 MODIM Multi-modal Interaction Manager Formalism for high-level description of multiple modalities interactions Based on interaction templates that can be instantiated to generate many actual interactions Represent robotic and interaction actions in a unique framework Not yet a framework to specify planning domains Actions types: robotic actions (e.g., move, approach, goto, deliver) interactions (e.g., ask, state, answer) Operators: conditional (e.g., evaluate an answer) choice (e.g., choose a question) 28

15 MODIM IMPLEMENTATION Interaction template 29 MODIM IMPLEMENTATION Interactions based on multi-modal interaction function 30

16 MODIM IMPLEMENTATION 31 MODIM EXAMPLE Input <LABEL1; ask < Quiz1; GetChoices() >; answer < Result1 >; Result1 = wrong?state < Answer; wrong" >; GOTO LABEL1 : state < Answer; right" >;.> Output Very well!. disappears user answer < Result1==disappears > remains the same user answer < Result1==remains the same > 32

17 MODIM user studies MODIM was used by a school teacher to design and realize a physics lesson and to perform HRI studies 33 Conclusions AI Decision making techniques useful in HRI tasks Conditional planning + robustification improve robustness of robot plans for HRI applications High-level frameworks for interaction design improve usability by non-expert designer and developer More work is needed to improve usability and applicability More experiments in real environments with real users and real limitations (no Wizard-of-Oz) Thank you for your attention 34

18 References [Hoffmann and Brafman 2005] Hoffmann, J., and Brafman, R. Contingent planning via heuristic forward search with implicit belief states. In Proc. of ICAPS, [Cashmore et al., 2015] Cashmore, M.; Fox, M.; Long, D.; Magazzeni, D.; Ridder, B.; Carrera, A.; Palomeras, N.; Hurtos, N.; and Carreras, M. Rosplan: Planning in the robot operating system. In Proc. of ICAPS, [Iocchi et al., 2016] Luca Iocchi, Laurent Jeanpierre, Maria Teresa Lazaro, Abdel-Illah Mouaddib: A Practical Framework for Robust Decision-Theoretic Planning and Execution for Service Robots. In Proc. of ICAPS, [Sanelli et al., 2017] V. Sanelli, M. Cashmore, D. Magazzeni, L. Iocchi. Short-Term Human Robot Interaction through Conditional Planning and Execution. In Proc. of ICAPS 2017 [Sebastiani et al. 2017] E. Sebastiani, R. Lallement, R. Alami, L. Iocchi. Dealing with On-line Human-Robot Negotiations in Hierarchical Agent-based Task Planner. In Proc. ICAPS [Ziparo et al., 2011] V. A. Ziparo, L. Iocchi, P. U. Lima, D. Nardi, P. F. Palamara. Petri Net Plans - A framework for collaboration and coordination in multi-robot systems. In Autonomous Agents and Multi-Agent Systems, 23 (3),