Artificial Intelligence & Robotics from RoboCup to Everyday Applications

Transcription

1 Artificial Intelligence & Robotics from RoboCup to Everyday Applications What is a robot? Luca Iocchi Department of Computer, Control, and Management Engineering Sapienza University of Rome, Italy 2 What is a robot? What is a robot? A machine (anthropomorphic) equipped with sensors, actuators and control unit, that performs useful tasks (autonomous or tele-operated) 3 4

2 Autonomous agent What is Artificial Intelligence? environment perceptions actions sensors reasoning Systems that think like humans Systems that act like humans Systems that think rationally Systems that act rationally actuators 5 6 Artificial Intelligence and Robotics Why are we not living yet with robots? Perception Planning Reasoning Learning Cooperation Intelligent functions Because of technology, economics, ethics, 7 8

3 Autonomy Autonomy: ability to take decisions and act without human direct control and guidance Autonomy is a fundamental feature for developing intelligent robotic systems Sometimes autonomy is considered not appropriate for robots in human society Autonomy in RoboCup vs. real applications Why there is a big gap between the high requirement of autonomy that we have in RoboCup and the limited use of autonomous robots in everyday applications? To what extent the society is prepared to trust autonomous robots? 9 10 Everyday applications Security robots (bomb-disposal, search&rescue) Service robots (vacuum cleaners,.) Entertainment robots (games, contents, ) Medical & Surgical robots (transportation, surgical helpers, ) Service robots tele-operation or push-a-button limited autonomy Domestic robots with more autonomy are not yet on the market 11 12

4 Medical and Surgical robots tele-operation limited or no autonomy Security robots mostly tele-operated limited autonomy Educational & Entertainment robots programmable autonomy Too simple or require AI&Robotics skills Questionnaire Would you trust an autonomous robot to - cut the grass? - clean the house? - exercise your pet? - prepare you a meal? - help you put your shoes on? - cut and style your hair? - measure your blood pressure? - put drops in your eyes? - give you an injection? - cut your finger nails? - fill or remove your teeth? - perform a surgical operation on you? Different user needs Different levels of autonomy 15 16

5 RoboCup Largest world-wide robotic competition (but there are others) 1. Technology transfer to everyday applications 2. Benchmarking techniques through common settings The dream: by 2050 build a team of humanoid soccer robots competing (and possibly winning) with the human (FIFA) world champion team. The Robot World Cup Initiative Achievable? Moon Chess ENIAC 1946 Deep Blue 1997 IBM NASA 19 20

6 Human-robot games RoboCup Editions 1997 Nagoya 1998 Paris 1999 Stockholm 2000 Melbourne 2001 Seattle 2002 Fukuoka 2003 Padua 2004 Lisbon 2005 Osaka 2006 Bremen 2007 Atlanta 2008 Suzhou 2009 Graz 2010 Singapore 2011 Istambul 2012 Mexico City RoboCup Soccer RoboCup Humanoid Soccer 23 24

7 RoboCup Rescue RoboCup Junior RoboCup Soccer Leagues: full autonomy Soccer Dance Rescue Rescue League: (semi-)autonomy tele-operation diff. levels of autonomy HRI 27 28

8 RoboCup vs. Everyday applications full autonomy vs. almost no autonomy some HRI vs. almost no HRI Question Is autonomy a required feature for robotic everyday applications? Autonomy is fundamental for introducing robots in everyday life. Different levels of autonomy intelligently chosen by the robot at run-time. Human-robot interaction is the way to show the right level of autonomy Case Studies / Argumentations Humanoid Soccer robots Search & rescue in emergency scenarios Surveillance applications Soccer intelligence The real complexity comes from the dynamics of the environment (opponent): Complex actions/behaviours Play cooperatively Team game strategies Domestic and service robots 31 32

9 Humanoid Soccer Robots Machine Learning for Humanoid Robots [Cherubini et al. 2009] Machine Learning for Humanoid Robots Model-based vs Model-free Learning 35 36

10 Model-based & Model-free Learning Soccer Vision The soccer field seen by the Aibo [Iocchi 2007] [Leonetti, Iocchi 2010] Perception & Motion Control Particle Filter Localization SPQR team RoboCup

11 Cooperation Dynamic Task Assignment Petri Net Plans for high-level robot programming Overall results Soccer robots completely autonomous in dynamic enviornment with noisy perception Complex task performance Multi-robot coordination Expressive model (concurr., failures) Formal verification Easy design/debug Robotic soccer techniques exported in other application domains Involvement of young researchers 43 44

12 Search&Rescue Robots Simulation of emergency scenarios Multi Agent Modeling (Foligno) Simulation of operation procedures Robot team Mission Interface USARSIM Multi Robot Modeling System test & development RoboCup Rescue Competitions DARPA Challenge 2012 Goal: find victims in a disaster scenario Rescue arenas from NIST Two categories in competition Autonomy Tele-operated Soon flying robots 47 48

13 Autonomy requirements Autonomous Mapping and Exploration Operation without direct visual feedback for operator. Continue operation without communication. Return to initial position.... [Grisetti et al. 2007] [Calisi et al. 2007] Lab experiments Autonomous Mapping and Exploration Probabilistic formulation p ( X t Z t, Z t-1,, Z 1 ) Probabilistic solution P( Xt Z1 : t) = α P( Zt Xt) P( Xt xt 1) P( xt 1 Z1 : t 1) xt

14 Quad-rotor UAV Remote sensing Autonomous hovering [Angeletti et al. 2008] Quad-rotor Software Architecture The real scenario L Aquila, April 2009 Multi-Robot Exploration [Calisi et al. 2007] In order to effectively deploy several robots they must cooperate: 1. Avoid conflict on resources 2. Share common goals Tasks Trajectory coordination Cooperative perception Dynamic task allocation 55 56

15 Multi-Robot Surveillance Ongoing projects: Multi-Robot Surveillance [Iocchi et al. 2011] Intelligent systems for exploration and surveillance of complex and dynamic environments [Uniroma1] Multi-Robot Teams for Environmental Monitoring [DHS, USA] Single Operator / Multi Robot Usab. experiments Adjustable Autonomy Egocentric and allocentric views Video Feedback Remote Control [Valero et al. 2008] HRI and Situation Awareness Overall results Prototype rescue robots with different levels of autonomy (tested in lab and in RoboCup) Prototype tele-operated UAV for search & rescue tested on the field Fill the gap between lab autonomy and real tele-operation More tests on the field by the end-user Use Multiple robots 59 60

16 Robot Soccer and Surveillance Soccer vs. Surveillance Vision From teams of autonomous soccer robots tracking ball and opponents to distributed video surveillance systems [Bloisi et al. 2009] People Localization and Tracking Stereo vision for robust people detection, localization and tracking Automatic event detection 2006 Posture Tracking Input: XYZ-RGB cloud points Output: 3D posture over time Algorithm: - 3D Model matching (ICP) - Tracking of principal points (KF) [Bahadori et al. 2007] 63 64

17 Car and People Counting Robust automatic counting of moving objects (cars, people, buses) for infomobility Automatic event detection Stereo Vision Plan-View Segmentation Plan-View Segmentation 67 68

18 Plan-view Tracking Posture Observation and HMM classification Observation vector z = {α, β, γ, δ, h NORM } HMM with discrete state variable γ P H [Iocchi, Bolles 2005] Kalman Filter with constant velocity model Data association solved as an optimization problem integrating location and appearance information P(γ z t:t0 ) = η P(z t γ t ) γ P(γ t γ ) P(γ z t-1:t0 ) P(γt γ ) models transition between postures depends on the environment and can be learned or determined empirically P P δ α β h NORM P F σ [Pellegrini, Iocchi,2008] γ Overall results Autonomous perception and target tracking in multi-robot (multi-camera) teams for surveillance applications. Extensive tests with end-users Domestic and Service Robots RoboCare: Robotics for the elderly ( ) RoboCup@Home (started in 2006) More real applications of surveillant robots More HRI 71 72

19 Some RoboCare Goals Reminder ( It s the time for the suppository ) Entertainment( Let s have a chess game ) Transport Check Search Escort ( Get me a coffee from the coffe machine ) ( Check the stove ) ( Look for Mr.Smith ) ( Go for a walk with Mrs.Brown ) [Bahadori et al. 2004] Current focus Functional abilities: Navigation Mapping Person recognition Person tracking Known and unknown Object recognition Manipulation of known and unknown objects Speech recognition Gesture recognition Cognition [Wisspeintner, et al. 2009] Implementation Tests in Stage 1 General rules 2 stages with different focus Stage 1 for singular, basic tasks Stage 2 for more complex, integrated tasks Robot has to be autonomous High level of uncertainty in the environment (no standardization) Only natural interaction allowed Very short setup time (usually 1 minute) Partial score system for tests Registration & Inspection: The robot has to register itself to the competition, it will be inspected also. Follow me: Lead the robot quickly on a path through an external scenario Go Get It!: The robot has to find and bring the object that the user asks for Who's Who: The robot has to find and introduce itself to unknown persons and recognize them later General Purpose Service Robot I: (see next slides) Open Challenge: Teams have to present and demonstrate their most important (scientific) achievements 75 76

20 Tests in Stage 2 RoboCup@Home 2010 Enhanced Who is Who: Find persons, receive orders for a drink and bring it to the correct person Shopping Mall: The robot helps in shopping (retrieve certain objects from shelves) in a real store General Purpose Service Robot II: (see next slides) Cleaning the house: Demonstrate abilities for cleaning, dish washing, re-arranging, loading a dishwasher, putting laundry in the washing machine, ironing clothes, Benchmarking Robot Minds General Purpose Service Robot How to benchmark cognition? How to test mental aspects of a robot? What are important mental capabilities that the robot needs? What is (probably) the most difficult part of intelligent robots to create? This requires an experimental approach There is no predefined order of actions Aims for high-level reasoning Put the robot in difficult / confusing situation Commands can include multiple objects and underspecified object/locations The test is about how much the robot understands about the environment 79 80

21 General Purpose Service Robot Where to go Category 1: A complex sequence of actions Get chips from the kitchen table and bring them to the person in the living room Category 2: Underspecified Bring me a drink Category 3: Confusing situation Find food in the dining room (but there is not) More tests in the real world More tests of cognitive skills More underdefined tests such as GPSR More testing of HRI More language skills Interteam robot-robot cooperation Keep improving the adaptive benchmarking Overall results RoboCare evaluated by elder people giving significant feedback for design and deployment of DSR RoboCup@Home: largest world-wide benchmark-competition for DSR More real applications Summary Autonomy is needed when robots - act in difficult environment - interact with humans - need to perform automatic tasks Autonomy must adapt to end-user needs (firemen, elderly, surveillants, ) Autonomous intelligent behaviour is still an open challenge in several application domains 83 84

22 Summary Fill the gap between lab autonomy and real tele-operation Better autonomy More real applications More end-user evaluation More HRI Acknowledgements Daniele Nardi Alessandro Farinelli Giorgio Grisetti Vittorio A. Ziparo D. Bloisi S. Pellegrini D. Calisi G. Randelli G. R. Leone G. Settembre M. Leonetti D. Tipaldi L. Marchetti A. Valero References G. Angeletti, J. R. Pereira Valente,, and D. Nardi. Autonomous indoor hovering with a quadrotor. In Proc. of Workshop on Mini and Micro UAV for Security and Surveillance, S. Bahadori, A. Cesta, G. Grisetti,, G. R. Leone, D. Nardi, F. Pecora, and R. Rasconi. RoboCare: Pervasive Intelligence for the Domestic Care of the Elderly. Intelligenza Artificiale, S. Bahadori,, G. R. Leone, D. Nardi, and L. Scozzafava. Real-time people localization and tracking through fixed stereo vision. Applied Intelligence, 26, D. Bloisi and. ARGOS - A video surveillance system for boat trafic monitoring in Venice. International Journal of Pattern Recognition and Artificial Intelligence, D. Calisi, A. Farinelli,, and D. Nardi. Multi-objective exploration and search for autonomous rescue robots. Journal of Field Robotics, 24, D. Calisi,, D. Nardi, C. M. Scalzo, and V. A. Ziparo. Context-based design of robotic systems. Robotics and Autonomous Systems, References A. Cherubini, F. Giannone,, M. Lombardo, and G. Oriolo. Policy Gradient Learning for a Humanoid Soccer Robot. In Robotics and Autonomous System, 57 (8) A. Farinelli,, D. Nardi, and V. A. Ziparo. Assignment of dynamically perceived tasks by token passing in multi-robot systems. Proceedings of the IEEE, 94(7), G. Grisetti, G.D. Tipaldi, C. Stachniss, W. Burgard, and D. Nardi. Fast and Accurate SLAM with Rao-Blackwellized Particle Filters. In Robotics and Autonomous Systems, 55 (1). 2007, R. C. Bolles. Integrating Plan-View Tracking and Color-Based Person Models for Multiple People Tracking. In Proc. of ICIP Robust color segmentation through adaptive color distribution transformation. In RoboCup 2006: Robot Soccer World Cup X, 2007., L. Marchetti, D. Nardi. Multi-Robot Patrolling with Coordinated Behaviours in Realistic Environments. In Proc. of the International Conference on Intelligent Robots and Systems (IROS)

23 References Thank you M. Leonetti,. Improving the performance of complex agent plans through reinforcement learning. In Proc. of the 9th International Conference on Autonomous Agents and Multiagent Systems (AAMAS) S. Pellegrini,. Human Posture Tracking and Classification through Stereo Vision and 3D Model Matching. In EURASIP Journal on Image and Video Processing A. Valero, F. Matia, M. Mecella,, D. Nardi. Adaptative Human-Robot Interaction for Mobile Robots. In 17th Int. Symposium on Robot and Human Interactive Communication. RO-MAN T. Wisspeintner, T. van der Zant,, S. Schiffer. Scientific Competition and Benchmarking for Domestic Service Robots, Interaction Studies, V.A. Ziparo,, D. Nardi, P.F. Palamara, and H. Costelha. PNP: A formal model for representation and execution of multi-robot plans. In Proc. of 7th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2008), Thank you for your attention Questions? 89 90