Visually Guided Errand Service for Home Robot

Transcription

1 ICRA 2007 Dec. 18, 2006 Visually Guided Errand Service for Home Robot Sukhan Lee Professor and Director

2 Where I am from

3 Current Status and Future Prospect on Service Robotics Operational industrial robots will increase from about 922,900 units (2005) to 1,112,500 units (2009) (average annual growth rate of 4.9%) 34,000 new service robots for professional use to be installed ( ). about 5.6 million units of service robots for personal use to be sold ( ). (Domestic robots : 3.9 million units, Entertainment and leisure robots : 1.6 million units) Source : Executive Summary Report, World Robotics 2006, IFR Roomba (irobot) Trilobite Robotsapien V2 Care-O-Bot (Fraunhofer) Asimo (Honda) Partner Robot (Toyota) Papero (NEC)

4 Korean National Initiative on Intelligent Robot National Growth Engine Digital TV Display Intelligent Robot Future Automobile Next-Generation Semi-Conductor Intelligent Home Network Next-Generation Mobile Telecommunication Digital Contents Next-Generation Battery New Bio-Medicine

5 Ubiquitous Robotic Service to Human Robotic Personal Assistant Networked Robot Brain Chip Sensor NetworkIntelligent Building Ubiquitous Network Machine-Embedded Smart Home Robot Home Network Home Robot Edutainment Robot Intelligent Vehicle Intelligent Service Robots Humanoid

6 MIC: Network Based URC Robot 2 kinds of robots were installed at airports, station, and post offices at Seoul area and now under test from November Convenience guide service Call button/touch panel Postal code guide Counter guide Information guide U-CARD service Happiness entertainment service Kid s song/story teller Bio-rhythm/fortune teller Natural scene movie Fat check/monitoring Safety security service Monitoring and alarm Security service Remote monitoring Net gun Inchon airport Seoul station

7 MIC: Network Based URC Robot 7 kinds of wheeled, cell-phone controlled domestic robots are being placed into 1000 households and kindergartens in Seoul and its vicinity, where the broadband convergence network (BcN( BcN) ) was installed, from Oct as a trial run for new robot technologies. Application Services Basic Services Common Services Jupiter Robot-Specific Services Nettoro Roboid Robot/User Authentication Automatic Charging Server QoS Voice Recognition Text To Speech Entertainment Contents Voice Chatting Educational Services Robot Mail (3D ) Radio Signal Tuner Motion Detection Context Awareness Robot Control Home Guard Home Monitoring Fast Photos Cleaning Jupiter Roboid Context Awareness Contents DB Information Services Entertainment Contents Entertainment Contents Entertainment Contents School Notices Nettoro

8 MOCIE: Service in House and Factory Personal Service Professional Service Industrial Robot Cleaning Robot Patrol Robot, etc 6 types. Surveillance Robot for Fire. Prototype 3 pcs. 165Kg, Mass Product 4 type. 20Kg etc 2 kinds of Prototype Vaccum 100 w, 1 hour works Start to sell Patrol-ability Humanoid Robot Named Albert Hubo Mid, Small body Hovering Robot Serial Type Design New Robot Controller Robot Vision for handling

9 21 c Frontier: Service for Elderly JAPAN (%) High-Elderly 21 SWEDEN Elderly Dominated 14 FRANCE U.S.A RATIO of ELDERLY Elderly Society KOREA (YEAR) Dialogue/ Dialogue/ Voice Voice Recognition Recognition Actuator Actuator Module Module Infotainment Robot Arm Arm Manipulation Manipulation Vision Vision based based Recognition Recognition Elderly Care Robot Errand Robot

10 Vision based Human-like Manipulation and Navigation for Errand Service Research Results of ISRC, SKKU

11 Vision based Human-like Manipulation and Navigation: Grand Challenges Table Setting Beverage/Food Errand Service Arrangement/Cleaning Self Modeling of Objects

12 Vision Based Human-like Manipulation & Navigation Framework Self-Construction of 3D Object/Env. Model ( SRIS ) Categorization Stereo Camera Generic/Specific Model CRE/CPE Sensor Planning o Extremely Ill-conditioned Workspace: cluttered, occlusion, reflection, illumination change Eye-Illumination on Hand 3D Camera 3D Sensing Platform Images Multi Resolution Volume Spatio-Temporal Image Based Evidence-Model Matching Framework (CM Particle Filtering) Global Geometric Feature Primitives Real-time 3D Object/Env. Recognition Recognized Objects Mobile Platform SRIS Based Visual SLAM 3D Realtime Object/Environment Modeling Perception Guided Manipulation - Visual Servoing -Motion/Grasp Planning Arm Hand

13 Vision Problem: Robustness under Environmental Variations Texture Illumination Incomplete and Noisy 3D Information Enough Invariant Features may not be Extracted Refrigerator Object Top View Front View

14 3D Object Recognition and Pose Estimation: Evidence-Model Matching in a Sequence of Images by Particle Filtering Scene at T+1 Scene at T CPE Evidence Selection/Collection Sensor Planning - Asynchronous/Concurrent Process Evidence Extraction Prediction T+1 Particle at T (Particle = Object Model) Model Measurement Re-sampling T+1 3D Object/Environment Information System Generic Model Specific Model Object or Global Geometric Feature Primitives

15 Behavioral Cognition: Automatic Evidence Selection and Collection Has_A (Freezer, Milk, Poisson, 6, 2) Located_In (Likely) Living room Air Conditioner Air cooling Room Located_In (Likely) Blue Book 3D Line 3D matching to DB SIFT Color PC Monitor 2D Line Square Edge Image Hot air inhalation Cool air ventilation Cooling (19~30) Inhalator Ventilator Cooler Depth Image 2D Image Process Structure Interaction Inhalation lines Ventilation lines Rectangle Turn on/off and Control Light Look around Approach to candidate proximity observation Rectangle Texture 2D/3D Line SIFT Head Pan/Tilt turn Go To Grip Arm Control Evidence Structure Behavior Structure

16 Human-like Real Time Workspace Modeling Approximate Representation of Workspace Using Global Geometric Features Projection of Recognized Objects into Model Multi-Resolution Octree Representation of Obstacles

17 3D Sensing Platform: 3D Camera based on Signal Separation CODEC Surface geometries and signal mixturing Signal Separation CODEC SoC Surface reflectance Signal Mixing Projection by DMD (Projector) Hierarchical Orthogonal Coding 3D Point Cloud workspace Captured Images (Camera) Signal Separation Decoding Programmable Flash

18 2D HOC and Two Camera-One Projector System for Removal of Reflections Projected 2D HOC Captured 2D Image Depth Image including Errors Epipolar Geometry of the Dual-camera Structured Light System erro r Corrected Depth Image Error Correction Corrected value

19 Object Self Modeling from Generic Knowledge Object Segmentation Based on Curvature Estimation Generic Model SVD based Curvature Estimation Model Deformation to Fit onto Actual Point Clouds

20 Real-time Access and Retrieval Planning by GPU based Visibility Testing Grasping and manipulating objects in crowded environment. Minkowski sum projected on the cube map The white area represents visible directions The original point cloud The enlarged obstacles using Minkowski sum

21 Object Vision Based Human-like Navigation with Service Robot Information System + + SrXML Base Map << Ontology >> Friger First-Aid Box Has_Cold Air Conditioner Door Shelf Refrigerator Bookshelf Refrigerator Model Cold Green Tea Shelf Has_A By using CAD tool (3D MAX or AutoCAD) Green Tea Model First-Aid Box

22 Vision Based Human-like Navigation with Service Robot Information System Recognized Objects as Natural Landmarks <- Object <- Robot

23 Behavioral Cognition for Robust and Reliable HRI and Robot Personality

24 Robot Café Demonstration

25 Conclusion The robustness and reliability for visually guided human-like manipulation and navigation may be obtained by the integration of behavioral cognition and multiple evidence model matching paradigm. The capability of autonomous and dexterous manipulation represents a key break through for the next generation of service robots. We may see the transition from digital to robotic convergence taking place in 20 years.

26 Emphasis from Information to Intelligence Intelligent Manufacturing Intelligent Appliance Contents Seamless Infrastructure Service Intelligent Robotic Service Intelligent Home/Room/ Office/Building Intelligent Transportation System Intelligent Personal Assistant Robotic Convergence Intelligent Health Care System

27 Thank You!!!