Dr. Houxiang Zhang. Speaker. TAMS, Department of Informatics University of Hamburg, Germany. MIN- Faculty Department of Informatics

Transcription

1 The State of the Art Research in Robotics and Intelligent Systems at the TAMS, University of Hamburg Speaker Dr. TAMS, University of Hamburg, Germany 1

2 Outline of this talk TAMS introduction Short introduction of TAMS group Staff Research Projects Other research on robotics Mobile robots Climbing robots Modular robots Others 2

3 Outline of this talk TAMS introduction Short introduction of TAMS group Staff Research Projects Other research on robotics Mobile robots Climbing robots Modular robots Others 3

4 TAMS introduction University of Hamburg Faculty of Mathematics, Informatics and Natural Sciences Department Informatics, Group TAMS Vogt-Kölln-Straße 30, D Hamburg, Ger rmany Tel.: ; Fax.: tams@informatik.uni-hamburg.de 4

5 TAMS group in Hamburg TAMS group (Technical Aspects of Multimodal Systems) built in 2002 One full Professor Three Postdoc research fellows Two graduate staff Eight Ph.D. students Three open positions at the moment 5-10 master students t 5

6 Research The main research topic is to examine the interaction of different modalities of sensory and cognitive systems including vision, writing, hearing, talking, touching/feeling and grasping. The main application area and demonstrator platform of our work intelligent service robotics. is Test new system architectures and algorithms for sensor-driven manipulation, intelligent sensors, cognitive robotics, and human-machine machine interaction. 6

7 Research Hardware Design Nano-Manipulation Robotics TASER Pioneer robot Humanoid robot Sky cleaner Gecko climbing robot Telebot project Cube-M modular robot Grasping Software and E-Learning 7

8 Projects Cross-modal Interaction in Natural and Artificial Cognitive Systems (CINACS) Developmental Pathway towards Autonomy and Dexterity in Robot In-Hand Manipulation (HANDLE) Multistandard Integrated Network Convergence for Global Mobile and Broadcast Technologies (MING-T) Intelligent and Accurate Vision-Systems to support Service Robots (IVUS) E-Learning-Consortium Hamburg (ELCH) Biologically Inspired Modular Climbing Caterpillar Robot Using Passive Adhesion (BICCA) 8

9 Projects Cross-modal Interaction in Natural and Artificial Cognitive Systems (CINACS) Developmental Pathway towards Autonomy and Dexterity in Robot In-Hand Manipulation (HANDLE) Multistandard Integrated Network Convergence for Global Mobile and Broadcast Technologies (MING-T) Intelligent and Accurate Vision-Systems to support Service Robots (IVUS) E-Learning-Consortium Hamburg (ELCH) Biologically Inspired Modular Climbing Caterpillar Robot Using Passive Adhesion (BICCA) 9

10 MING-T Multistandard Integrated Network Convergence for Global Mobile and Broadcast Technologies The project from 01/2007 until 12/2 2008, with a total EC funding of around 2 million Euros. Consortium University of Hamburg, Germany Georg-August-University of Goettingen, Germany CREATE-NET, Italy Frontier Silicon, United Kingdom ENENSYS Technologies, France Nokia Siemens Networks China, China Tsinghua University, Beijing, China Beijing University of Post and Telecommunications, China China Telecom Corporation Limited Beijing Research Institute, China Contact: Dr. Norman Hendrich 10

11 MING-T Research, develop, prototype, integrate and validate the interoperability and handover issues of the representative mobile digital broadcast standards developed in China and Europe. Foster cooperation between Europe and China in both industry and research communities. Address the following issues convergence between broadcast standards and mobile communications network technologies. convergence of the European DVB-H standard with the emerging Chinese standard (temporarily named as DMB-T) with a view to providing a harmonized framework how to deliver these services with a consistent level of quality to rich media applications across mixed broadcast networks. 11

12 MING-T Develop an architecture providing seamless access in converged digital broadcasting and mobile communications networks. Incorporate Chinese DMBT into the multistandard framework. Apply advanced coding techniques for the services in the converged digital broadcasting and mobile communications networks environment. 12

13 Projects Cross-modal Interaction in Natural and Artificial Cognitive Systems (CINACS) Developmental Pathway towards Autonomy and Dexterity in Robot In-Hand Manipulation (HANDLE) Multistandard Integrated Network Convergence for Global Mobile and Broadcast Technologies (MING-T) Intelligent and Accurate Vision-Systems to support Service Robots (IVUS) E-Learning-Consortium Hamburg (ELCH) Biologically Inspired Modular Climbing Caterpillar Robot Using Passive Adhesion (BICCA) 13

14 CINACS: international graduate school 14

15 CINACS: international graduate school Cross-modal Interaction in Natural Carried out by the University of Hamburg and Tsinghua University, Beijing, is funded d by the DFG and the NSFC of the Chinese Ministry i of Education; 9 year project in three phases. and Artificial Cognitive Systems Contact: Prof. Dr. Jianwei Zhang 15

16 CINACS General research questions: Which architectures are suitable for certain types of cross- modal tasks? How to transform between modalities? What are the mechanisms of cross-mo dal perceptual phenomena? What are the general principles for resolving cross-modal conflicts? How are multimodal percepts generated and represented? How can cross-modal integration be re ealized in technical systems? 16

17 CINACS Our basic research concepts Extraction of biological principles and improved understanding of the cross-modal integ gration in humans. Introduction of biological principles into artificial intelligent systems. 17

18 CINACS: PhD-Thesis Hierarchical memory organizationn of multimodal robot skills for plan- based robot control Two-layered robot control l( (symbolic and reactive levels) l Hierarchical planner Library of atomic robot actions Symbol grounding (pragmatic) Active perception p of the environment 18

19 CINACS: PhD-Thesis Weser et.al., HTN Robot Planning in Partially Observable Dynamic Environments (ICRA 2010) 19

20 TASER Project Taser Service-robot of the University of Hamburg Mobile platform with differential drive Two Mitsubishi PA10-6C manipulators Two 3-finger robotic hands Stereovision camera head Omni-directional vision system Two SICK laser range finders Wireless LAN communication 20

21 TASER Project 21

22 TASER Project Taser Mobile manipulation Grasping Mobile robot navigation People tracking 3D image processing Man-machine interaction 22

23 Grasp learning Zgrasp simulator: Concept of grasp semantics Real-world objects Reinforcement-learning Experience replay for speedup Application to real experiments 3-finger Barrett-Hand 23

24 Vision-based learning manipulation Imitation learning of human hand movements optical hand tracking with active contours self-organizing map learns human motions 24

25 Projects Cross-modal Interaction in Natural and Artificial Cognitive Systems (CINACS) Developmental Pathway towards Autonomy and Dexterity in Robot In-Hand Manipulation (HANDLE) Multistandard Integrated Network Convergence for Global Mobile and Broadcast Technologies (MING-T) Intelligent and Accurate Vision-Systems to support Service Robots (IVUS) E-Learning-Consortium Hamburg (ELCH) Biologically Inspired Modular Climbing Caterpillar Robot Using Passive Adhesion (BICCA) 25

26 Handle project Developmental Pathway towards Autonomy and Dexterity in Robot In- Hand Manipulation (HANDLE) The project runs from 02/2009 unti il 01/2013, with a total EC funding of 6.35 million Euros. Consortium University Pierre et Marie Curie, Paris, France Shadow Robot Company, Ltd., London, UK Universidad Carlos III, Madrid, Spain Universidade de Coimbra, Portugal King's College, London, UK Oerebro University, Sweden University of Hamburg, Germany Commissariat a l'energie atomique, France Universidade Tecnica de Lisboa, Portugal Contact: Dr. Norman Hendrich 26

27 Handle project Understanding how humans perform the manipulation of objects in order to replicate grasping and skilled in-hand movements with an anthropomorphic artificial hand. Evolving robot grippers from current best practice towards more autonomous, natural and effective articulated hands. Tactile sensor Learning Visual sensor Data glove 27

28 Newly funded ECHORD project A Hyper-Flexible Work Cell for Laboratory Automation (HYFLAM) The project runs from 01/2011 until 06/2012, with a total funding of around 400, Euros. Details: As PI together with partners Shadow Company and Health Protection Agency (HPA) grasp and place the required objects perform handover operations to lab workers perform actions independently Contact: Hannes Bistry 28

29 Projects Cross-modal Interaction in Natural and Artificial Cognitive Systems (CINACS) Developmental Pathway towards Autonomy and Dexterity in Robot In-Hand Manipulation (HANDLE) Multistandard Integrated Network Convergence for Global Mobile and Broadcast Technologies (MING-T) Intelligent and Accurate Vision-Systems to support Service Robots (IVUS) E-Learning-Consortium Hamburg (ELCH) Biologically Inspired Modular Climbing Caterpillar Robot Using Passive Adhesion (BICCA) 29

30 Project IVUS: smart sensors SMART camera systems for mobile Service Robots funded by BMBF (program Leitinnovation Servicerobotik ) project partner Basler AG; from 2006 to 2009 tasks of the University of Hamburg development of the camera softwar e integration and test on the robot platform Main aspects of concepts & implem mentation: carry out arbitrary image processing steps on a smart camera system take advantage of the smart camera architecture optimize capturing parameters for regions of interest Contact: Hannes Bistry 30

31 Related projects on hardware design FPGA-based prototyping systems Flexibility: embedded (software) system Performance: specialized hardware Workflow Algorithm on Control Computer Transferred to embedded processor or DSP Hardware accelerated, when needed FPGA-based, for dynamic reconfiguration Contact: Dr. Andreas Mäder 31

32 Omnidirectional to panoramic image conversion 32

33 Projects Cross-modal Interaction in Natural and Artificial Cognitive Systems (CINACS) Developmental Pathway towards Autonomy and Dexterity in Robot In-Hand Manipulation (HANDLE) Multistandard Integrated Network Convergence for Global Mobile and Broadcast Technologies (MING-T) Intelligent and Accurate Vision-Systems to support Service Robots (IVUS) E-Learning-Consortium Hamburg (ELCH) Biologically Inspired Modular Climbing Caterpillar Robot Using Passive Adhesion (BICCA) 33

34 Research prototypes Telebot Telebot (TAMS group based on cooperation with BUAA, ) 9 channels for sensor inputs; 4 outputs for actuators Communication interface Java and C++ programming easy More flexible and extended dfunctions 34

35 Research prototypes Telebot 35

36 Research prototypes Telebot Applications Moving along a line Avoid an obstacle Looking for an object Following a moving object Mapping the 36

37 Projects Cross-modal Interaction in Natural and Artificial Cognitive Systems (CINACS) Developmental Pathway towards Autonomy and Dexterity in Robot In-Hand Manipulation (HANDLE) Multistandard Integrated Network Convergence for Global Mobile and Broadcast Technologies (MING-T) Intelligent and Accurate Vision-Systems to support Service Robots (IVUS) E-Learning-Consortium Hamburg (ELCH) Biologically Inspired Modular Climbing Caterpillar Robot Using Passive Adhesion (BICCA) 37

38 A bio-inspired modular climbingg caterpillar robot (BICCA) Walking and climbing not only on rugged terrain but also on the vertical surfaces and ceilings inside buildings; Locomotion capacities including pitching, yawing, lateral shift, and rotating; Sensor-servo-based active percepti on of the environment. Picture taken from a 3D-animation of the planned robotic caterpillar in a variety of postures. Contact: 38

39 Concept of the BICCA The climbing robotic caterpillar structure is based on bio-inspired methodology and a distributed modular approach. A novel low-frequency vibrating passive adhesion principle. A hybrid control architecture including CPGs containing i a reflex mechanism as well as biological features and a learning algorithm for sensor-servo-based behaviour control and active perception of the environment to complete the system. 39

40 Research on the mechanical design Joint module Dimension: 35mm x 37mm x 30mm Weight: 19.2g Max Output Torque: 0.2Nm Output Angle: 0~180Deg Control Modules Attachment module Dimension: 26mm x 32mm x 20mm Weight: 27.8g Max Attaching Force: 40N Max Permitted Sliding Force: 15N Max Permitted Turing Torque:0.1Nm Distributed onboard controller: controls one joint and one attachment module, communicates by I 2 C bus; PC: preprograms gait, communicates with robot through serial interface. 40

41 Real experiments-inchworm robot Dissymmetrical control method Gait displacement: 4.7mm One gait time: 5s 41

42 Real experiments- new results Passive attachment New version of caterpillar robot 42

43 Outline of this talk TAMS introduction Short introduction of TAMS group Staff Research Projects Our research on robotics Mobile robots Climbing robots Modular robots Others 43

44 Outline of this talk Short personal information TAMS introduction Short introduction of TAMS group Staff Research Projects Our research on robotics Mobile robots Climbing robots Modular robots Others 44

45 Humanoid robot navigation Based on a multimodal cognitive interface Present an intuitive interface to describe a navigation task for a mobile robot. Give the robot the ability to use human-like spatial language. Develop a robust and high detection rate object processing system. Implement an efficient motion planner to calculate the path and the footstep placements for the humano oid robot. Contact: Prof. Dr. Jianwei Zhang 45

46 Research prototypes Humanoid robot Tams humanoid robots 2 HOAP- 2 from Fujitsu Automation Co., Ltd. 25 DOF Legs DOF: 2 x 6 Arm DOF: 2 x 5 Head DOF: 2 Body DOF: 1 Five-finger Hands Wih7k Weight 7kg Height 50 cm USB Interface (USB 1.1) Geode GX1 CPU 2 Logitech Quickcams with ¼ inch CMOS Sensor 46

47 System architecture 47

48 Miniature city M. Elmogy, C. Habel, and J. Zhang. Online motion planning for HOAP-2 humanoid robot navigation. In proceedings of the IROS2009, pp St. Louis, Missouri, USA,

49 Other prototype JL-I JL-I with three uniform modules was developed. 35 centimetres long, 25 centimetres wide and 15 centimetres high. Two powered tracks, a serial mechanism, a parallel mechanism, a docking mechanism. and Changing its posture by pitching, yawing and rotating. Contact: 49

50 Other prototype JL-I Docking process Crossing a step 90 self-recovery 180 self-recovery Crossing a narrow fence 50

51 Outline of this talk TAMS introduction Short introduction of TAMS group Staff Research Projects Our research on robotics Mobile robots Climbing robots Modular robots Others 51

52 Research prototypes Sky Cleaners Sky Cleaner-I Sky Cleaner-II Sky Cleaner-III Sky Cleaner-IV 52

53 Research prototypes Sky Cleaners A following unit for protection A supporting vehicle for air, water, and power. Sky Cleaner robot X and Y cylinders composed of the robot. A waist joint in the center Four foot cylinders to lift and lower vacuum suckers Real-time control system 53

54 Research prototypes Gecko robot First version A lightweight smart wall-climbing robot, which was developed as a flexible mobile platform carrying a CCD camera and other sensors (2004). The second version We improved the first version for several aspects in Much lighter and smaller With sensors and wireless interface Controlled in two different ways 54

55 Outline of this talk TAMS introduction Short introduction of TAMS group Staff Research Projects Our research on robotics Mobile robots Climbing robots Modular robots Others 55

56 Research prototypes Modular robot Main idea: Building robots composed of modules The design is focused on the module, not on a particular robot The different combinations of modules are called configurations Some advantages: Versatility Fast prototyping Testing new ideas Contact: 56

57 Research prototypes Modular robot Y1 module, 2004 Y1 modular minimal i configuration, i 2005 Y1 pitching-yawing connecting research, 2006 GZ-I mechanical improvement design, GZ-I system integration, 2007 Related research,

58 Research prototypes Modular robot 58

59 New version of modular robot Strong and robust With more mechanical parts in ABS material With more connecting possibilities With sensor interface 59

60 Modular grasping Integrate various grasping and flexible mobility based on modular approach

61 Modular grasping Integrate various grasping and flexible mobility based on modular approach From kinematics viewpoint, a solid result to confirm the idea 61

62 Outline of this talk TAMS introduction Short introduction of TAMS group Staff Research Projects Our research on robotics Mobile robots Climbing robots Modular robots Others 62

63 Multi-sensor fusion stereo-vision (rectified) fusion with laser-range finder fusion with Omni-vison: self-localization fusion with grasping/touch-sensorss 3D object recognition i smart-camera concept active illumination i Contact: Denis Klimentjew 63

64 Object recognition and reconstruction 64

65 Multisensor-based perception of doors feature detector using omnidirectional image feature detector using laser range scan, the results are transformed to image coordinates for better visualization integrated multimodal result 65

66 3D robotic mapping outdoor 3D Robotic mapping Automatic sensing and modeling is a fundamental scientific issue in robotic community, for it is an essential ability for autonomous mobile robot. Laser range finder is the state of the art sensor Six degree of freedoms are involved ou utdoor Also called 3D mapping 6D SLAM Figure courtesy of Henrik Andreasson 66

67 Summary of the talk A brief introduction to TAMS; Our research and all projects were presented; Some interesting robotic projects were introduced in detail. 67

68 Thanks for your attention! Any questions, comments and suggestions are welcome! 68