Jane Li. Assistant Professor Mechanical Engineering Department, Robotic Engineering Program Worcester Polytechnic Institute

Transcription

1 Jane Li Assistant Professor Mechanical Engineering Department, Robotic Engineering Program Worcester Polytechnic Institute

2 State one reason for investigating and building humanoid robot (4 pts) List two kinds of morphological communication (3 pts) Describe an example of in-hand manipulation (3 pts) 9/6/2018 2

3 Human surpasses current robots for overall performance Dexterity, energy-efficiency, versatility, reasoning, learning, selfadaption, compensation and recovery Build robotic embodiment to imitate human characteristics Appearance, voice, motion, intelligence Use humanoids as a tool for better understanding of human 9/6/2018 3

4 Expressive morphology and behavior Interpreting human expression Natural response in physical human-robot interaction 9/6/2018 4

5 9/6/2018 5

6

7 Platform Course projects 9/6/2018 7

8 Major components Baxter robot Mobile base Compliant grippers 3D cameras 9/6/2018 8

9 Simulation mode Physical mode GUI Various input devices Multi-perspective camera views 9/6/2018 9

10 Hardware 10 cameras Vero 2.2 Max frame rate = 330 Hz Resolution 2048 x 1088 = 2.2 MP Covered space = 17 ft X 21 ft 8 on railing + 2 on ground (for closer view) Software Nexus human motion capture and analysis Tracker moving object tracking 9/6/

11 9/6/

12 9/6/

13 9/6/

14 9/6/

15

16 Standard projects Shared-autonomous motion coordination of nursing robot Contact TA Team 1 Hand-arm coordination Team 2 Perception-action coordination Team 3 Loco-manipulation Human-robot collaboration Contact Heramb Nemlekar Team 4 Robot-mediated handover Team 5 High-level learning of human-robot motion coordination Small projects 1-student: Soft robot hand (me), Kinect teleoperation interface (Sihui) 2-student: Physical fatigue assessment (me) 9/6/

17 Functional robot platform Debugging hardware is a pain! May be stuck by technical details of a software Experienced members in the team Starting from sketch takes much longer time Commitment of every team members Your project grade may be ruined if the team fall apart Healthy team dynamics Your leader is knowledgeable, reasonable, and helpful Your partners are trustworthy 9/6/

18 A good project team means Successful project outcome (35%) High-quality group literature review (10%) Continue current project in RBE 550 and direct research for thesis 9/6/

19 Find the team right for you Project survey form allow you to fill in preferred teammates Talk to project contact to see whether you like and project and have the right skill set Talk to your classmates and find the ones you like to work with It may be a good idea to choose a small project You can have full control of the project progress You can work at your comfortable pace 9/6/

20 HUMAN MOTION COORDINATION Human motion study Analyze human motion coordination Develop novel models for human motion prediction Learn human motion strategies Model validation Compare model prediction with more human motion data SHARED AUTONOMOUS NURSING ROBOT User study Evaluate task performance Identify user challenges Propose shared-autonomous robot control design Refer to human motion strategies User study for performance evaluation Compare with/without autonomy

21 Autonomous reach-to-grasp, perception-action coordination, loco-manipulation Assessment of physical fatigue in robot teleoperation

22 Risk-sensitive task, unstructured environments Nursing, in-home caring, surgery, rescue Needs human in the loop Teleoperation interface for robot learning Can also be used for robot teaching human 9/6/

23

24

25 Much slower than human performance Perception issues, robot s physical capability, interface design Large training efforts Need one hour or more for the demonstrated task Hard to control motion coordination Subjects can only focus on controlling one robot components Some interfaces does not support simultaneous control of many DOFs 9/6/

26 Reduce novice users effort Mental/physical, training and using Improve task performance Efficiency, robustness, safety Enable complex robot behaviors 9/6/

27 Teleoperation Interface Evaluation (A-term) Propose sharedautonomous robot control (B-term) Team 1 Team 2 Team 3 Teleoperate the nursing robot yourself to find: (1) How teleoperation is difficult? (2) How teleoperation is different for different interfaces? (3) What part of task can be shared-autonomous? (4) Desirable level of automation? Reach-to-grasp Perception-action coordination Loco-manipulation Leaders Sihui Li Alexandra Kene

28 Sihui Li 2 nd year PhD Research focus Shared autonomous tele-nursing robots Teleoperation interfaces, Learning from demonstration 9/6/

29 Kenechukwu C. Mbanisi 3rd year PhD Research focus Human motion modeling and learning Human performance assessment Human-vehicle interaction 9/6/

30 Alexandra Valiton 2nd year PhD Research focus Shared autonomous tele-nursing robot Interactive perception 9/6/

31 Distorted perspective Lack of depth perception Keyhole Effect: narrow FOV 9/6/

32 We conduct human experiment to examine How does behavior evolve with expertise? How do head and clavicle camera behavior differ? Is there any way to be successful with an active wrist camera? How does the subject use the perception camera? Which camera(s) does the subject choose, when given a choice? 9/6/

33 9/6/

34 Understanding human perception-action coordination to inspire development of robotic teleoperation interfaces How can a robot help the operator perceive a remote environment? How can remote perception be improved? How are vision and action coupled in human motion? 9/6/201834

35 Collect data in human and robot teleoperation experiments Model and Compare the camera usage in human motion coordination and teleoperated motion coordination Propose autonomous perception control based on human preference Implement on TRINA platform demonstration Sept Oct Nov Dec 9/6/

36

37 Heramb Nemlekar 2rd year Master Research focus Human-robot handover High-level motion planning

38 RBE 550 Motion Planning Instructor: Jane Li, Mechanical Engineering Department & Robotic Engineering Program - WPI 9/6/

39 9/6/

40 9/6/

41 9/6/

42 Learning parameters from human-human handovers Giver and receiver are not face-to-face Giver is Sitting vs Standing Giver is Moving Receiver using one hand vs both hands Generalizing prediction and robot response across variants of handovers 9/6/

43 Human motion analysis Conduct motion study for handovers Pre-process and segment human demonstrations Algorithm development Formulate regression models for OTP prediction Improve object detection for grasping Modify probabilistic estimation in ProMPs 9/6/

44 Parastegari et al, Modeling human reaching phase in human-human object handover with application in robot-human handover, in IROS D. Vogt et al, One-shot learning of human-robot handovers with triadic interaction meshes, Autonomous Robots, vol. 42, pp , M. Prada et al, Implementation and experimental validation of Dynamic Movement Primitives for object handover, in 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, sep 2014, pp G. Maeda et al, Phase estimation for fast action recognition and trajectory generation in human robot collaboration, The International Journal of Robotics Research, vol. 36, (13-14), pp , /6/

45

46 Heramb Nemlekar 2rd year Master Research focus Human-robot handover High-level motion planning

47 Motivation High-level task plan can be built upon low-level motor skills Learning and planning can be more efficient at high-level 1. Leg1 Joint Leg3 Joint Arm Joint Hand Preshape Move to door 2. Grasp handle 3. Pull door open 4. Move ahead 5....

48 For robot to plan actions on a high level - Learn high-level actions (options) and states (symbols) Training data from human demonstrations Understand reward for each high-level state (symbol) Generate an optimal task plan for a daily activity Objectives Learn symbols & rewards for a loco-manipulation task Perform table-top organization task involving handovers 9/6/

49 Human motion analysis Conduct motion study Pre-process and segment human demonstrations Algorithm development Implement low-level options as autonomous functions Improve SVM classification Generate symbols for loco-manipulation task 9/6/

50 References: G.D. Konidaris, L.P. Kaelbling, and T. Lozano-Perez. From Skills to Symbols: Learning Symbolic Representations for Abstract High-Level Planning. Journal of Artificial Intelligence Research 61, pages , January Literature Keywords: High-level planning, Symbolic MDP, PDDL, STRIPS, Hierarchical reinforcement learning 9/6/

51

52 Tasks Integrate hand with tele-nursing robot Test hands with various grasping Compare to ReFlex SF hand for performance Outcome Interface for hand motion control Demonstration videos Fits for ECE/ME background Learning ROS 9/6/

53 Problem with current Kinect teleoperation: Hand orientation detection is problematic with Kinect only. Needs other hardwares (e.g. IMU) to be integrated. Goal - Full body teleoperation with Kinect Implement proposed methods and integrate into the Trina system. Compare Kinect teleoperation with Vicon teleoperation. Fits for: 1 student, CS background, familiar with Kinect 9/6/

54 What is fatigue? A feeling of tiredness and being unable to perform tasks effectively. Change in muscle and brain electrical activity Affects heart rate and oxygen levels Alters movement patterns and coordination, including posture and perception Fatigue Performance 9/6/

55 Delsys Trigno EMG/IMU Sensors 9/6/

56 When comparing interfaces, which causes the most/least fatigue? How does level of fatigue compare with speed or ease of use? How does fatigue develop differently in the different interfaces? Which teleoperation tasks cause the most/least fatigue? How can these results be combined to suggest an ideal teleoperation interface? 9/6/

57 Assist Kene and Alexandra in gathering pilot data Based on preliminary data, compare teleoperation interfaces Examine the data for trends in fatigue development Propose a teleoperation interface design that limits fatigue and maximizes effectiveness. 9/6/

58 Fill in the project choice survey 9/6/

59 Goal Integrate PPU with off-the-shelf parts Reference F. Wang, G. Chen, and K. Hauser. Robot Button Pressing In Human Environments. IEEE Intl Conf. on Robotics and Automation (ICRA), /6/