Multi-Modal Robot Skins: Proximity Servoing and its Applications

Transcription

1 Multi-Modal Robot Skins: Proximity Servoing and its Applications Workshop See and Touch: 1st Workshop on multimodal sensor-based robot control for HRI and soft manipulation at IROS 2015 Stefan Escaida Navarro and Björn Hein INSTITUTE FOR ANTHROPOMATICS AND ROBOTICS - INTELLIGENT PROCESS CONTROL AND ROBOTICS, LAB (IAR-IPR) KARLSRUHE INSTITUTE OF TECHNOLOGY IROS 2015, Hamburg KIT University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association

2 Outline Introduction State of the Art Proximity Servoing Preshaping and Grasping Haptic Exploration Teleoperation (Collision Prediction and Detection) Contour Following/Collision Avoidance Conclusions and Outlook 2

3 Introduction 3

4 What is Proximity Sensing? What is the role of proximity sensing in the perception taxonomy? It shares treats with haptics: Perception is local and can only be interpreted together with proprioception, i.e. with use of the kinematic chain A spatial resolution is implemented in the skin of the robot (on the robot s exterior and grippers) As example from nature, the human skin is also locally sensitive to heat radiation 4

5 Why Proximity Sensing? 2D and 3D camera systems are inadequate for the detection and modeling of events in the near proximity of the robot Perception of camera systems is encumbered by occlusions, extraneous light influence and shadows problematic for safety Tactile perception can only detect the presence of obstacles when the contact has already occurred, aka too late Every touch event has a prior proximity event Good complement All in all: important for safety because it closes a still existing perception gap 5

6 Capacitive Proximity Sensing with Spatial Resolution Advantages: Insensitivity regarding lighting and visibility conditions Insensitivity regarding optical surface properties Wide area around the module is covered (sensing is not oriented) Disadvantages: (Sensitivity to object material properties, geometry and size) Can be used to identify object? Signal to distance non-linearity Signal processing overhead due to demodulation ( Latency) 6

7 State of the Art 7

8 State of the Art (excerpt) Preshaping with adjustment of hand pose and fingers (Hsiao et al., IR, and Wistort et al., capacitive) Collision Avoidance in C-Space (Lumelsky and Cheung, IR) Hsiao et al., 2009 [3] Low latency/high speed preshaping and collision avoidance by Shimojo Labs (IR) Wistort et al [2] Not many of the approaches also consider the tactile modality or all possible DoFs Shimojo Labs, 2011 [4] 8 Shimojo Labs, 2013 [5] Cheung and Lumelsky 1992 [1]

9 Self-Developed Sensor Modules ( ) Large Area Module: 4x4cm 1 proximity cell 1 tactile cell Detection range: ~10cm Up to 16 modules on one channel Sensor array: 3x16 = 48 sensors ~25 FPS ~100ms latency The arrangement of sensors as a regular grid delivers proximity and tactile images 9

10 Proximity Servoing 10

11 Proximity Servoing Proximity Servoing: Closed-loop position or velocity control using proximity sensors Most common control goal: achieve a configuration where sensor values are equilibrated Use methods from CV to extract features from proximity images Proximity Servoing is a base for many applications: Preshaping and Grasping, Haptic Exploration, Obstacle Avoidance, etc. 11

12 Proximity Servoing Challenges Rear lights in the fog -problem ( Franz Heger) Reduced max. velocity due small distances and non-negligible reaction times (latency) No signal/ no feedback for control Target is too far away Signal is weak/noisy Signal is too close to the detection threshold Signal is ok/good Distance to target is safe Signal is very good Distance to target is not safe Detection Threshold Distance to Target Collision Some latency Safe Zone Negligible latency Safe Zone 12

13 Applications 13

14 Preshaping [8] Challenge when grasping with tactile-only sensors non-negligible contact force has to be established for perception Preshaping means to find a suitable configuration for an gripper and its fingers -before touch is established-, that favors robustness when grasping Preshaping is proximity servoing An industrial gripper equipped with 2x2 CPTS in its jaws 14

15 6D Preshaping using CTPS in Gripper Calibration of the sensors of the two jaws to equal output under same conditions High resolution/low noise is given at small distances (<1mm) 15

16 6D Preshaping using CTPS in Gripper Grouping of measured sensor values of modules for gradient detection: Implementation of closed-loop control Gradient Descent results in preshaping Comparison of lowest distance/highest proximity values of the groups Grouping for translational and rotational adjustments: 16

17 6D Preshaping using CPTS in Gripper Simultaneous adjustment in all 6 DoF Continuous adaption while closing the gripper 17

18 6D Preshaping Evaluation with different materials Wood: Front Back Alum.: Front Back Alignment always successful Higher deviation with wood and plastic compared to aluminum Good repeat accuracy Low derivation at end of phase two 18

19 Proximity + Haptic Exploration Haptic Exploration deals with acquiring object and world information through a systematic exploration strategy using touch Combined and complementary tactile- and proximity-based approach Based on proximity servoing/preshaping 19

20 Telemanipulation [9] Display proximity values acquired inside the gripper as forces on a haptic input device to assist the user in telemanipulation Additionally, assist the user with partial autonomy 20

21 Telemanipulation Cascade Controller Implementation using a cascade-controller Inner loop is a force-position controller The user opposes full resistance, i.e. maintains effector at 0 pose Full force display and no robot movement User opposes no resistance Effector pose corresponds to proximity gradient and robot moves accordingly Outer loop is velocity controller like before 21

22 Telemanipulation Video 22

23 Telemanipulation -- Results Workload and intuitiveness was assessed in a user study against a baseline of perfect vision Workload reduction is possible when operating with partial autonomy Complementary proximity information considered to be useful Small improvement in performance Details: Talk for the paper Novel approach: Manipulation of moving objects 23

24 Contact-Prediction (Depth Tracking) [8] Contact time given by tactile modality 24

25 6D Contour Following/Obstacle Avoidance An end-effector equipped with an array of proximity sensors is used to demonstrate contour following/obstacle avoidance Problem statement: Starting from pose S, follow a path to goal G parallel to an obstacle s surface first detected at pose D, as long as the obstacle is in the way to the goal. Otherwise, choose a direct path (from S to D and E to G) 25

26 Closed-Loop Control and Curvature Prediction Reactive control approach: Track obstacle tangent plane P over time to plan next increment Get misalignment from gradients in proximity image and use them as feedback for distance and orientation controllers Predictive control approach: Track curvature of obstacle ( 1, vertical rr and horizontal) over time Use curvature to take a shortcut TT that is the chord of the corresponding circle If necessary, pre-align with rotation φφ 1 rr horizontal 1 rr vertical 26

27 Preliminary Results Curvature prediction should deliver speedup when alignment constraints are tight Correction path corresponds to object shape Implementation of HRI should be straight forward 27

28 Conclusions and Outlook 28

29 Conclusions It is purposeful to treat proximity sensing similar to tactile sensing Spatial resolution Image Processing It shares treats with traditional haptics Proximity Servoing has applications in many scenarios Often tactile modality can be used complementarily (Grasping, Haptic Exploration) Many measurement principles exist with advantages and disadvantages It still is unclear whether there is a best practice solution 29

30 Outlook, future work Reduce latency ( Rear lights in the fog -problem) Grasping smaller skin patches (almost there, work in progress) Big sensor networks challenge for the future Sensor module design (work in progress) Modularity: signal processing in each module Increase spatial resolution If possible, without loosing sensing range Add features to the sensors Receive mode for better detection of insulating materials, etc. Shear-force measurements (tactile mode) 30

31 Thank your for your kind attention! Questions? 31

32 References [1] E. Cheung and V. Lumelsky, A sensitive skin system for motion control of robot arm manipulators, Robotics and Autonomous Systems, vol. 10, no. 1, pp. 9 32, [2] R. Wistort and J. R. Smith, Electric field servoing for robotic manipulation, in Intelligent Robots and Systems, IROS IEEE/RSJ International Conference on, sept. 2008, pp [3] K. Hsiao, P. Nangeroni, M. Huber, A. Saxena, A. Y Ng, Reactive Grasping Using Optical Proximity Sensors, in Intelligent Robots and Systems (IROS), 2009 IEEE/RSJ International Conference on, 2009, pp [4] K. Terada, Y. Suzuki, H. Hasegawa, S. Sone, A. Ming, M. Ishikawa, and M. Shimojo, Development of omnidirectional and fastresponsive net-structure proximity sensor, in International Conference on Intelligent Robots and Systems, [5] Keisuke Koyama, Hiroaki Hasegawa, Yosuke Suzuki, Aiguo Ming, and Makoto Shimojo, Pre-shaping for Various Objects by the Robot Hand Equipped with Resistor Network Structure Proximity Sensors, in Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on, 2013, pp [6] D. Göger, M. Blankertz, and H. Wörn, A Tactile Proximity Sensor, in IEEE Sensors 2010, 2010, pp [7] S. Escaida Navarro, M. Marufo, Y. Ding, S. Puls, D. Göger, B. Hein, and H. Wörn, Methods for Safe Human-Robot- Interaction Using Capacitive Tactile Proximity Sensors, in Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on, 2013, pp [8] S. Escaida Navarro, M. Schonert, B. Hein, and H. Wörn, 6D Proximity Servoing for Preshaping and Haptic Exploration using Capacitive Tactile Proximity Sensors, in Intelligent Robots and Systems (IROS), 2014 IEEE/RSJ International Conference on, 2014, pp [9] S. Escaida Navarro, F. Heger, F. Putze, T. Beyl, T. Schultz and B. Hein, Telemanipulation with Force-based Feedback of Proximity Fields, in Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on,

33 Sensor Circuit and Modules in Our Group Proximity Mode (self-capacitance) Principled sensor circuit design from D. Göger, M. Blankertz, and H. Wörn: A Tactile Proximity Sensor, in IEEE Senors 2010 Tactile Mode (mutual-capacitance) 33