Machine Learning in Robot Assisted Therapy Ensuring the Safety of an Autonomous Robot in Interaction with Children Challenges and Considerations Stefan Walke stefan.walke@tum.de SS 2018
Overview Physical Safety Mental Unharmedness Proposed Guidelines Future Work 1
Assessment Therapy Success Chair of Robotics, Artificial Intelligence and Embedded Systems Pushing the lower boundary Many talks on how to improve therapy: Add features Improve therapy logic and reactions Pushing the upper limit of success Safety Considerations: Remove or hold back new features Decrease chance of physical or mental harm Improving the lower boundary Time Problem: When is a system safe? Danger + - Minimizing False Negatives + True Positive False Positive - False Negative True Negative 2
Ensuring Physical Safety Source: Interaction Design Foundation 2016 3
Ensuring Physical Safety We can use general human robot interaction as a baseline Adjusting for: Decreased experience of cause-effect chains Increased curiosity Lower pain threshold Different reaction to unforeseen situation Increased possibility of permanent trauma 4
Compliant Robots A compliant actuator allows deviations from its equilibrium position, depending applied external forces: Spring-type actuators Hydraulic-type actuators Tendon-type actuators Example: Variable Stiffness Actuator [1] Real-time control of reference position and mechanical impedance Two motors that run in opposite directions vary the tension Main shaft experiences different dampening Perspective Practical Implementation view of the Variable of the Stiffness Actuator. variation [1] The transmission belt 1 connects the DC Motors pulleys 2-3 to the joint shaft 4, and it is tensioned by passive elastic elements 5-6-7 [1] 5
Soft Tissue & Artificial Skin The Idea of wrapping the robot skeleton in soft material This has multiple advantages: More life-like feel Dampen contact force Include sensors to stop movement in case of contact Other sensors (heart rate, pulse oximeter, etc) Source: TUM 2015 Soft material allows covering up joints in contrast to stiff material Source: Carnegie Mellon University, Soft Robotics and Bionics Lab 6
An atlas of physical human robot interaction [2] A map of robotics for anthropic domains: main issues and superposition for phri [2] 7
An atlas of physical human robot interaction [2] Problems in the approach: Current Research is separated in two areas (CS and ME) Often no interaction (feed forward control) Danger of false mental Model (especially with children) Difficulty of defining Injury Guidelines in High DoF Systems General/Basic Safety Rules: No sharp edges to reduce risk of lacerations Lightweight but stiff materials to reduce inertia Visco-elastic coatings or compliant actuators to dampen impacts 8
An atlas of physical human robot interaction [2] Injury* is not a good metric for safe interaction: Actuators: Do not allow active Impact by monitoring space Speed and precision vs Safety Trade-off Variable-impedance actuation and Distributed macro-mini actuation AIS score Severity Type 0 None None 1 Minor Superficial 2 Moderate Recoverable 3 Serious Possibly recoverable Control: Unstructured anthropic domains no detailed description of the environment Real-time motion planning Impedance control *There is some research into pain tolerance, but as that varies widely from person to person and especially children it is not easy to define [3] 9
Mental Unharmedness Source: AI won't just make us better at our jobs - they'll make us better humans, too. (Reuters/Francois Lenoir) 10
The Problem with Mental Unharmedness Hard to measure how well child is responding Reduce features: Not concerned about disinterest or distraction Focus on fear, stress and wish to stop Often the first indicators are very subtle and might not be notice Very difficult topic, therefore we pick one special case: Automated detection and classification of positive vs negative robot interactions with children with autism using distance based features [4] 11
Experiment Setup Participants: 8 children with ASD from 5 families 5-10 years of age (min. 2 years on communication subscale) In a room with robot and a parent for 3 x 5min Learning phase (get all functions explained) No protocol, child can do what it wants Room has one camera, Robot has IR emitters for orientation The humanoid robot used in the experiment [4] Background subtraction used to find child and parent (differentiate by shirt color) 12
Robot Behaviour & Child State Classes Robot (one contingent behaviour session and one random behaviour session) Child approaches/retreats happy/sad (will approach child if further than 1m away) Button pushed or child talks blow bubbles and spin Will try to face child, except if child is behind ignore Parent is only an obstacle Child Avoiding/Retreating Interaction with robot or bubbles Still (not near parent or wall) Near parent (not interacting) Near wall Capture of the camera with participant and robot marked by algorithm [4] 13
Positive vs. Negative Robot Interactions Expert classification of states and overall positivity Previous approach used a heuristic measure to determine state Only about 85% accuracy Percentage of session time spent in each interaction state [4] 14
Results Classifier with 8-dimensional feature vector: v = (d r c, d p c, d w c, φ r c, v c, v r c, v w c, φ r c v r c ) Trained with human labeled data A Gaussian Mixture Model was used as a Classifier Overall 91.4% accuracy in state recognition Real-time feasible (classification time below camera framerate) avoidance interaction parent wall avoidance 52.8 0.8 1.4 2.6 interaction 34.8 97.5 7.6 11.5 parent 9.9 1.5 90.8 3.7 wall 2.5 0.2 0.2 82.2 15
How to build a safe system Proposal Build Systems Bottom up: Add one feature ( hardware or software ) at a time Minimize error before expanding This reduces error propagation and eliminates unforeseen interactions Guidelines for features: Needs graceful exit Needs to be adaptable and online learning enabled Should have provable functionality ( see functional programming) 16
The Data Problem Future Work Huge problem of getting enough data with (autistic) children Main hindrance are ethical and medical concerns, unlikely to decrease Problem will probably persist for quite a while Potential Fix: Simulation Simulating physical interaction is easy in principle For emotional interactions there are large amount of classifiers Use classifiers as generators Develop a framework that can simulate emotional states and corresponding gesture 17
Questions? 18
Sources [1] Design and Control of a Variable Stiffness Actuator for Safe and Fast Physical Human/Robot Interaction G. Tonietti, R. Schiavi, A. Bicchi - 2005 [2] An atlas of physical human-robot interaction A. De Santis, et al. - 2007 [3] A Failure-to-Safety Kyozon System with Simple Contact Detection and Stop Capabilities for Safe Human-Autonomous Robot Coexistence K. Suita, H. Ikeda, et al. - 1995 [4] Automated detection and classification of positive vs negative robot interactions with children with autism using distance based features D. Feil-Seifer, M. J. Matari c - 2011 19