Enhancing Robot Teleoperator Situation Awareness and Performance using Vibro-tactile and Graphical Feedback by Paulo G. de Barros Robert W. Lindeman Matthew O. Ward Human Interaction in Vortual Environments Lab Worcester Polytechnic Institute
Summary Introduction Related work on HRI and vibro-tactile interfaces Interface description Hypotheses User study Methodology Results & Analysis Conclusions P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 2
Cycle of HRI Remote real environment Representation of remote real environment Local real environment Acts on environment Senses the environment Robot-readable operator commands Robot Robot-sensed environment data Operator input-device commands HRI application Human-readable environment data Operator P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 3
Typical USAR Interface USAR = Urban Search and Rescue E.g., building collapse Little automation, apart from navigation Input: Mouse, keyboard & joystick Display: Graphical Single interface for two tasks Robot navigation, Search for victims P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 4
Typical USAR Interface (Yanco et al., 2006 ) P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 5
Typical USAR Interface (Yanco et al., 2006 ) P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 6
Typical USAR Interface (Yanco et al., 2006 ) P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 7
Our Focus How do current data display techniques impact USAR task performance? Compare use of multiple channels Vibro-tactile feedback Vision Measure improvements in Task performance Situation awareness P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 8
Related Work USAR is a very applied field of work More than a decade old Many different robots and interfaces from different labs No standards or guidelines yet widely accepted Basic guidelines and metrics already developed Current Interfaces Focus on situation awareness and efficiency Scalability, portability, reuse, robustness? P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 9
Related Work Vibro-tactile feedback Widely used in Virtual Reality and 3DUI Shown to cause improvement in performance of some tasks By adding vibrational feedback to USAR, we intend to Reduce operator cognitive load Increase operator situation awareness P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 10
Collision Proximity Feedback Interface Attempts to follow existing HRI guidelines Virtual robot Game engine simulates remote robot, remote world, and operator robot interface Third-person view Multi-sensory collision proximity feedback (CFP) Through graphical ring around the robot Through set of 8 tactors around the operator waist (TactaBelt) Map blueprint using by robot sensed data P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 11
CPF Components P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 12
Video P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 13
Hypotheses Using either the vibro-tactile or the graphical ring interfaces should improve performance and situation awareness (SA) Using both interfaces should improve performance and SA even more P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 14
User Study Goal Validate hypotheses with sample USAR scenario Task Locate red spheres in ruins of a closed room Find as many as fast as possible Measured factors (dependent variables) Number of robot collisions Task performance time Number of objects found Quality of objects location report P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 15
Methodology Between subjects 4 groups Experimental steps Demographics Instructions Practice Experiment Sketch map Group 1 (None) Group 2 (Ring) Group 3 (TactaBelt) Group 4 (Both) Graphical Ring TactaBelt P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 16
Virtual Environment Training and experiment rooms P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 17
Demographics 13 female and 14 male Average age: 20.52 (std. dev. 5.24) No statistically significant difference (SSD) found among experimental groups in terms of Videogame experience Robot experience P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 18
Results: Collisions Number of collisions per minute SSD between ( None, Ring ) and ( Ring, Vibro-tactile ) P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 19
Results: Search Number of spheres found per minute SSD between ( Ring, Both ) Ring negatively impacting performance Vibro-tactile causing no performance impact P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 20
Results: Sketchmaps Sketch map quality compared to map A 1 2 3 D4 B C F E5 P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 21
Results: Sketchmaps Improvement in spatial understanding SSD between ( None, Both ) Trend towards significance between groups Vibro-tactile and Both P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 22
Results Task time reduction No SSD among groups Expected correlations found Map goodness vs number spheres found (R=0.41) Map goodness and collisions/minute (R=-0.44) P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 23
Discussion Using both types of feedback led to more accurate sketch maps compared to using no feedback Neither type had a positive or negative effect in terms of task time and number of spheres found Ring interface had a negative impact on the number of collisions, perhaps due to occlusion Videogame experience seems to have some effect on performance P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 24
Conclusion Using a multi-sensory interface can improve operator SA without detriment to operator cognitive load Subjects complained about intensity of vibrational feedback Future work Need to improve the ring interface Using a real robot or commonly used platform (USARSim) Study different ways of providing vibrational feedback Add other types of robot sensor feedback P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 25
Questions P. de Barros, R.W. Lindeman, M.O. Ward - WPI HIVE Lab, Dept. of Computer Science 26