TRUST-BASED CONTROL AND MOTION PLANNING FOR MULTI-ROBOT SYSTEMS WITH A HUMAN-IN-THE-LOOP

TRUST-BASED CONTROL AND MOTION PLANNING FOR MULTI-ROBOT SYSTEMS WITH A HUMAN-IN-THE-LOOP Yue Wang, Ph.D. Warren H. Owen - Duke Energy Assistant Professor of Engineering Interdisciplinary & Intelligent Research (I 2 R) Laboratory Department of Mechanical Engineering, Clemson University

Background and Motivation Computational Trust Models Trust-based Teleoperation of Multi-Robot Systems Trust-based Multi-Robot Symbolic Motion Planning Conclusions 2/25

Background and Motivation Computational Trust Models Trust-based Teleoperation of Multi-Robot Systems Trust-based Multi-Robot Symbolic Motion Planning Conclusions 3/25

Background & Motivation & Motivation Trust Trust is the central enabler in determining human s acceptance and allocation of automation Qualitative models, measurements before and after operations Offer little insight for real-time robotic operations and performance guarantees Subjective Trust Measurement HRI Human-machine interface (HMI) design Lack understanding of the dynamic interaction for real-time robotic operations in complex and uncertain environments Lack system-level performance guarantees and verification Human-robot interaction (HRI) in multi-robot systems is especially challenging Novelty Social/Psychological/Physical HRI for real-time robotic operations Quantitative analysis Embed human factors analysis into control theory, decision theory, and robot motion planning 4/25

Background and Motivation Computational Trust Models Trust-based Teleoperation of Multi-Robot Systems Trust-based Multi-Robot Symbolic Motion Planning Conclusions 5/25

Computational Trust Models Trust - the attitude that an agent will help achieve an individual s goals in a situation characterized by uncertainty and vulnerability. [Lee & See, Human Factors, 2004] Our Trust Models Time-series trust model [Wang et. al. Springer 2014; B. Sadrfaridpour et. al. Springer 2015; Rahman et. al. DSCC 2015a; Saeidi & Wang, CDC 2015; Saeidi et. al. ACC 2016; Sadrfaridpour et. al. CASE 2016; Rahman et. al. CASE 2016a; Spencer et. al., IROS 2016; Mahani & Wang, DSCC 2016; Saeidi et. al., T-RO 2017] Dynamic Bayesian Network (DBN) Human-centered design for autonomous systems to enable utilization of autonomy Subjective Trust Design of human-like, unbiased decision-making for balanced joint system performance & human experience Objective Trust /Trustworthiness trust model [Wang et. al., ACM TiiS 2017] Robot-to-human trust model [Walker et. al. MSCI 2015; Rahman et. al. CASE 2016a] Mutual trust model [Wang et. al. ACC 2015, CPS 2015; Wang & Zhang ed., Springer 2017] RoboTrust for multi-robot systems [Saeidi et. al., IROS 2017] 6/25

DBN Trust Models [Wang et. al., ACM TiiS 2017] Main Factors Autonomy Xu & Dudek. OPTIMo: Online probabilistic trust inference Allocation Prob. of Collaboration model for asymmetric human-robot collaborations, In Proc. ACM/IEEE Int. Conf. HRI, 2015. Lee & Moray, Trust, control strategy, and allocation of function in human-machine systems, Ergonomics,1992. Hancock, et al. A meta-analysis of factors affecting trust in human-robot interaction, Human Factors, 2011. Hoff & Bashir, Trust in Automation: Integrating Empirical Evidence on Factors that Influence Trust, Human Factors, 2014 7/25

DBN Trust Models Real-time trust computation: (a) human, robot performance, fault, (b) trust belief, (c) human collaboration, (d) trust change, and (e) direct subjective trust measure 8/25

Background and Motivation Computational Trust Models Trust-based Teleoperation of Multi-Robot Systems Trust-based Multi-Robot Symbolic Motion Planning Conclusions 9/25

Tele-autonomous Operation of Mobile Robots Delay Delay Automated decision aids via objective computational trustbased measures Primary decision-maker Milder negative effects of improper trust in mobile robotic applications Improper trust Disproportionate Autonomy allocation Decreased task performance and increased human workload 10/25

Mixed-Initiative Bilateral Haptic Teleoperation of Mobile Robots based on Mutual Trust Analysis [Saeidi et. al. ACC 2016; IEEE T-RO 2017; Fu et. al., ACC 2016] Function of robot-to-human trust Possible sources of instability! Passivity theory Function of human-to-robot trust 11/25

Ein Mixed-Initiative Bilateral Haptic Teleoperation of Mobile Robots based on Mutual Trust Analysis Passivity Theory & Port Network Theory Σ Σ : x = f( x( t), u( t), t) E out E in Eout Passive systems are stable Interconnection of passive n-ports results in a larger passive system In teleoperation schemes force and velocities commands form the power ports filter Feedback r-passivity Wave transformation PO & PC Port-based model for the mixed-initiative bilateral haptic teleoperation 12/25

Mixed-Initiative Bilateral Haptic Teleoperation of Mobile Robots based on Mutual Trust Analysis Dynamics of the Master Haptic Device Feedback r Passivity of the Master, Master Dynamics Master Dynamics 13/25

Mixed-Initiative Bilateral Haptic Teleoperation of Mobile Robots based on Mutual Trust Analysis Passivity of the Communication Channel with Variable Time-Delay and Variable Scaling Block diagram for the communication channel with time-varying delays and variable power scaling 14/25

Mixed-Initiative Bilateral Haptic Teleoperation of Mobile Robots based on Mutual Trust Analysis Passivity of the Slave using PO & PC Total velocity command to the robot Local autonomous controller (high-level controller) Guarantee passivity of the slave Force feedback algorithm Velocity tracking controller (low-level controller) Environmental disturbance 15/25

Trust-based Bilateral Teleoperation of Multi-Robot Motivation: Systems [Saeidi et. al. IROS 2017] Motivation Reduced manpower for control of a robot team Increased robustness and flexibility of the robotic agents via cooperation Challenges Stability of the proposed scheme under the effects of switching Tracking performance of the system under the proposed scheme 16/25

Passivity of Switched Systems [Saeidi et. al. IROS 2017] 17/25

Passivity of the Trust-based Bilateral Teleoperation of Multi-Robot Systems [Saeidi et. al. IROS 2017] Select a leader robot with the highest trust level input output The relative position of the leader robot with its neighbors as haptic feedback 18/25

Background and Motivation Computational Trust Models Trust-based Teleoperation of Multi-Robot Systems Trust-based Multi-Robot Symbolic Motion Planning Conclusions 19/25

Trust-based Multi-Robot Symbolic Motion Planning Linear Temporal Logic (LTL) Propositional Operators: negation, conjunction, disjunction, implication, equivalence Temporal Operators: until U, always, eventually Expressive Motion Tasks: Reachability (liveliness) π 1, obstacle avoidance and safety π 2, convergence (stability) π 1, sequencing and temporal ordering Model Checking Checking whether TT satisfies over Π is called model checking. A model checker will return true if the formula is satisfied. If no such run can be found, the model checker returns a counterexample. Labeled Transition System (LTS) A transition system is a tuple TT = (SS,SS 0, Act, δ, Π, L), where SS is a set of states, SS 0 is a set of initial states, Act is a set of actions, δ is a transition relation, Π is the set of atomic propositions, and L is a labeling function. 22/25

Trust-based Multi-Robot Symbolic Motion Planning [Spencer et. al., IROS 2016; Mahani and Wang, DSCC 2016; Wang et. al., ACM TiiS 2017] Example High-level Specification for ISR Tasks 20/25

Trust-based Multi-Robot Symbolic Motion Planning [Spencer et. al., IROS 2016; Mahani and Wang, DSCC 2016; Wang et. al., ACM TiiS 2017] Trust-based Decomposition High trust: more tasks assigned to more trusted robot Low trust: fewer tasks assigned to less trusted robot Deadlock- and Livelock- Free Algorithms Goal destination reachability Inter-robot collision avoidance A human-in-the-loop Compositional Reasoning Interleaving of transition systems Unconditional fairness AP for Comm, Obs, & Ctrl 21/25

Trust-based Multi-Robot Symbolic Motion Planning [Spencer et. al., IROS 2016; Mahani and Wang, DSCC 2016; Wang et. al., ACM TiiS 2017 ] Monitoring and Checking (MaC) Simplex 22/25

Background and Motivation Computational Trust Models Trust-based Teleoperation of Multi-Robot Systems Trust-based Multi-Robot Symbolic Motion Planning Conclusions 23/25

Conclusions Computational trust models that are dynamic, quantitative, and probabilistic for real-time robotic operation Human trust in robot; Robot trustworthiness; Robot trust in human; Trust propagation in multi-robot systems Mutual trust based bilateral teleoperation of mobile robots Trust based shared control; Trust based haptic feedback; Passivity based analysis for multi-robot switches Trust based symbolic multi-robot motion planning with a human-in-the-loop Trust based compositional reasoning for multi-robot systems; Trust based run time verification for switches between manual and autonomous motion planning 24/25

Dr. Hamed Saeidi Dr. Rahman Mizanoor Mr. Behzad Sadr Mr. Adam Spencer Mr. Xiaotian Wang Mr. Zhanrui Liao Ms. Qiuchen Wang Mr. Jonathan Todd Mr. James Svacha Mr. Foster McLane Mr. Longsheng Jiang Mr. Maziar Mahani Mr. Fangjian Li Mr. Huanfei Zheng Mr. Evan Sand Ms. Swetha Mahadevan Mr. Sanghamitra Ahirrao Thank you! 25/25