Passive Bilateral Teleoperation Project: Reconfigurable Control of Robotic Systems Over Networks Márton Lırinc Dept. Of Electrical Engineering Sapientia University
Overview What is bilateral teleoperation? Notion of passive systems Methods for ``passivation of teleoperation systems - Scattering matrix approach - Wave variables approach - Passivity observer/controller approach - The problem of packet loss Some closing slides...
Bilateral teleoperation Basic elements of the system: Master robot - Slave robot Unilateral teleoperation: the operator moves the slave (generally low power robot), the master follows the motion of the slave Bilateral teleoperation: the master (when it is in contact with the environment) reflects back the contact force to the slave, which is reflected to the operator Early teleoperation system (no network)
The elements of a networked teleoperation system Operator Haptic Device Network - Internet Master robot Camera (Slave) and Environment Display
The elements of a networked teleoperation system II. Master (m) and slave (s) robot model: x position f - force
Networked teleoperation The control objectives in the teleoperation systems are: - the stability of the system (especially when contact with environment is expected) - telepresence, transparency of the teleoperation: position tracking and force reflection The Internet in the control system is modeled as a variable (bidirectional) time delay + package losses The network induced delay further decrease the stability properties According to previous experiments the typical operator behavior in teleoperation systems: move and wait
Passive electrical and mechanical networks One-port network, two port network, multiport networks f voltage v current f v power Energy - integral of power The network is passive if the sum of energy flows always positive (always more energy flows in than out)
Passive electrical and mechanical networks II. Examples: Mechanical electrical equivalency f force - voltage (input) v speed - current (output) Stability: The passive systems dissipates energy through resistors, friction.. Passive systems are BIBO (Bounded Input Bounded Output) Stable. Power continuous interconnection of passive systems yields passive systems
Passivation of non-passive networks Series passive element If R N < 0 (negative resistor) the system is not passive Passivation: R>R N Parallel passive element If V s > U>0 not passive Passivation: R L >R s U/(V s -U) R>0 or with passive feedback controllers
Back to teleoperation systems The Human is assumed a passive controller. The Environment is assumed a passive spring + damper, with high spring constant. We have to make the system highlighted below passive by controlling correspondingly the robot actuators. Based on the available measurements (delayed, sometimes lost..)
Back to teleoperation systems II. The objectives can be formulated as:
Passive teleoperation - Scattering approach Applicable for linear systems, ideas taken from the transmission line theory: The hybrid matrix: Definition Interpretation Ideal
Passive teleoperation - Scattering approach II. Passivity theorem: Sketch of the proof:
Passive teleoperation - Scattering approach III. Passivation with constant time delays: Master (m) and slave (s) robot model: Controller: (slave: position error + velocity error) Passivation of the communication channel: Resulting scattering matrix:
Passive teleoperation Wave variables Instead of exchanging as reference signals the power variables force and velocity the wave variables are transmitted For constant delay we have same passivity results as in the case of the scattering variables
Passive teleoperation Wave variables II. Time varying delays: the wave variables are amplified with a gain that depends on the rate of the delay With this modification the communication channel is passive (proof is omitted).
Passive teleoperation Wave variables III. Position drift: General wave or scattering variables encode information about velocities and forces on both sides of the teleoperation channel; however, no explicit information about position is available which may result in position mismatch between the master and slave systems due to initial transient response or numerical roundoff errors. Modification (for constant delay): the position is sent across the communication channel both from master to slave and viceversa, where it is used in a proportional type control. With this modifications the passivity is preserved and the position coordination can be guaranteed. Similar results are available for time varying delays
Passive teleoperation Wave variables IV. Impedance matching: it is well known that if the load that terminates the line has a different impedance than the characteristic impedance of the transmission line then wave reflections occur. In the case of bilateral teleoperation, such reflections degrade the performance of the system. This led to the introduction of impedance-matching elements b at each end of the communication channel
Passive Teleoperation: Passive Observer/Controller No delay case: Passivity Observer Passivity Controller: (α 1, α 2 is not zero only if the passivity decreases )
Passive Teleoperation: Passive Observer/Controller II. Define the directional energy flows: Use two observers: The controllers have similar forms:
Packet loss and passivity Packet loss is an inherent problem with packet-switched networks due to several factors such as transmission time-outs, transmission errors and limited buffer size. Null packet strategy: the passivity preserves, but coordination is compromised: Last packet strategy: the passivity is compromised:
Packet loss and passivity II. Summer, Packet Formatter Packet Reader, Subtractor
Packet loss and passivity III. Interpolator in the case of n packet loss we apply the mean value of the n loss packets n times It can be shown that the energy of the system not increases in the case of packet loss (E 0 E r )
Alternative representations of teleoperation systems Electrical network representation: Network as a flexible beam:
Predictive displays Augmented Reality view of the environment. A virtual slave robot is moved real time on the top of the video feedback. After a confirmation from the operator, the real slave robot follows the motion of the virtual slave robot.
Some Youtube videos with passive teleoperation experiments http://www.youtube.com/watch?v=gw7xab4ffse http://www.youtube.com/watch?v=gcf6mhi24rk http://www.youtube.com/watch?v=iuggparmi9y
What we want to do? The human operator is not necessarily passive in Internet based teleoperation. The deviation between the seen motion (visual feedback) and felt motion (force feedback) may de-passify the operator. Human in the Loop problem in a teleoperation approach. Stochastic modeling approach to describe the common human + network + teleoperator behavior. Quantifying real-time the quality of the video and signal transmission controller, path planner re-tuning in the function of the network Video transmission for reliable teleoperation (real time zoom, real time image quality change in function of the motion) Image processing methods, use of extra sensors to help the operator