Robotic System Simulation and Modeling Stefan Jörg Robotic and Mechatronic Center

Transcription

1 Robotic System Simulation and ing Stefan Jörg Robotic and Mechatronic Center

2 Outline Introduction The SAFROS Robotic System Simulator Robotic System ing Conclusions Folie 2

3 DLR s Mirosurge: A versatile MIRS scenario Slave Master Folie 3

4 DLR MIRO TM 3 kg payload; 10 kg weight 7 torque-controlled joints Folie 4

5 DLR MIRO TM Various control concepts Telepresent Semi-autonomous Autonomous Folie 5

6 DLR MIRO TM Open surgery Minimally invasive surgery Plate Shaft Magnet Button Folie 6

7 DLR MICA A 8 A 9 Manipulation forces up to 6 N; 0,85 kg weight 3 actuated axes; Cardanic joint: ± mm diameter Integrated electronics Sensing of manipulation forces/torques A 10 Folie 7

8 DLR s Mirosurge: A versatile MIRS scenario Folie 8

9 Motivation Increase patient safety through the accurate simulation of the surgical robotic system. Goal: Develop a surgical simulator perfectly interchangeable with the real surgical robot Robotic System Simulator (RSS) (for MIRS) Application Control (MIRS) Hardware Abstraction Layer Workflow Safety Checks Control Hardware Abstraction Layer (HAL) Robot Simulator 1 Sensor/ Actuator. Robot Simulator 2 Robot Dynamics Interface to world simulator Folie 9

10 Outline Introduction The SAFROS Robotic System Simulator Robotic System ing Conclusions Folie 10

11 The SAFROS Robotic System Simulator (RSS) Goal: Develop a surgical simulator perfectly interchangeable with the real surgical robot adaptable to various use cases focus on real-time simulation with hardware-in-the-loop ( haptics 1kHz ) scaling of simulation effort (layers of detail) accurate dynamics models Use application control of the real system Robotic System Simulator (RSS) (for MIRS) Application Control (MIRS) Hardware Abstraction Layer Workflow Safety Checks Control Hardware Abstraction Layer (HAL) Robot Simulator 1 Sensor/ Actuator. Robot Simulator 2 Robot Dynamics Interface to world simulator Folie 11

12 RSS - Simulation Platform For Workflow Design WORKFLOW DESIGN Robotic System Simulator (RSS) (for MIRS) World Simulator (Patient) Interface to surgeon console and training Application Control (MIRS) Hardware Abstraction Layer Workflow Safety Checks Control Hardware Abstraction Layer (HAL) Robot Simulator 1 Sensor/ Actuator. Robot Dynamics Interface to world simulator Motion Forces/ Torques Haptic Rendering World Visual Rendering Virtual Reality Robot Simulator 2 Design Under Test Simulator Folie 12

13 RSS - Simulation Platform For Workflow Design Relevant to surgical robotic workflow: change the surgical tool Folie 13

14 RSS - Simulation Platform For User Interface Design and Monitoring USER INTERFACE DESIGN and MONITORING WORKFLOW DESIGN User Interface (Surgeon Console) Robotic System Simulator (RSS) (for MIRS) World Simulator (Patient) Real Omegas Display Pedals GUI Planning Forces/Torques Positions Monitoring Parameterization Interface to surgeon console and training Application Control (MIRS) Hardware Abstraction Layer Workflow Safety Checks Control Hardware Abstraction Layer (HAL) Robot Simulator 1 Sensor/ Actuator. Robot Dynamics Interface to world simulator Motion Forces/ Torques Haptic Rendering World Visual Rendering Virtual Reality Robot Simulator 2 Design Under Test Simulator Folie 14

15 RSS - Simulation Platform For User Interface Design and Monitoring Folie 15

16 RSS - Simulation Platform For User Interface Design and Monitoring First tele-operated robot in space Compare simulation with reality during operation. Folie 16

17 RSS - Simulation Platform For Surgeon Training TRAINING Training Application USER INTERFACE DESIGN and MONITORING WORKFLOW DESIGN User Interface (Surgeon Console) Robotic System Simulator (RSS) (for MIRS) World Simulator (Patient) Surgeon Real Omegas Display Pedals GUI Planning Forces/torques Positions Monitoring Parameterization Interface to surgeon console and training Application Control (MIRS) Hardware Abstraction Layer Workflow Safety Checks Control Hardware Abstraction Layer (HAL) Robot Simulator 1 Sensor/ Actuator. Robot Dynamics Interface to world simulator Motion Forces/ Torques Haptic Rendering World Visual Rendering Virtual Reality Robot Simulator 2 User Under Test Simulator Folie 17

18 RSS Modular Simulation Platform For Various Use Cases TRAINING Training Application USER INTERFACE DESIGN and MONITORING WORKFLOW DESIGN User Interface (Surgeon Console) Robotic System Simulator (RSS) (for MIRS) World Simulator (Patient) Surgeon Real Omegas Display Pedals GUI Planning Forces/torques Positions Monitoring Parameterization Interface to surgeon console and training Application Control (MIRS) Hardware Abstraction Layer Workflow Safety Checks Control Hardware Abstraction Layer (HAL) Robot Simulator 1 Sensor/ Actuator. Robot Dynamics Interface to world simulator Motion Forces/ Torques Haptic Rendering World Visual Rendering Virtual Reality Robot Simulator 2 User Under Test Simulator Folie 18

19 RSS - Simulation at Different Levels Of ing Detail TRAINING Training Application USER INTERFACE DESIGN and MONITORING WORKFLOW DESIGN User Interface (Surgeon Console) Robotic System Simulator (RSS) (for MIRS) World Simulator (Patient) Surgeon Real Omegas Display Pedals GUI Planning Forces/torques Positions Monitoring Parameterization Interface to surgeon console and training Application Control (MIRS) Hardware Abstraction Layer Workflow Safety Checks Control Hardware Abstraction Layer (HAL) Robot Simulator 1 Sensor/ Actuator. Robot Dynamics Interface to world simulator Motion Forces/ Torques Haptic Rendering World Visual Rendering Virtual Reality Robot Simulator 2 User Under Test Simulator Folie 19

20 Layers of modeling details Layer 3: Kinematics Layer 2: Dynamics Layer 1: Implementation Layer 0: Real System High abstraction High detail Scaling of effort (Implementation and Computation) Different use cases require different modeling details Define component interfaces for each layer Folie 20

21 Layers of modeling details Layer 3: Kinematics Layer 2: Dynamics Layer 1: Implementation Layer 0: Real System High abstraction High detail models: kinematics, position, motion abstracts: dynamics (masses and forces) Folie 21

22 Layers of modeling details Layer 3: Kinematics Layer 2: Dynamics Layer 1: Implementation Layer 0: Real System High abstraction High detail models: abstracts: forces, torques w.r.t. motion, motor ripple, latency, friction, compliance implementation and computing hardware details Folie 22

23 Layers of modeling details Layer 3: Kinematics Layer 2: Dynamics Layer 1: Implementation Layer 0: Real System High abstraction High detail models: implementation details (cycle-true) abstracts: the real system Folie 23

24 Layers of modeling details Layer 3: Kinematics Layer 2: Dynamics Layer 1: Implementation Layer 0: Real System High abstraction High detail robotic hardware (e.g. MIRO, MICA, ) user interface (e.g. GUI, 3D Display, Omega Device) patient, animal model, phantom Layer 0 requires real-time simulation ( 1kHz ) Folie 24

25 Implementation Design: Main Components of a MIRS Simulator Folie 25

26 Implementation Design: Interface Definition for each Layer ibd [block] MIRS Simulator [Layer 3] Discrete Event communication shall be implemented as UDP 1kHz rate robotgeometry:objects db: GeometryDataBase :Objects robotgeometry:objects out:uimotion in:uimotion user interface: Surgeon Console cmd:commands robot :Robotic System Simulator out:motionlist in:motionlist world :World Simulator monitor:monitor monitor:monitor 3dImage:DVI Link The two main interfaces are defined for each Layer of modeling detail (Example shows Layer 3:Kinematics) Folie 26

27 Implementation Design: Adapters between Layers ibd [block] Surgeon Console [Adapter Layer 0 to Layer 2] out:uimotion left: Omega Force Feedback Device right: Omega Force Feedback Device :Omega DI :Omega DI left:omega DI right:omega DI RealSystemToLayer2 :Adapter in:uiforcestorques cmds:commands monitor:monitor :Display :DVI Example: Adapt Omega Input Devices (Layer 0) to Layer 2 Note: Real hardware binds the virtual simulation time to the physical real time. Folie 27

28 Implementation Design: Robotic System Simulator Example: RSS with three MIRO models One robot model for each robot Application Control connects to the Surgeon Console World Interface connects to the World Simulator Folie 28

29 Outline Introduction The SAFROS Robotic System Simulator Robotic System ing Conclusions Folie 29

30 Robotic System ing Robot Dynamics (RDM) Sensor/Actuator s Robotic System Simulator (RSS) (for MIRS) World Simulator (Patient) Application Control (MIRS) Hardware Abstraction Layer Workflow Safety Checks Control Motor current Motion/ Torques (measured) HAL Robot Simulator 1 Actuator (friction, ripple) Sensor (gain, offset, latency) Torques Motion/ Torques Robot Dynamics (kinematics, stiffness, damping, inertia). Interface to world simulator Motion Forces/ Torques Haptic Rendering World Visual Rendering Virtual Reality Robot Simulator 2 Together, RDMs, Sensor, and Actuator models simulate the robot s physical (Layer 2,3) and implemented (Layer 1) behavior. Folie 30

31 Robotic System ing: Robot Dynamics (RDM) Depending on the use-case different level of detail within a layer: Layer 2 Single Body (TCP) Low Implementation Time, Dynamics Multi-Body (robot-segments) Computational Costs, Finite Elements (robot-structure) High Degree of Details 1. Find the right model 2. Develop a calibration procedure to find the paramters for a specific robot Folie 31

32 Robotic System ing: Robot Dynamics (RDM) Kinematics Joint and structure stiffnesses/damping Inertia Influence of inertia on forces and torques Stiffness of a structure component Scheme of the MIRO kinematics Folie 32

33 Robotic System ing: Calibration Methods Static calibration is performed stepwise reduces complexity In 3 steps, different subsets of parameters are optimized Static robot poses for measurements are planned The measurement setup and a calibration tool are planned Steps are repeated to account for their dependencies [Klodmann et al., Static Calibration of the DLR Medical Robot MIRO, a Flexible Lightweight Robot with Integrated Torque Sensors, IROS 2011] Folie 33

34 Robotic System ing: Calibration Methods Folie 34

35 Conclusions Dynamic of the Robotic System enables Accurate Simulation Accurate Simulation of the Robotic System allows better Workflow Design User Interface Design Monitoring during operations Training Increased Patient Saftey Folie 35