Tactile Interactions During Robot Assisted Surgical Interventions. Lakmal Seneviratne

Transcription

1 Tactile Interactions During Robot Assisted Surgical Interventions Lakmal Seneviratne Professor of Mechatronics Kings College London Professor of Mechanical Eng. Khalifa Univeristy, Abu Dhabi. 1

2 Overview 1. Surgical Robotics Soft Robots 2. Tactile Interactions - Learning 3. MRI Compatible Force Sensing 4. Haptic Interfaces 5. In Hand Manipulations 2

3 King s College London 3

4 Khalifa University Established in 2009 Vision - To be a leading international center of higher education and research 4

5 1. Surgical Robotics Soft Robots 5

6 Surgical Robotics - Da Vinci Surgical System 6

7 Surgical Robotics Robotic Catheterisation Hansen Medical: Robot-steered catheterization tool for cardiac ablation procedures. 7

8 MIS Tools enter the body through narrow openings and manipulate soft organs that can move, deform, or change in stiffness. Teleoperated Restricted access (through Trocar ports), minimal haptic feedback, rigid robot tools, confined space, safety-critical, real-time, bio-compatible, sterilized tools, MRI compatible. 8

9 STIFF-FLOP - Inspiration 10

10 STIFF-FLOP Biological inspiration taken from octopus Soft, highly redundant manipulation device, Embedded distributed sensing cognitive development and intelligent control Learning and cognitive reasoning - learn from physical interactions with environment, 11

11 STIFF-FLOP STIFF-Flop Concept 12

12 Octopus-like robot arm EU Project OCTOPUS 13

13 2. Tactile Interfaces and Learning 14

14 Tissue Palpation Ex-Vivo Test Rig for Indentation Ex-vivo indentation tests on bovine liver - Measure Force Vs Displacement characteristics 15

15 Dual Maxwell Model for Palpation Bovine Liver Static indentation (6mm probe) Talal M. Al-ja'afreh, Yahya H. Zweiri, Lakmal D. Seneviratne, and Kaspar Althoefer, A New Soft Tissue Indentation Model for Estimating Force-Displacement' Characteristics using Circular Indenters. Proc IMechE,

16 Rolling Palpations Ex-Vivo Test Rig (Rolling Device) Horizontal Motion Fz (Normal Force) Vertical Motion Fx (Drawbar Pull) Force/Torque Sensor Wheel Tissue 6-DOF robotic Manipulator 17

17 Rolling Palpations Soft Tissue Kidney sample with an embedded nodule Test results on pork kidney embedded with simulated tumor Liu, H, Noonan, D. P., Challacombe, B. J., Dasgupta, P., Seneviratne, L. D., Althoefer*, K, Rolling Mechanical Imaging for Tissue Abnormality Localization During Minimally Invasive Surgery, IEEE Transactions on Biomedical Engineering,

18 Rolling Imaging Rolling imaging from a silicone phantom embedded with 9 nodules 19

19 Rolling Images 20

20 Rolling Indentation of Human Prostates Phantom Omni Rolling Probe 2D 3D 21

21 FE Model for RI 22

22 3 nodules, 10 mm diameter - Arruda-Boyce model. FE model parameters from unixial tests. Material μ,shear Modulus (kpa) λ m, Locking stretch Mass Density (kg/m 3 ) Type of mesh Rubber (N1) CPS4R FE results are in good agreement with the corresponding experimental data, RMS range % Silicone (RTV6166 gel) CPS4R K. Sangpradit, H. Liu, P. Dasgupta, K. Althoefer, and L. Seneviratne, Finite Element Modelling of Rolling Indentation IEEE Transactions on Biomedical Engineering,

23 3. Force Sensing for Surgical Applications 24

24 Fibre Optic Uni-Axial Force Sensor P. Puangmali, H. Liu, K. Althoefer, and L. D. Seneviratne, Optical Fibre Sensor for Soft Tissue Investigation during Minimally Invasive Surgery, ICRA

25 Fibre Optic 3 Axis Force Sensor 3-Axis Force Sensor P Puangmali, H Liu, L D Seneviratne, P Dasgupta, K Althoefer. Miniature 3-Axis Distal Force Sensor for Minimally Invasive Surgical Palpation. IEEE/ASME Transactions on Mechatronics. 2011, 26

26 Fibre Optic Stiffness Sensor Force sensor The miniaturized sensor (11 mm diameter) consists of a force sensor and four displacement sensors Fibre optic technique is applied, MRI-compatible Displacement sensor Panagiotis Polygerinos, Lakmal D. Seneviratne, and Kaspar Althoefer, Triaxial Catheter-Tip Force Sensor for MRI-Guided Cardiac Procedures, IEEE/ASME Transactions on Mechatronics

27 Fibre Optic Stiffness Sensor Indentation Depth sensing (different Orientations) Force Sensing (a) (b) Mini FID θ z Calibration results of fibre-optic force sensor (c) Silicone Hongbin Liu, Jichun Li, Xiaojing Song, Lakmal Seneviratne, Kaspar Althoefer. "Rolling Indentation Probe for Tissue Abnormality Identification during Minimally Invasive Surgery", IEEE Transactions Robotics

28 Fibre Optic Force Sensing for Cardiac Catheters Wei Yao, Tobias Schaeffter, L Seneviratne. K Althoefer, MR-compatible Catheter for Cardiac Catheterization, ASME Journal of Medical Devices,

29 3-Axis Catheter Force Sensor 12Fr catheter-tip integrated with tri-axial force sensor. Fibre-optic catheter-tip force sensor. Panagiotis Polygerinos, Asghar Ataollahi, Tobias Schaeffter, Reza Razavi, Lakmal D. Seneviratne, and Kaspar Althoefer. MRI-Compatible Intensity-Modulated Force Sensor for Cardiac Catheterization Procedures. IEEE Transactions on Biomedical Engineering, 58 (3), pp

30 Airflow Force Sensor Constant pressure on 9mm Sphere. Displacement of the ball indicates a change in stiffness of the surface tissue. 31

31 Air Flow Tactile Probe Direct stiffness indication Tuneable force range (depend on the inlet air pressure) Friction free rolling over the tissue Array of Tactile Element (sphere diameter 4mm) Sensitivity , diameter 18mm Indika Wanninayake, Lakmal Seneviratne, Kaspar Althoefer, Novel Indentation Depth Measuring System for Stiffness Characterization in Soft Tissue Palpation. IEEE ICRA

32 Air Flow Tactile Probe 33 33

33 Airflow Force Sensor Experimental Evaluation The location of the 3 nodules on in the silicone phantom correspond to peaks. 34

34 Airflow Force Sensor Low friction sensor motion Simple design with potential for miniaturisation Can be built from MR-compatible materials 35

35 Pseudo-Haptic Feedback Real soft tissue Tissue properties Virtual soft tissue Tissue properties Rolling indentati on probe Haptic feedback RMIS system Force matrices of different indentation depths Robot arm Rolling probe Soft tissue Force sensor Palpation input device 36

36 Palpation simulation system with touch pad as input device 37

37 Palpation with Pseudo-Haptic Feedback z Virtual force Cursor v Real position x y Soft tissue Tumor 38

38 Pseudo-Haptic Feedback (a) cursor speed (b) cursor size Min Li, Lakmal Seneviratne, Kaspar Althoefer. Tissue Stiffness Simulation and Abnormality Localization using Pseudo-Haptic Feedback. IEEE ICRA

39 HANDLE EU FP7 IP UPMC (France), Shadow (UK), UC3M (Spain), FCTUC (Portugal), KCL (UK) ORU (Sweden), UHAM (Germany), CEA (France), IST (Portugal) Project Objectives Characterization of object affordances Learning and imitation of human strategies in handling tasks Improving skills through 'babbling' Autonomous in-hand dextrous manipulation 40

40 Learning Through Touch Hongbin Liu, Lakmal Seneviratne, Kaspar Althoefer. Real-Time Local Contact Shape and Pose Classification using a Tactile Array Sensor. IEEE ICRA

41 Learning Through Touch Right angle edge e 1 e 2 e 3 λ 1 =121.9 λ 2 =22.5 λ 3 =7.5 vertex (corner) e 2 e 3 e 1 λ 1 =30.1 λ 2 =8.45 λ 3 =6.23 e 3 flat surface e 1 e 2 λ 1 =123.6 λ 2 =48.8 λ 3 =3.7 cylinder (r = 6 mm) e 2 e 3 e 1 λ 1 =50.7 λ 2 =20.6 λ 3 =20.5 e 1 e 3 concave surface (r = 45 mm) e 2 e 3 e 1 λ 1 =123.6 λ 2 =53.3 λ 3 =7.5 square (a = 8 mm) e 2 e 1 λ 1 =35.1 λ 2 =16.7 λ 3 =15.7 sphere (r = 6 mm) e 2 e 3 e 1 λ 1 =28.7 λ 2 =18.1 λ 3 =15.6 ring (r outer = 10 mm, r inner = 4 mm ) e 3 e 2 λ 1 =42.2 λ 2 =36.3 λ 3 =

42 Challenges Human-Robot system Human in loop to deal with uncertainties. Monitoring, error recovery Perception Multi-modal (Tactile, Vision, etc) Grasping and Manipulations Tactile Interactions Control, learn Soft systems 43

43 Ribeiro 44