Robotic Applications in BioMedicine: From Molecular Imaging to Heart Surgery Medical Robotics Laboratory, PGH 315 Department of Computer Science

Transcription

1 Robotic Applications in BioMedicine: From Molecular Imaging to Heart Surgery Medical Robotics Laboratory, PGH 315 Department of Computer Science Nikolaos V. Tsekos Junmo An, Nicholas von Sternberg, Giulia Toti (*), Habib Zaid, Mahmut Unan PhD: Nikhil Navkar, Ahmet Sonmez, Erol Yeniaras, Fabien Benoteau, MS: Johann Lamaury, Yousef Hedayati, Daniele Faris BSc (BME): Habib Zaid

2 What do we do? Develop computational methods and hardware, and integrate them for performing image-guided and robot-assisted interventions and surgeries From Concepts to Actual Working Prototypes! MRI robot control MRI Surgical Robots Haptics Endoscopic Sensors 1 mm MRI Robot Tracking

3 MIROS: A Cyber-Physical System Built from the Ground Up in Our Department! We bridge and link the physical and the cyber world

4 MIROS Multimodal Image-guided RObot-assisted Surgeries MR Scanner ScannerState(t) MRData(t) AoP(t) GPU Visualization Module Visual interface MRData(t) Sensor-Control Module Motion-Filter Module GPGPU FPGA-2 Data Processing Module {TP J (r, t)} {GC J (r, t)} AoP(t) AoP(t): {Encoders ROBOT (t)} MARK/TUBE Collision Detection Collision Response OP IN (t) OP GUIDE (t) Force- Feedback Module {Encoders FFI (t)} {DoF FFI (t)} ROBOT FPGA-1 Robot Control Module RobotState(t) Force-feedback Interface {DoF ROBOT (t)}

5 Computational Thread: Processing Dr. N. Navkar Real-time MRI feed Slices #1 #2 #3 Three oblique-to-each-other slices: 49 ms/slice (20 Hz) Projection Bands Slice #1 Slices #2 & #3 Control Curves Diastole Systole Diastole Systole Force Feedback Interface Dynamic Virtual Fixtures Visual Interface

6 Cardiac Surgeries (Fast-fast-fast!!!) Virtual Fixtures Generated On-the-Fly from the Continuous Feed of the Real-time MRI Guidance Virtual Fixtures Forbidden Region Virtual Fixtures Maneuver on the curve Maneuver inside the corridor NO NEED for Models, Assumptions and unnecessary computational layers!

7 Translation rods dspace Unit Knob A Yaw motor (b) capstan wire Tip Angulation Motor UNIT A (a) UNIT B Knob B (c) Pitch Motor Translation Motor 1 wires (c) Motor 2 Roll Motor Knob A Tip Angulation Motor Knob B (d) Linear bearings (e) (f) wire Capstan

8 Enhanced Reality and Haptics Dr. N. Navkar & Dr. E. Yeniaras The forces exerted on the operator direct the actual robot (gray) to match its proxy (green) The proxy is calculated to stay always inside the forbidden region corridor Force-Feedback Interface (In-house developed 5 DoF) Visualization Interface (High Definition LCD)

9 In-house Developed MR-compatible Robot N. Sternberg (PhD); H. Zaid, BSBME; J. An (PhD) Upper stage 1 DOF Middle stage 2 DOF

10 A B

11 3mm Tracking A Robot With MRI Junmo An (PhD) L C R 3mm Container with MR signal source Tuning Capacitor Photo-resistor Robot Controls which markers are MR visible! -Faster - Less Computing! Light protective sleeve Lens cap Optical fiber Cable (15m)

12 The MRI+Optical Needle Probe Dr. Ahmet Sonmez MR Spectroscopy Sensitive Area of the RF Coil Light Induced Fluorescence SI 1500 (a.u.) mm comp-3 comp-2 comp um co-axial cable 5000 to RF coil 0 SI (a.u.) comp-3 comp-2 comp-1 RF Coil Needle Wall Optical Sensor Laser emission (a) Light detection 1 Z (cm) x 300 um 2 optical fibers 3 to optical sensor (b) 4 1 Chemical shift (ppm) 0 1 Z (cm) RF Coil Wavelength (nm) The first ever endoscopic work on concurrent MRI and optical sensing! 600

13 Thank you! Visit us at PGH 315