ience e Schoo School of Computer Science Bangor University

Size: px

Start display at page:

Download "ience e Schoo School of Computer Science Bangor University"

Ophelia Maria Watts
6 years ago
Views:

1 ience e Schoo ol of Com mpute er Sc Visual Computing in Medicine The Bangor Perspective School of Computer Science Bangor University

2 Pryn hwn da Croeso y RIVIC am Prifysgol Abertawe Siarad Cymraeg? Schoo ol of Computer Sciencee

Visualization and Medical Graphics Group

John Ik Soo Lim Jonathan Roberts Bill Teahan

5 years: Satava Award 2006 50+ Publications

3 Visualization and Medical Graphics Group Established Sept 2003 Five academics Nigel John Ik Soo Lim Jonathan Roberts Bill Teahan Rafal Mantiuk Five RAs Chris Hughes Serban Pop Rick Walker Meirion Hughes Llyr ap Cynedd Five PhD Students Highlights of first 5 years: Satava Award Publications 1.8M Research Grants Hosted International Conferences

Growing trend to use Medical Virtual Environments and Augmented Reality Educational Tools Diagnostic Aid Patient specific data 3D visualizaton of

4 Growing trend to use Medical Virtual Environments and Augmented Reality Educational Tools Diagnostic Aid Patient specific data 3D visualizaton of medical image modalities Virtual Endoscopy Planning Aid Surgical planning Interventional radiology Radiotherapy treatment Rehearsal Aid Guiding Aid Training Aid

5 Multidisciplinary i li Team is Needed d Surgeons and clinicians the end users Clinical engineers Data measurements Build hardware Computer Scientists Write the software Integrate and maybe build the hardware system Psychologists Task Analysis Validation Studies School of Computer Science

6 Task Analysis Design of a medical simulator must begin with a detailed d task analysis Indentify all steps of the procedure What happens when things go wrong? Identify critical steps for safe completion of task; these can be used for metrics

7 Documented Task Analysis 7

8 Documented Task Analysis 8

9 Components of a Medical Simulator Input DICOM Data; 3D Geometry, e.g. surgical tools, needles Computer Segmentation 3D Reconstruction Collision detection Soft tissue deformation Force models Respiration Image generation Interaction Stereoscopy Haptics Physical Model

10 Interventional ti Radiology (IR) IR procedures use medical imaging to guide instruments to treat t specific pathology minimally invasively Vascular IR techniques involve blood vessels (arteries, veins) Visceral IR is concerned with organ systems (liver, bile ducts, urinary tract, gut) We have been building several simulators for (IR) that incorporate haptics...

11 Simulated Ultrasound Guidance (HTD funded) d)

12 Ultrasound-like like images are generated by: First extracting MPR image from the voxel data, or segmented mesh data, based on the position and orientation of the transducer Acoustic shadowing effects and noise are added using multitexturing. Ultrasound-like lik Images on the GPU 12

Simulation of X-ray Attenuation ti on the GPU With volume

computed on the GPU using Ray Tracing By considering the

data from only one time step See Poster at EG2010!

13 Simulation of X-ray Attenuation ti on the GPU With volume data, X-ray attenuation using the Beer Lambert Law can be computed on the GPU using Ray Tracing By considering the thickness penetrated by a ray into the object Usually have data from only one time step See Poster at EG2010! Works in conjunction with simulation of movement due to respiration Based on a surface model

14 ImaGINe-S Prototype t Validated for Biopsy and Nephrostomy procedures Second Place in 2009 Eurographics Medical Prize 14

15 Training i in Vascular IR Procedures 15

16 Replicating Real World Feel Needle force / motion tracking during arterial puncture Time (s) J Zhai, T How, Liverpool Clinical Engineering

17 Palpation Simulator Palpation can be used to identify the site of a needle puncture Requires both tactile and force feedback Prototype Haptics device for palpation NovINT Falcon provides force feedback Experimenting with tactile feedback interface - piezoelectric materials, speakers, pin arrays, hydraulics

18 Palpation Simulator Also using chromakey technology so that real hands can interact with CGI objects Novel for medical VE Collaboration with Italian Institute of Technology School of Computer Science

19 Fast Blood Flow Model with Particle Systems Flow in a horizontal channel with a bifurcation: As the vessel narrows the flow velocity increases:

20 Charged Particles for Soft Tissue Deformation with Haptics

21 Cost Effective Basic Ultrasound Training i Cost Effective Training for Ultrasonography Using Wii Remote with MotionPlus Aim is to deploy in developing countries Remote tutoring with Skype Touch cues Vibrate Wii Remote Shadow of transducer on patient s body 21

22 Bangor Augmented Reality Education for Teaching Anatomy (BARETA) RP Model is tracked using an Ascension minibird Software supports slab rendering and virtual resection We are part of a Marie Curie Initial Training Networks (ITN) proposalp Surgical Training with Augmented Reality Techniques, START School of Computer Science

23 Coloured Brain (EPSRC) Feasibility study with Bradford University and Leeds General Infirmary Create BRDF for a photorealistic surface model of a living human brain School of Computer Science

24 Web 3D Consortium (TATRC funding) Co-chair Medical Working Group Developed medx3d: Standards Enabled Desktop Medical 3D DICOM support Volume rendering in X3D Import/export library Ontology support Actively engaged with DICOM committees X3D could become DICOM s 3D standard 24

training process Moorthy, K., Mansoori, M.

Evaluation of the benefit of VR simulation

In Medicine meets virtual reality 12. pp.

25 Low Fidelity Simulators These training tools were built for Web-based delivery Used keyboard and mouse No haptics Validation study proved that they aid the training process Moorthy, K., Mansoori, M., Bello, F., et al. (2004). Evaluation of the benefit of VR simulation in a multi-media web-based educational tool. In Medicine meets virtual reality 12. pp Match the fidelity to the cues and actions required for task training

26 Mentoring in Second Life Create a secure teaching environment for remote mentoring of medical students Collaboration with Luton Hospital Uses Second Life and secure web server running SQL Booking and search facilities School of Computer Science

27 Research Challenges Strive for realism with real time interaction How high a fidelity simulation can be achieved? What fidelity of simulation is actually needed? Computer graphics research Photo realistic rendering, using the GPU, soft tissue deformation, blood flow, modelling tools, etc. Haptics research Holding virtual objects with your fingers, rather than via a tool Realistic force and tactile models School of Computer Science

28 Contact t Details Visualization and Medical Graphics Research Group, School of Computer Science, Bangor University n.w.john@bangor.ac.uk

Using Web-Based Computer Graphics to Teach Surgery

Using Web-Based Computer Graphics to Teach Surgery Ken Brodlie Nuha El-Khalili Ying Li School of Computer Studies University of Leeds Position Paper for GVE99, Coimbra, Portugal Surgical Training Surgical