Realistic Force Reflection in a Spine Biopsy Simulator

Proceedings of the 2001 IEEE International Conference on Robotics & Automation Seoul, Korea May 21-26, 2001 Realistic Force Reflection in a Spine Biopsy Simulator Dong-Soo Kwon*, Ki-Uk Kyung*, Sung Min Kwon**, Jong Beom Ra**, Hyun Wook Park** Heung Sik Kang***, Jianchao Zeng****, and Kevin R Cleary**** * Dept. of Mech. Eng. KAIST (Tel : +82-42-869-3042; Fax : +82-42-869-3210 ; E-mail: kwonds@me.kaist.ac.kr) ** Dept. of Elec. Eng & Computer Science. KAIST (Tel : +82-42-869-5434; E-mail: jbra@ee.kaist.ac.kr) *** Dept. of Diagnostic Radiology, Seoul National University College of Medicine (E-mail: kanghs@radcom.snu.ac.kr) **** ISIS of Georgetown Univ. (Tel : 202-687-8253; Fax : 202-42-784-3479 ; E-mail: cleary@isis.imac.georgetown.edu) Abstract This paper proposes a scheme to produce realistic force reflection in a needle insertion problem. The target system is a spine needle biopsy simulator for tumor inspection by needle insertion. Simulated force is calculated using a 3D human tissue model and the orientation and position of the needle, and is produced through the PHANToM TM device. To generate realistic force reflection, the directional force of the needle has been generated by a related tissue model, and the rotational force is generated using a pivot to keep the needle in the initial inserted direction after puncturing the skin. Since the haptic device has limitations in generating high stiffness and large damping, a scale-down model and digital filter are used to stabilize the system. 1. Introduction Medical engineering technology has kept apace with remarkable developments in computer hardware. Such development of medical technology has resulted in many improvements for the public welfare and for medical education practices. Especially there have been large strides in the development of medical simulators that help in the acquiring of medical skill and knowledge. The major difference compared to previous simulators is the use of haptic feedback. Watson et al. developed stereoscopic-haptic virtual environments [1]. Reinig et al. studied real-time visually and haptically accurate surgical simulation. In this study, they implemented an algorithm to produce the sensation of surgical cutting [2]. The application area of force feedback is also becoming larger and larger and many basic research projects have been also performed. Heimenz and Maa??experimented to determine the physical characteristics of different tissues [3,4]. Hopkins analyzed the biomechanical characteristics of muscle and bone [5]. Margaret developed a 3 DOF (degree of freedom) joystick that provides force feedback. Kwon studied a 6 DOF haptic controller for a telerobot system [6]. Massie developed the PHANToM TM device that has 6 DOF motion and can produce 3 DOF force [9]. The immersion Corporation developed the Laparoscopic Impulse Engine [10]. PHANToM TM and the Laparoscopic Impulse Engine are commercialized and used in many medical simulators. In this paper, we propose a method to generate realistic force reflection during needle insertion into a hollow dummy in a spine biopsy simulator. A spine needle biopsy is a useful non-invasive operation to detect and verify a spine tumor. For this operation, tissue is extracted from the spine of a living body using a hollow needle. However, since there are many critical organs near the spine, the spine needle biopsy operation requires complicated and accurate procedures. Hence, much training is essential for reliable operations and the haptic simulator can be a valuable tool int the training process. Singh and Popa measured the force from the needle during a needle biopsy and they suggest a scheme to generate a similar force profile in a needle biopsy simulation [7,8]. Cleary and Greco applied the PHANToM TM device to a 2D spine needle biopsy simulator [11]. In this paper, the haptic feedback scheme for a 3D spine needle biopsy simulator is proposed. The rest of the paper is organized as follows. We briefly introduce a surgical simulator for spine needle biopsy in Section 2. In Section 3, we present implementation details for force reflection. Finally, we present our results and state our conclusions in Section 4. 2. Spine Needle Biopsy Simulator In this section, we introduce a spine needle biopsy simulator that provides realistic visual and force feedback to a trainee in the PC environment [12]. This system is composed of four parts: a 3D human model, a visual feedback part, a force feedback part, and an evaluation part. 2.1 3D Human Model A 3D human model plays an important role in the spine needle biopsy simulator because most parts of this system use this model. Therefore all kinds of data should be included. The 3D human model has two types of data. One data type is common data. All voxels have common data such as CT density data, segmentation data, gradient data, and normal vector data. These common data are used by the visual feedback part, but in particular segmentation data is used by the force feedback part also. The other data type is individual data that is different for each kind of organ. Each organ has special characteristics such as 3D rendering parameters (color, opacity), and the force generation parameter (spring constant). The 3D model uses 335 XCT(computerized tomography) slices having an intra-slice resolution of 0.7mm and an inter-slice resolution of 1mm. This data set 0-7803-6475-9/01/$10.00 2001 IEEE 1358

projection. For fast 3D volume rendering, a block-based technique is adopted for the efficient handling of large amounts of data and for the easy control of rendering parameters such as opacity [15]. Figure 1. The Spine Needle Biopsy Simulator was provided by Seoul National University Hospital. In order to produce segmented 3D organ models based on these CT slices, a homemade semiautomatic segmentation tool is used. The target organs are selected as organs that should be needed in the procedure of simulating a spine needle biopsy. Doctors in the department of diagnostic radiology at Seoul National University Hospital determined these organs. For the lumbar vertebra region, bone, lung, esophagus, artery, skin, muscle, and fat are selected as the target. For the thoracic vertebra region, vein and kidney are additionally chosen. Therefore a total of nine organs are modeled in our 3D human model. 2.2 Visual Feedback Part Visual feedback includes all the GUI (graphic user interface) of the simulation system. Through the GUI of the whole procedure, a trainee has a visual experience of performing the spine biopsy. In our simulator, there are two types of visual feedback. One is a 2D virtual CT console and the other is a 3D view. The 2D virtual CT console box mimics the GUI and functions of a real CT system. In the 2D virtual CT console, a trainee can select a lesion that is a candidate for spine tumor. Then he simulates scanning CT images around the lesion. Many kinds of analytical tools are provided to the trainee in this console, such as finding the distance between two points, measuring the angle between two lines, zooming a specific CT image, etc. The second visual feedback part is a 3D view. In this visual feedback part, a 3D path planning tool is provided for the needle path from the skin to the lesion. Unlike previous 2D simulators, it supports realistic 3D volume rendering views interactively for 3D path planning. It provides axial, sagittal, coronal and 3D visualization images around the region also. Especially for the 3D visualization image, a trainee selects one image from among the color volume rendering result, gray volume rendering result, MIP (maximum intensity projection), and summed voxel 2.3 Force Feedback Part Force feedback increases the reality in the simulation system. The PHANToM TM haptic device of SensAble Technologies, Inc. is used and the spine biopsy needle replaces a stylus pen in order to implement directional and rotational force without any additional mechanical calculation. A haptic device is used to provide force feedback to the trainee through the biopsy needle during simulation. For proper force feedback generation, the 3D human model data that contains the force feedback parameters of organs near the spine is used and the haptic device is kept operating at 1kHz. A mock operation table that has same height as a real one is constructed and a dummy is laid down on the table. The trainee then inserts the spine biopsy needle into the hollow dummy on the table and feels force in a realistic environment. We provide further details in section 3. 2.4 Simulation This system is implemented by attaching a 3 DOF PHANToM TM device to a PC that has 600MHz Pentium III Dual CPUs and 512Mbyte RAM. A trainee simulates taking CT images of a patient and selects a lesion that is a candidate for a spine tumor in the 2D virtual CT console. In the 3D view, the 256x256x256 3D human model is rendered so that the trainee can plan the appropriate needle path from the skin to the lesion so as not to touch the other critical organs. After planning, he performs a needle biopsy virtually using the haptic device. When he moves the needle, the haptic device captures the movements of the needle and these captured movements are rendered over the 3D human model. According to the position of the needle, the trainee feels different force Figure 2. The physical user interface of the developed spine biopsy simulator. 1359

Equation (1) shows that the total force( F ) has three components: 1) the force required to penetrate tissue( F R ); 2) a correction force to keep the needle along the direction of movement( F ); and 3) an ambient force to compensate for C gravity( F ). This section explains how to calculate these G forces. Figure 3. Graphic user interface reflection. For a 256x256x256 XCT abdomen volume data set, if he moves the spine needle only, it provides an updaterate over 25 Hz for visual-feedback and over 1kHz for forcefeedback. When he touches the lesion with the needle tip, the simulation is over. 2.5 Evaluation Part The evaluation part provides a full performance analysis to the trainee after the simulation. This part includes information regarding the final position of the needle tip, a list of punctured critical organs, the maximum deviation from the planned path, the number of trials, and the total time spent. 3. Implementation of Force Reflection Figure 5 illustrates the force reflection mechanism for the needle biopsy simulator. 3.1 Tissue Penetration Resistive Force This force is simulated from the point of initial skin puncture until bone is touched. Since the maximum force that can be generated using PHANToM TM is 8.5N, the device has limited ability to simulate stiff bone[7]. In addition, since soft tissue, muscle and fat have nonlinear stiffness and damping properties, experiments and analysis are required for accurate modeling. Maa? developed a model after measuring their elastic properties using ultrasonic waves[3]. However, a needle biopsy has not only an elastic process, but also a tissue fracture process. In addition, the resistive force depends on the needle insertion speed[4]. In our simulator, experimental data was used to develop a tissue model. Singh showed experimental results for needle punctures and developed an interactive lumbar puncture simulator with tactile feedback[7],[8]. The direction of the tissue penetration resistive force depends on the needle direction. The upper limit of this force is set to 6N because the PHANToM can only generate 8.5N as a peak force. Figure 6 shows Singh s experimental data which was collected from a needle inserted at a constant velocity of 1 cm/sec [3]. We modeled each tissue as a spring without damping. Since position resolution at a needle tip is low and the damping display of a haptic device is poor[9], damping is not considered. To simulate a realistic force profile, stiffness varies with the depth of the inserted F? F? F? F (1) R C G Figure 4. Evaluation reports Figure 5. Force Reflection Mechanism 1360

Figure 6. Needle Puncturing Resistive Force [7] Figure 8. Simulated Resistive Force simulation results of the resistive force. Note the result of the model is similar to the experimental data of Figure 6. Figure 7. Force Generation Algorithm needle and the image segmentation value. Although the image segmentation classifies tissues as air, skin, muscle, lung, vein, artery, kidney and bone, the simulated force is derived from only air, skin, fat and bone because these are the tissues penetrated by the needle in most spine biopsies. If the biopsy path touches a lung or a kidney, the force falls to zero and a red warning sign appears on the screen. Figure 7 shows the algorithm used to calculate the tissue penetration force. Xr is a reference position to calculate force and this point is located on the needle. Xp is its tip position. k is stiffness of the tissue and vel is the speed and d is the depth of the needle insertion. Xr is determined with the inserted needle direction and the tissue that is punctured by needle. Variables, Xd, Xd2 are used to decide the time to change Xr. It is assumed that the skin is punctured when the needle is inserted to some determined depth. Figure 8 shows 3.2 Correction Force to Simulate Tissue Constraint In real a needle biopsy, the motion of the needle is constrained after insertion by the internal tissue. Moving the needle along the direction of the motion is not difficult, but it is very difficult to move to an orthogonal direction. Since the PHANToM(ver1.5) can only generate haptic translational forces (and not torques), an additional force is required to constrain the needle once it is inserted into the dummy. To solve this problem we make a hole at the surface of the dummy which is used as a pivot point to create rotational forces to constrain the needle along the inserted direction. At first, the direction vector of the needle is computed after the skin puncture. For calculation of the correction force, we assumed the following:? The needle is touching only muscle or fat.? The skin punctured position is a point and does not change.? The rotational angle is much smaller than 1 radian. The first assumption is reasonable because the skin is very thin and there is only muscle and fat between the skin and the bone. The second assumption is trivial. Since the side surface of the needle is not sharp like the tip and its motion is constrained by internal tissues except in the movement of the inserted direction, the third one is acceptable. Using these assumptions, the correction force is calculated using the torque derived from the force at the needle tip. Figure 5 shows the initial needle direction vector Vi and the current needle direction vector Vc. Vi is derived by multiplying the needle inserted unit vector with the needle length. Vc is derived from the current tip point and the gimbal point of the needle. The correction force Fc can then be derived from the difference of these two vectors. The upper limit of this force is 2N since the haptic device can only generate 8.5N and the limit of the tissue penetration force is 6N. Since the tissues 1361

between skin and bone are muscle and fat, the stiffness for the correction force is determined from muscle and fat and this force is calculated by multiplying the correction vector by a stiffness and a constant. From the experimental results, we verified that this correction force helps to keep the needle along the initial inserted direction. We can confirm that the correction helps keep the needle in the initial inserted direction. Figure 9 shows the simulated force that must be applied by the haptic device. Fc_sim is derived as follows: ll Fc _ sim?? Fc (2) l 2 Figure 9. Conversion of Correction Force Figure 10. Puncture Resistive Force over Time 3.2 Compensation of Stiffness only Model and Stabilization for hard contact to bone Since the spring model is applied to virtual tissue, the needle jumps out when the user releases it during insertion. The user even feels the pushed force during pulling the needle out. Figure 7 shows the algorithm to simulate the cut feeling after tissue puncture. The velocity value is used to detect the current moving direction, and pulling out and inserting is distinguished from this value. Using these, the resistive force falls to zero as the needle is pulled out. For more realistic modeling, viscosity or a viscoelastic tissue model is required. Vibration occurs when the needle contacts the bone. There are two reasons for this. First, mixed volume data with low resolution and abruptly changing stiffness can cause force discontinuities near the bone. This problem has been solved by using a digital filter. Second, the haptic interface has a limited ability to simulate high stiffness. Since the stiffness of the bone is high, it is not typically penetrated by needle puncture alone. As Colgate suggested s passivity condition to guarantee stability[13], stiffness is scaled down to satisfy the stability condition. Figure 10 and Figure 11 show the tissue penetration resistive force and inserted depth of the needle over time. The system is stable when it touches the bone. The resistive force falls to zero as the needle is pulled out. 4. Conclusion The simulated force is calculated from the relationship between volume image data and the orientation and position of the needle. For more realistic force reflection in the spine needle biopsy simulator, the directional force of the needle Figure 11. Inserted depth of needle has been generated by a tissue model. The rotational force is generated using a pivot to keep the needle in the initial inserted direction after puncturing the skin. Since the haptic device employed has limited ability to generate high stiffness and large damping, a scaled down stiffness model and digital filter are used to stabilize the system. For a more realistic simulator, tissue modeling based on biomechanics and haptic rendering technique considering voxel data are required. In addition, a device and controller to generate high stiffness and damping with stability is necessary. Acknowledgements This research was supported by KAIST VRRC(Virtual Reality Research Center) and the Korean Ministry of Information and Communication as an international joint project with ISIS(Image Science and Information Systems) Center at Georgetown University Medical Center, USA. REFERENCES [1] Watson K. et al., Development of stereoscopic-haptic virtual enviro nments, 12th IEEE Symposium on Computer-based medical systems, pp. 29-34, 1999. [2] Karl D. Reinig et al., Real-time Visually and Haptically Accurate surgical simulation, 1362

http://www.uchsc.edu/sm/chs/research/mmvr4.html. [3] Maaß H.and Kuhnapfel U., Noninvasive Measurement of Elastic Properties of Living Tissue, 13th International. Congress on Computer Assisted Radiology and Surgery (CARS '99), Paris, France, June 23-26, pp 865-870, 1999. [4] L. Heimenz and A. Litsky, Puncture Mechanics for the Insertion of an Epidual Needle, Americal Society of Biomechanics 21th Annual Meeting, 1997. [5] David A. Hopkins, Musculoskeletal Biomechanics Research, http://dahweb.engr.ucdavis.edu/dahweb/ dahsite/dahsite.htm, 2000. [6] Dong-Soo Kwon, Ki Young Woo, and Hyung Suck Cho, Haptic Control of the Master Hand Controller for a Microsurgical Telerobot System, Proc. of the IEEE Int. Conf. on Robotics and Automation, pp. 1722~1727, 1999. [7] Sunil K. Singh and Mikael Bostrom, Design an Interactive lumbar Puncture Simulator with Tactile Feedback, IEEE International Workshop on Robot and Human Communication, 1993. [8] Dan O. Popa et al., Creating Realistic Force Sensations in a Virtual Environment: Experimental System, Fundamental issues and Results, Proc. of the IEEE International Conference on Robotics and Automation, 1998. [9] Thomas Massie, Design of a Force Reflecting Fingertip Stimulator, Bachelor s Thesis, Department of Electrical Engineering and Computer Science, MIT, 1993. [10] Immersion Co., Products and Technologies, http://www.immerse.com/products.html, 2000. [11] Kevin Cleary and Robert Greco, Development and Evaluation of a Spine Biopsy Simulator, Proceeding of Medicine Meets Virtual Reality, vol. 50, pp. 375-376, 1998. [12] J. Kim et al, Development of a Spine Needle Biopsy Simulator with Vis ual and Force Feedback, Proceeding of Congress on Computer Assisted Radiology and Surgery (CARS) 99, pp. 1052, Paris, France, June 23-26, 1999. [13] Edward J. Colgate and Gerd G. Schenkel, Passivity of a Class of Sampled-Data Systems: Application to Haptic Interface, American Control Conference, Baltimore, pp. 3236-3240, 1994. [14] Paul Gorman, Thomas Krummel et al, A Prototype Haptic Lumbar Puncture Simulator, Proceeding of Medicine Meets Virtual Reality 2000, IOS Press, vol. 70, pp. 106-109, 2000. [15] S. M. Kwon, J. K. Kim, and J. B. Ra, A Block-Based Volume Rendering Scheme for the Spine Biopsy Simulator, Proc. IEEE ICIP 1999, vol. 2, pp. 187-191, Kobe, Japan, 1999. 1363