An Initial Trial off a Prototype Telepathology System Featuring Static Imaging With Discrete Control off the Remote Microscope

Transcription

1 Anatomic Pathology / ORIGINAL ARTICLE An Initial Trial off a Prototype Telepathology System Featuring Static Imaging With Discrete Control off the Remote Microscope Thomas Vishnu S. Winokur, Reddy, Gary Grimes, MD,1* Stan McClellan, MD,1 Catherine PhD,* PhD,* Gene P. Siegal, MD, M. Listinsky, MD,1 David Conner, PhD,* and J ay M. McDonald, Greg Vaughn, PhD,1* PhD,* Jay Goldman, DSc, MD1* Key Words: Telepathology; Remote diagnosis Abstract Routine diagnosis of pathology images transmitted over telecommunications lines remains an elusive goal. Part of the resistance stems from the difficulty of enabling image selection by the remote pathologist. To address this problem, a telepathology microscope system (TelePath, TeleMedicine Solutions, Birmingham, Ala) that has features associated with static and dynamic imaging systems was constructed. Features of the system include near real time image transmission, provision of a tiled overview image, free choice of any fields at any desired optical magnification, and automated tracking of the pathologists image selection. All commands and images are discrete, avoiding many inherent problems of full motion video and continuous remote control. A set of 64 slides was reviewed by 3 pathologists in a simulated frozen section environment. Each pathologist provided diagnoses for all 64 slides, as well as qualitative information about the system. Thirty-one of 19 diagnoses disagreed with the reference diagnosis that had been reached before the trial began. Of the 31, 13 were deferrals and 1 were diagnoses of cases that had a deferral as the reference diagnosis. In 6 cases, the diagnosis disagreed with the reference diagnosis yielding an overall accuracy of 96.9%. Confidence levels in the diagnoses were high. This trial suggests that this system provides high-quality anatomic pathology services, including intraoperative diagnoses, over telecommunications lines. Increasingly in medicine, electronic images are replacing film-based media. Radiology, remains the one medical specialty that routinely uses electronic images for diagnosis.1-3 The reasons for the disparity between radiology and other medical specialties are varied but of special importance is the requirement for color in pathology, as well as in other clinical specialties. Pathology also requires multiple fields of view and multiple optical magnifications. Telepathology, the use of images transmitted over telecommunications media for diagnosis, has been successfully implemented in a small number of settings, but widespread acceptance has remained elusive.4-9 Two quite different models of telepathology have been developed. The "dynamic model" uses real time transmission of video images along with remote control of the microscope to provide immediate diagnoses using images selected at the home site. The "static model" uses still digital images selected at the remote site and transmitted at a later time for remote diagnosis.10-1 Each of these models has distinct advantages. The dynamic model allows the remote user to select the images for viewing. It is useful for real time intraoperative diagnosis or consultation between pathologists, but it requires high bandwidth, and most of the images transmitted are of modest quality. The static model usually provides images of superior quality, but the number of images is limited. It is usually not feasible to transmit the images in real time, and the selection of images by the remote site requires that pathologists share the interaction. Because bandwidth requirements are low for this system, images can be transmitted over the Internet.13 Investigators at the University of Alabama at Birmingham Department of Pathology and the University of Alabama at Birmingham Center for Telecommunications Education and Research have collaborated with the sponsorship and cooperation of the BellSouth Corporation (Atlanta, Am J Clin Pathol 1998; 110:

2 Winokur et al / A PROTOTYPE TELEPATHOLOGY SYSTEM Ga) to design and implement a telepathology microscopy system (TelePath, TeleMedicine Solutions) that has important features of static and dynamic telepathology systems. Unlike the currently available dynamic systems, no video images are transmitted and no continuous control of the microscope is involved. All images are high-resolution and full color. The pathologist controls the distant microscope, including the x-y stage position, light intensity, and autofocus. In addition, a low-magnification image, produced by tiling individual microscopic images, serves as an image map allowing the pathologist to choose any field of interest at any desired optical magnification. It is our hypothesis that this will prove to be a highly satisfactory system for providing intraoperative diagnoses from a remote site. We report the success of our initial trial in using frozen and fixed pathologic specimens to give diagnoses remotely. Preliminary results of this study have been presented at pathology and engineering conferences Materials and Methods System Architecture The TelePath system was designed as a client-server application. The server, located at the remote site, consists of an Olympus Vanox microscope (Tokyo, Japan) with a Prior motorized stage to provide control over the x-y coordinates of the slide. A Sony DKC 5000 (Catseye; Park Ridge, NJ) camera is mounted to the microscope using a Image I I Overview image obtained by tiling multiple microscopic images. Images are abutted to each other without an attempt to match image features. Each image is separated by a 1-pixel border. Final optical magnification of each field is x1.33. Each tile is 19 x 18 pixels. 44 Am J Clin Pathol 1998;110:43-49 Diagnostics adapter with a 0.67x reducing magnification to increase the field of view of the camera. This is a 3 CCD camera head with an attached digitizer that can provide an interpolated image of 1,590 x 1,196 pixels or a native image of 795 x 598 pixels. All images used in the TelePath system are 795 x 598 pixels. This camera is capable of resolving 800 TV lines. The camera is connected through a SCSI interface to a Hewlett Packard Pentium (90 Mhz; Palo Alto, Calif) computer running Windows 95 (or Windows NT) (Microsoft, Redmond, Wash). Images are captured and transmitted at a color depth of 4 bits per pixel. Images have been transferred over multiple telecommunications media including ethernet, ATM (asynchronous transmission mode), a dedicated Tl line, multiplexed ISDN (51 kilobits/s [Kbs]), and single ISDN (18 Kbs) although any common transmission medium that uses the transmission control protocol/internet protocol (TCP/IP) transmission protocol is acceptable. The client, which the remote reviewing pathologist uses to command the microscope and view images, is a Hewlett Packard Pentium (90 Mhz) computer running Windows 95 (or Windows NT). An NEC 1" monitor with a viewing size of 1,600 x 1,00 pixels is attached. The client provides a virtual microscope environment and commands the server to issue autofocus, illumination intensity, and slide position commands. The client is multisession-capable allowing a client to connect to multiple servers simultaneously. All images are viewed at 4 bits per pixel. System Operation The client establishes a connection with the server and requests a slide overview. In the current trial, the tissue coordinates are present within a database stored in the server. After an autofocus procedure, the server captures the entire tissue section as a series of images, each of which is captured at a total optical magnification of xl.34 and a digital resolution of 19 x 18 pixels. Stage movement and image capture occurs in less than 1 second. These images are assembled by tiling to form a single image of the tissue present on the slide Image II. This image is transmitted and displayed on the client monitor. A right mouse click with the cursor over the image will result in a short menu of choices for the pathologist I Figure II. After the menu selection, a box appears on the screen that can be moved over any area of interest in the overview image. After a left mouse click, the stage moves to the location selected with the cursor, changes the objective, and the microscope performs an autofocus to assure that the image will be focused when captured. If the image is too dim, the light intensity can be changed by a command to the server to change a neutral density filter contained within the microscope. The uppermost image on the desktop is always the active image. Further actions on that image will return

3 Anatomic Pathology / ORIGINAL ARTICLE K ST- ;,.- W,, U,l,. 3 n a a! 1 O-IAYI >j : l.. ;..! v. l i v *H- hl.-rj L. * - \I ->JK i M M m Y tv j M to 1 ii *^IL yp*m i -^^fe^a^fe^ Figure I I A, Full digital resolution image selected from the tiled image in Image 1 showing the menu available to the pathologist with a right mouse click. Choosing "zoom" results in a freely movable selection box the same size as the field of view of the next objective. A subsequent left mouse click changes the objective, autofocuses the microscope, and transmits the selected field. B, Full digital resolution image using a 40x objective. Diagnosis Benign Classification Malignant Classification Deferred Confidence level Sure Very confident Somewhat confident Unsure nnnn Trial Design Reason for uncertainty Image quality Nature of specimen Insufficient expertise nnn Three pathologists used the TelePath system to diagnose a library of 64 slides. These slides were evenly divided between actual frozen sections and paraffin sections of unfrozen tissue from the same case. This selection of a mixture of frozen and paraffin preparations was chosen to assure that the image quality was sufficient to allow diagnosis of suboptimal tissue preparations. Slides were presented to the pathologists in different random orders. In all but cases, the diagnosis of the frozen section and the paraffin section were identical. Each pathologist entered the diagnosis on a coding form that contained a 1-sentence patient history IFigure 1. The diagnoses were given in fashions. First a diagnosis of malignant, benign, or deferred was given. This was supplemented, if necessary, with a text diagnosis (some slides, such as margins of resection, had no text diagnosis provided). In addition, the pathologist was asked to give an estimate of confidence in the diagnosis, a reason for uncertainty, an evaluation of the image quality, and an evaluation of system speed. Slide number Date Pathologist Brief Clinical History Image quality Excellent Good Acceptable Unacceptable Good Acceptable Unacceptable nnn the slide to the appropriate position and the objective turret to the correct magnification. All images requested by the client are archived along with a time stamp with their coordinates relative to the stored overview image allowing reconstruction of the pathologists "trajectory" in viewing the slide. A database viewer is also available to view the images that were captured. Speed IFigure 1 Data form used by the pathologists in the trial. Classification refers to a text diagnosis. Diagnoses were compared with a reference diagnosis prepared before the trial by the study-coordinating pathologist who did not participate in the telepathology trial. These Am J Clin Pathol 1998; 110:

4 Winokur et al / A PROTOTYPE TELEPATHOLOGY SYSTEM diagnoses also were given as malignant, benign, or deferred. A text diagnosis also was available. Concordance was evaluated at the level of whether the correct evaluation of malignancy was reached, similar to the evaluation used in assessing accuracy of frozen sections. Discordance was evaluated for clinical significance and simple disagreement. Results A total of 19 diagnoses were given by the 3 pathologists. The results are given in ITable II. On 45 of the 64 slides, there was complete and accurate agreement. Thirtyone diagnoses disagreed with the reference diagnosis. Of these, 13 disagreements were deferrals when the reference diagnosis was definitive, and 1 were slides given a correct definite diagnosis when the reference diagnosis was a ITable II Results of the Trial Compiled by Pathologists* Pathologist Total diagnoses Standard deferred1 Deferral* False negative False positive Mean confidence Mean confidence!! A 15 C /3 1s / / deferral. The pathologists individually had 11, 10, and 6 deferrals. In only cases did all 3 pathologists defer the diagnosis. One of these was an atypical intraductal hyperplasia of the breast for which the reference diagnosis was a deferral. The second was a case of tubular carcinoma of the breast. In all other cases at least 1 pathologist reached a definite diagnosis. The cases for which a deferred diagnosis was provided are listed in ITable 1. Only six diagnoses resulted in disagreements with the reference diagnosis. These are summarized in ITable 31. Two pathologists misdiagnosed a follicular variant of papillary carcinoma as benign using the frozen and paraffin sections. Because of the problematic nature of the case, it was referred to an expert consultant who agreed with the reference diagnosis of papillary carcinoma. The expert, however, noted that the majority of the nodule had the appearance of a hyperplastic lesion (the diagnosis given by the trial pathologists). An invasive ductal carcinoma was misdiagnosed as benign, and an intraductal hyperplasia was overdiagnosed as an intraductal carcinoma. An apparent error occurred on an endometrial biopsy specimen that was called malignant. In this case, the text diagnosis was correctly specified as adenomatous hyperplasia. The overall diagnostic accuracy of the trial was 96.9% (6 errors of 19 diagnoses). If the deferred diagnoses are removed, the overall accuracy falls slightly to 96.4% (6 errors of 165 diagnoses). The subjective confidence level was specified on a scale of 1 to 4 with 1 indicating "sure." The pathologists ratings *Confidence levels were scored as 1 = sure; = very confident; 3 = somewhat in the study clustered around, which was classified on the confident; and 4 = unsure. There was complete agreement on 45 of 64 slides. coding form as "very confident." Removal of the deferred ^Cases in which the reference diagnosis was deferred and the pathologist gave a diagnosis. diagnoses improves the confidence levels slightly. As shown *Total number of deferrals/number of deferrals in which the reference diagnosis was in ITable 41, uncertainty was attributed to all 3 available not a deferral. ^Reference diagnosis, adenomatous hyperplasia of endometrium. Pathologist A, responses, with the "nature of the specimen" the most adenomatous hyperplasia vs carcinoma in situ. common cause of uncertainty followed by "image quality." ^Confidence excluding deferred diagnoses. ITable 1 Cases in Which the Diagnosis Was Deferred* Case No. Description Type of Section (No. of Deferrals) ? Osteomyelitis in a patient with chronic myelogenous leukemia Lymph node frozen section with atypical lymphoid infiltrate Lymphoma Follicular variant of papillary carcinoma Endometrial hyperplasia Atypical hyperplasia of the breast Cystadenocarcinoma of pancreas Synovial sarcoma Chondroid hamartoma Nodular hyperplasia of the thyroid Tubular carcinoma of the breast Invasive ductal carcinoma of the breast Fibrocystic disease of the breast with florid hyperplasia Intraductal proliferation of the breast with atypia Frozen (); paraffin Frozen () Paraffin Frozen (3) Paraffin Frozen Paraffin (3) Paraffin Frozen () *The number of deferrals for each case is in parentheses. These cases are, in general, difficult and many diagnoses were deferred for the frozen and the paraffin sections. 46 Am J Clin Pathol 1998;110:43-49 American Society of Clinical Pathologists

5 Anatomic Pathology / ORIGINAL ARTICLE Table 31 Disagreement With the Reference Diagnosis* Type (No.) of Deviations From the Reference Diagnosis Diagnosis Follicular variant of papillary carcinoma of the thyroid Invasive ductal carcinoma of the breast Invasive ductal carcinoma of the breast False negative (4) False negative False positive The case of papillary carcinoma of the thyroid was problematic for the pathologists. Table 41 Reasons Cited for Uncertainty of Diagnosis Pathologist Reason for Uncertainty Image quality* Nature of specimen Insufficient expertise Multiple responses selected No reason given (All are "sure" or "confident" diagnoses) A B C *Cited as the reason for uncertainty for 4 frozen sections and 14 paraffin sections. Image quality was generally considered acceptable, but unacceptable images were more often those associated with frozen sections. Interestingly, the pathologist who deferred the fewest cases cited image quality as a problem most often. Discussion Telepathology is a technology that has been discussed at least since the Massachusetts General Hospital-Logan Airport (Boston) TeleMedicine project during the 1960s,18 and commercial systems have been available since the late 1980s. It has the potential to extend the expertise of individual pathologists into underserved areas or to provide subspecialty expertise to general pathologists. Imaging and telecommunications technology are reaching a level sufficient to enable routine use of telepathology, but general acceptance has not become a reality. The TelePath system was designed with several unique features. This system allows control of the full functionality of a distant microscope, using discrete commands (in contradistinction to continuous remote control) for light intensity, optical magnification, x-y stage position, and autofocus. A panoramic overview image allows the pathologist to remain oriented to the features on the entire slide. The remote pathologist can select any area of the panoramic view to obtain a higher magnification image, and all subsequent images share these control features. The images are of excellent resolution and have a field of view comparable to that of a 35-mm camera mounted on a microscope. Because only single images are transmitted, they can be sent quickly without compression artifacts that degrade the image. This initial trial shows excellent accuracy and precision. The only slide that presented consistent difficulty to the pathologists was a follicular variant of papillary carcinoma, an extremely difficult lesion acknowledged by an extramural international expert to have features of a hyperplastic nodule as well as carcinoma. Of some concern in this trial was the substantial number of deferred diagnoses. Increased rates of deferral have been seen in other telepathology trials. This may result from several factors. First, as shown in the list of cases deferred, the cases presented difficult diagnoses that may be deferred in the normal practice of frozen section pathology. Second, the relative inexperience of the pathologists with the electronic images may make them hesitant to commit to a definite diagnosis. Finally, it is possible that the telepathology medium in general or this system specifically compromises the ability of the pathologist to diagnose extremely difficult slides with confidence. The relative contributions of the latter factors will only become clear with further experience and trials. A second clinical trial with parallel frozen section diagnoses is underway to address some of these issues. The acceptance of electronic images in pathology has clearly lagged behind radiology and some other clinical specialties, most notably those that use electronic endoscopic images. The reasons for this remain somewhat unclear. Good to excellent diagnostic accuracy of telepathology has been demonstrated in several trials Superior accuracy has usually been associated with the dynamic mode of Am J Clin Pathol 1998; 110:

6 W i n o k u r e t a l / A PROTOTYPE TELEPATHOLOGY SYSTEM telepathology, probably because the remote pathologist chooses the fields of view. Because the dynamic systems use full-motion video at a frame rate of 30 frames per second, transmission over telecommunications lines requires substantial image compression, as well as a reduced field of view. Both of these factors result in an image that is significantly different from the microscopic image. This has been dealt with by making still digital images available to the remote pathologist in so-called hybrid dynamic-static systems. Some of these systems also include a separate image of the slide to maintain orientation. TelePath presents a new model of operation that eliminates the inherent limitations of transmitting video over telecommunications lines. All images presented are at least moderately high resolution, and the overview of the tissue is present on the desktop exactly as for the other images. The system, as it is currently configured, requires relatively high bandwidth telecommunications, but because all commands are discrete, microscope control is possible over any available telecommunications medium, including the Internet. The images (1.4 megabyte file), however, still require several seconds for transmission (the actual time depends on the transmission medium and ranges from seconds on ATM to approximately 8 seconds using a Tl connection to 1 minute using a single ISDN connection). *The authors indicated are stockholders oftelemedicine Solutions, lnc, a corporation formed by them for the purpose of commercializing the telepathology system described in this article. Acknowledgments: We wish to acknowledge the technical support of Ed Kujawski. References 1. Mun SK, Elsayed AM, Tohme WG, et al. Teleradiology/ telepathology requirements and implementation. ] Med Syst. 1995;19: Coen H. Previous experience in teleradiology could serve as a first lesson to telepathologists. Arch Anat Cytol Pathol. 1995;43: Goldberg MA. Teleradiology and telemedicine. Radiol Clin North Am. 1996;34: Weinstein RS, Bhattacharyya A, Yu Y P, et al. Pathology consultation services via the Arizona-International Telemedicine Network. Arch Anat Cytol Pathol. 1995;43: Shimosato Y, Yagi Y, Yamagishi K, et al. Experience and present status of telepathology in the National Cancer Center Hospital, Tokyo. Zentralbl Pathol. 199;138: Nordrum I, Engum B, Rinde E, et al. Remote frozen section service: a telepathology project in northern Norway. Hum Pathol. 1991;: Kayser K. Telepathology in Europe: its practical use. Arch There is no image compression used in the system. A Anat C}to! Pathol. 1995;43: conscious decision was made to avoid "lossy image compres8. Ito H, Adachi H, Taniyama K, et al. Telepathology is sion" to maintain image quality. Image compression is most available for transplantation-pathology: experience in Japan using an integrated, low-cost, and high-quality system. Mod likely to degrade images, such as pathology images, with Pathol. 1994;7: many objects or edges, the so called high-frequency images. 9. Adachi H, Inoue J, Nozu T, et al. Frozen-section services by A few studies have addressed this problem by asking patholotelepathology: experience of 100 cases in the San-in district, gists what level of compression is acceptable, but no Japan. Pathol Int. 1996;46: consensus has been reached for the acceptable degree of 10. Weinstein RS, Bloom KJ, Rozek LS. Telepathology and the compression.56 The project is also evaluating alternative networking of pathology diagnostic services. Arch Pathol Lab Med. 1987;111: ways to decrease the amount of information transmitted while 11. Weinstein RS, Bloom KJ, Krupinski EA, et al. Human allowing the pathologist to request the unaltered image. performance studies of the video microscopy component of a Because the system saves the pathologists images as dynamic telepathology system. Zentralbl Pathol. 199;138: the pathologist proceeds through a slide, the system also is ideally configured to serve as an educational tool. We hope 1. Weinstein RS, Bhattacharyya AK, Graham AR, et al. Telepathology: a ten-year progress report. Hum Pathol. this system represents an alternative model of work and 1997;8:1-7. control that is more acceptable to the pathologist and that 13. Eusebi E, Roschini L, Erde S, et al. Transcontinental increases the acceptance of telepathology. consults in surgical pathology via the Internet. Hum Pathol. 1997;8: From the Department of Pathology and The Center for Telecommunications Education and Research, the University of 14- Grimes G, McClellanS, Goldman J, et al. Applications of virtual reality technology in pathology. In: Morgan KS, Alabama at Birmingham. Hoffman HM, Stredney D, et al, eds. Medicine Meets Virtual Reality. Amsterdam, the Netherlands: IOS Press; This work was supported by a contract with the BellSouth 1997: Corporation and a grant from the National Institute of Standards ( ) administered by the South Carolina Research 15. McClellan S, Winokur T, Kujawski E. A prototype system Authority, Charleston, SC. for telepathology. Proceedings of the First Annual Telecommunications Conference. Austin, Tex: University of Manuscript received July 11, 1997; revision accepted Texas Press; 1996: October 14, Address reprint requests to Dr Winokur: Department of 16. McClellan S, Winokur T. TelePath: real-time remote Pathology, KB 534, University ofalabama at Birmingham, pathology. Southeastcon, Piscataway, NJ: IEEE Press; Birmingham, AL : Am J Clin Pathol 1998,110:43-49

7 Anatomic Pathology / ORIGINAL ARTICLE 17. Winokur T, McClellan S, Kujawski E, et al. A prototype telepathology system combining features of dynamic and static telepathology. Cell Vision. 1996;3: Murphy R, Bird K. A new community health resource (observations on the feasibility of telediagnosis based on 100 patient transactions). Am] Public Health. 1974;64: Becker RJ, Specht CS, Jones R, et al. Use of remote video microscopy (telepathology) as an adjunct to neurosurgical frozen section consultation. Hum Pathol. 1993;4: Kayser K, Fritz P, Drlicek M, et al. Expert consultation by use of telepathology: the Heidelberg experiences. Anal Cell Pathol. 1995;9: Kayser K, Drlicek M, Rahn W. Aids of telepathology in intra-operative histomorphological tumor diagnosis and classification. In Vivo. 1993;7: Nordrum I, Eide TJ. Remote frozen section service in Norway. Ardi Anat Cytol Pathol. 1995;43: Oberholzer M, Fischer HR, Christen H. Telepathology: frozen section diagnosis at a distance. Virchovos Arch. 1995;46: Weinstein RS, Bloom KJ, Rozek LS. Telepathology: longdistance diagnosis. Am J Clin Pathol. 1989;91(suppl):S Suzuki J, Furukawa I, Okumura A, et al. Perfomance analysis of compression techniques for pathological microscopic images. Proc SPIE. 1995;451: Foran DJ, Meer PP, Papathomas T, et al. Compression guidelines for diagnostic telepathology. IEEE Trans Information Technol Biomed. 1997;1: Am J Clin Pathol 1998; 110: