2067_C002.fm Page 7 Wednesday, May 11, :02 AM. Section I Technological Advances

Transcription

1 2067_C002.fm Page 7 Wednesday, May 11, :02 AM Section I Technological Advances

2 2067_C002.fm Page 8 Wednesday, May 11, :02 AM

3 2067_C002.fm Page 9 Wednesday, May 11, :02 AM 2 Reinvention of Light Microscopy: Array Microscopy and Ultrarapidly Scanned Virtual Slides for Diagnostic Pathology and Medical Education Ronald S. Weinstein, Michael R. Descour, Chen Liang, Lynne Richter, William C. Russum, James F. Goodall, Pixuan Zhou, Artur G. Olszak, and Peter H. Bartels Any sufficiently advanced technology is indistinguishable from magic. Clarke s Second Law [1] 2.1 INTRODUCTION TO NEXT-GENERATION HEALTHCARE Healthcare, an information-based industry, is in a state of transition. The current focus of attention on health information is taking place in an environment in which better access to effective healthcare has been identified by governmental leaders as a societal goal. In several countries, including the U.S., governments are funding programs to develop comprehensive patient electronic health records. This reflects recognition of the critical importance of information management in almost every aspect of the healthcare enterprise, ranging from individual patient care to disease prevention and public health. The ultimate goal of creating large information systems to permanently archive cradle-to-grave electronic patient health records is regarded as technically feasible as well as achievable in the foreseeable future. Such patient electronic health records would contain plenary data sets, including the digital images of all imaging studies ever performed on the patient, securely archived but readily accessible, on-line, to patients and their designated service providers. The electronic patient health record concept can be taken even further by expanding its scope to include patient healthcare-related education information. This might be accomplished by appending a personal electronic health education portfolio to each electronic health record. Tools would be developed to assist patients

4 2067_C002.fm Page 10 Wednesday, May 11, :02 AM Virtual Microscopy and Virtual Slides in Teaching, Diagnosis, and Research in navigating between the information in their linked electronic health records and their personal health education portfolios. Using mass customization techniques, patient information could be continuously updated on an individualized basis. Another futuristic concept is the development of on-line patient self-evaluations. Patient proficiency testing could be used to evaluate the patient s current capacity for self-help to manage his own healthcare in light of his health status, his environment, and other relevant information. The creation of such comprehensive electronic health records would eventually have implications for all facets of the healthcare industry, including the practice of pathology. For pathology to fully participate in the digital revolution in healthcare, pathology imaging will have to be in digital formats. It is anticipated that virtual slides, as described in this chapter, will be a key component of laboratory reports in electronic health records and will be universally retrievable by patients healthcare providers. 2.2 VIRTUAL SLIDES, PICTURE ARCHIVING SYSTEMS, AND ELECTRONIC HEALTH RECORDS The novel array microscope described in this chapter is a key component and, indeed, the enabling technology for the first of a new generation of virtual slide processors (Table 2.1; Figure 2.1). The ultrarapid virtual slide processor is defined as a processor that can process a virtual slide in under one minute [2]. These processors have evolved over the past 10 years and add to the list of applications of telepathology (Table 2.2). The ultrarapid virtual slide processor described in this chapter, the first such processor that has been commercialized, has been custom designed to serve as a pathology digital image input device for what is referred to as a pathology picture archiving and communication system (PACS). A pathology PACS is a laboratory information system that stores the results of laboratory tests, including images of patients laboratory specimens such as digital images of whole histopathology glass slides, referred to as virtual slides. Virtual slides can be viewed on a computer using a special viewer, called a microscopy emulator. Pathologists can make diagnoses at a distance by viewing virtual slides over telecommunications linkages using a computer, a process referred to as telepathology. A goal of healthcare planners is to have pathology PACS telepathology systems, as well as the information systems of all other medical specialties, linked by telecommunications to networks of information systems that archive comprehensive patient electronic health records. Thus, virtual slides would become an integral component of electronic health records as part of laboratory reports. The new generation of virtual slide processors will be critically important because they will enable pathology laboratories to go fully digital without interfering with a laboratory s workflow or throughput, for the first time. The medical specialty that generates the largest number of digital images today is radiology. Remarkable progress has been made in taking radiology departments filmless and fully digital over the past decade. In radiology, going fully digital means that all imaging processes are in digital formats, from the point of image acquisition to image storage. Radiology PACS are regarded as standard equipment at larger institutions. The benefits of having a radiology PACS are numerous and include making radiology studies immediately available on hospital wards and in 10

5 2067_C002.fm Page 11 Wednesday, May 11, :02 AM Reinvention of Light Microscopy TABLE 2.1 Classification of Telepathology Systems Dates/Generation Class Symbol Category Enabling Technologies Comments st Generation Systems 1A DNR Dynamic nonrobotic Videomicroscopy Resolution issues 1B DR Dynamic robotic Robotic microscopy Resolution issues 2A SFNR Store-and-forward nonrobotic 2B SFR Store-and-forward robotic Image grabbing based Limited sampling nd Generation Systems 2C SFSR Store-and-forward stitch robotic 3A HDSF-NR Hybrid dynamic store-and-forward nonrobotic 3B HDSF-R Hybrid dynamic/ store-and-forward robotic 4A VSA Virtual slide/automatic/ nonrobotic processor 4B VSI Virtual slide/interactive (robotic) processor Electronic stitch software Slow image acquisition > 10 minute processing time rd Generation Systems 5A HVS Hybrid virtual slide processor 5B RVS Rapid virtual slide processor Combined automatic and interactive Continuous stage motion Strobe illumination 1 10 minute processing time 2001 to present 4th Generation System 5C UVS Ultrarapid virtual slide processor Array microscopy <1 minute processing time Source: Adapted from Weinstein, R.S., et al., Hum. Pathol., 32, 1283,

6 2067_C002.fm Page 12 Wednesday, May 11, :02 AM Virtual Microscopy and Virtual Slides in Teaching, Diagnosis, and Research Slides Per Day Ultrarapid Processor Rapid 300 Processor Legacy Processors Year Figure 2.1 Improvements and innovations in virtual slide processors. In the past decade, there have been steady improvements in the throughput of virtual slide scanners. Legacy systems, which incorporate a single-optical axis light microscope and tiles for imaging, have slowly improved in their performance. The introduction of new illumination and scanning systems further improved performance. The ultrarapid virtual slide scanner, which incorporates an array microscope, can process 1000 slides per day. Although the S-curve shows a leveling off, the array microscope may enable future progress in increasing processing rates. (Adapted from Weinstein, R.S. et al., Hum. Pathol., 32, 1283, 2001.) TABLE 2.2 Virtual Slide Digitizers Applications Application Class 4A a (> 10 min.) Type of Virtual Slide Processor (Slide Processing Time) Class 4B (>10 min.) Class 5A (>10 min.) Class 5B (1 10 min.) Class 5C (<1 min.) Distance learning X X X X X Proficiency testing X X X X X Low throughout telepathology X X X X X Routine surgical pathology X X X X (low volume) FNA specimen adequacy b X X X X Intraoperative frozen sections b X X X X Routine surgical pathology X X (high volume) Interactive proficiency testing X X Targeted therapy diagnostics X X Pathology PACS c X a Class of telepathology system, see Table 2.1. b FNA specimen adequacy determinations and intraoperative frozen sections may be carried out with Class 4B and 5A systems, but procedure times may be prolonged. c PACS picture archiving and communications system. Source: Adapted from Weinstein, R.S., et al., Hum. Pathol., 32, 1283,

7 2067_C002.fm Page 13 Wednesday, May 11, :02 AM Reinvention of Light Microscopy decentralized doctors offices. It permits the simultaneous access of physicians to the results of studies carried out by various imaging modalities (i.e., CAT scans, MRIs, PET scans, etc.). The radiology PACS is a critical component of the infrastructure for many digital teleradiology services. As an example of a rapidly growing regional teleradiology service, in Arizona and its neighboring states 22 hospitals currently receive teleradiology services, 24/7, from radiologists and residents at the University Medical Center of the University of Arizona College of Medicine, in Tucson, AZ. Some patients receiving these teleradiology services are over 400 miles away. In 2003 alone, 70,000 digital radiology cases were handled by this universitybased group practice [3]. The development of teleradiology services in Arizona is not unique. In fact, teleradiology is a growth industry worldwide. Many communities across the U.S. routinely utilize night-time teleradiology coverage by radiologists located in countries as far away as Australia and India, and that trend is growing. 2.3 THE FULLY DIGITAL PATHOLOGY DEPARTMENT Going fully digital has a somewhat different connotation for the field of pathology. Whereas a radiology image can be recorded directly on an electronic sensor, a pathology specimen mounted on a glass slide is a physical object. It is unlikely that pathology laboratories would be able to go fully digital in exactly the same way as radiology departments, although some work is being done on the direct imaging of paraffin-embedded tissue blocks. It is reasonable to expect that glass slides will remain part of pathology practices into the foreseeable future, although pathologists might stop looking through light microscopes if and when virtual microscopy becomes routine. Once pathology glass slides of histopathology sections, cytopathology preparations, blood smears, microbiology stains, etc., are routinely archived in pathology PACS telepathology systems, the range of telepathology services can be expanded. For example, at institutions equipped with pathology PACS telepathology systems, patients could potentially gain access to immediate second opinions on their cases on-line from top-notch experts. This type of service is currently available at a few institutions on a rather limited basis [4,5]. In addition, pathology reports of patients with specific diseases could be re-reviewed as new concepts of the disease are validated and new therapies are introduced. With the widespread introduction of new data mining services, old pathology reports would become living documents, available for reassessment and forming the basis for new actions that could benefit patients later in life. It has been estimated that 200,000,000 paraffin blocks of surgical pathology cases, and their corresponding glass slides, are currently in storage at laboratories in the U.S. [6]. Useful information is almost certainly sequestered away in these massive collections of human tissue, which is one reason why tissue blocks are routinely warehoused. 2.4 ULTRARAPID VIRTUAL SLIDE PROCESSORS Background Historically, one of the earliest versions of whole slide digital images was incorporated into the dynamic robotic telepathology 13

8 2067_C002.fm Page 14 Wednesday, May 11, :02 AM Virtual Microscopy and Virtual Slides in Teaching, Diagnosis, and Research system developed by Corabi International Telemetrics, Inc., in 1986 (Corabi International Telemetrics, Inc., Fairfax, VA) [7]. Micromapper and Microtracker software, components of the Corabi DX-1000 system, were used to digitize whole glass slides at low resolution and track slide positioning on the microscope stage. In order to do telepathology with the Corabi system, a glass slide was digitized on a customdesigned light box, using a digital camera, prior to mounting the glass slide on the stage of the motorized microscope. During operation of the Corabi system, the precise location of the tissue section on the glass slide, in relation to the objective lens of the robotically controlled microscope, was displayed as a graphic representation on a system control monitor. This tissue map was continuously updated as the system operator remotely controlled the microscope stage s X- and Y-coordinates [7 10]. This allowed the pathologist operating the telepathology system to track all stage movements. The procedure ensured that remote imaging of glass histopathology slides was inclusive [11]. The first virtual slides that could be navigated and viewed at multiple magnifications were introduced in the 1990s [12 15]. The virtual slide processors that produced these virtual slides required several hours to process a single virtual slide at reasonably high resolution [16]. In recent years, processing rates have improved but are still inadequate for many applications. Prior to the development of the array microscope in the past three years, virtual slide processors were typically designed around optics adopted from the compound single-optical-axis light microscope, a device originally developed around 1600 for direct visual use [17]. In the four centuries since that time, microscopes used for biomedical applications have evolved to meet the requirements of a human operator. Microscope objectives are designed with combinations of magnification and resolving capability that match the average acuity of the human eye. Microscope eye-piece designs have evolved for the comfort of the human operator. The microscope s eye-pieces project a virtual image to a relaxed-eye position, feature long eye-relief distance, and are located at eye-level for the microscope operator. The microscope stage is conveniently positioned so that glass slides can be moved around on the microscope stage either by using a mechanical slide positioning device or, as many pathologists prefer, hand manipulation of the slide using one or two fingers. Focus knobs are positioned just above the bench top so they can be conveniently controlled by hand and finger movements, in some instances with the wrist resting comfortably on the bench top. The size of the instrument is based, to a considerable extent, on human dimensions while accommodating the requirement for an optical pathway of some minimal length. Of course, none of the ergonomically correct features are relevant to the design of the most efficient virtual slide processor. In fact, incorporating single-axis light-microscope optics into a virtual slide processor means incorporating all the implicit design features evolved for visual rather than digital use of a microscope. The performance of a resultant virtual slide processor is therefore unnecessarily constrained. On the other hand, the array microscope described in this chapter represents a significant breakthrough in light-microscope design. The array microscope was not designed for direct human visual use but was specifically designed as an efficient digital image input device for an ultrarapid virtual slide processor. The array microscope s specifications were specifically 14

9 2067_C002.fm Page 15 Wednesday, May 11, :02 AM Reinvention of Light Microscopy TABLE 2.3 Components of the DMetrix Ultrarapid Virtual Slide Processor Component Description Objective Lens DMetrix high-resolution NA = objective array/dmetrix engineered Camera (Sensor) DMetrix DM5760 CMOS image sensor and custom camera/dmetrix engineered Scanning Stage Ultra-precision platform with extendable carrier stage/dmetrix engineered Illumination System High-brightness LED Kohler system/dmetrix engineered Data Processing Real-time virtual-slide compression/dmetrix engineered Data Storage PC-based image server with 1 terabyte of expandable storage space Network Interface Card 100/1000 Mbit/sec Ethernet Acquisition/Application Software digitalretina software Virtual Slide Viewing Software digitaleyepiece Instrument Dimensions 117 cm in height, 46 cm in width, 58 cm in depth tailored for that application. Almost all of the components of the instrument were custom engineered (Table 2.3). This statement is particularly true in the case of the novel microscope optics, but it also applies to the image sensor and associated electronics, ultraprecise mechanisms, and the automated slide loader. Unlike previous and competing virtual slide processors, few of the key components of the instrument are commercial off-the-shelf Brief Overview of Scanning Microscopes A very early example of a scanning microscope resides at the Armed Forces Institute of Pathology (AFIP) Billings Microscope Collection. This microscope is an Ernst Leitz (Wetzlar, Germany) microscope device which was conceived by Dr. E. Nebelthau of Marburg, Germany, in 1896 for examination of large tissue sections or culture plates [17]. This scanning microscope permits traversal of a large section of a specimen by movement of the microscope along one direction, using a manually advanced lead screw and movement of the stage by a rack-and-pinion arrangement in an orthogonal direction. It is tempting to speculate that were Dr. Nebelthau presented with a modern-day digital scanning microscope, he would have no trouble identifying the optical parts such as the objective but little else besides. Were he presented with an array microscope, we speculate that Clarke s Second Law would apply without exception [1]. Starting in the 1950s, high-speed electronic light-detection and beamdeflection devices such as photomultiplier and cathode-ray tubes found application in microscopy of cells and tissues [18 22]. The most clinically significant device was the Deeley scanner, which allowed the accurate recording of total DNA contents of nuclei from cervical cells. In 1955, W.E. Tolles described the pioneering Cytoanalyzer [22]. The Cytoanalyzer was an early instance of an automated microscope system that incorporated nowfamiliar features such as automatic slide 15

10 2067_C002.fm Page 16 Wednesday, May 11, :02 AM Virtual Microscopy and Virtual Slides in Teaching, Diagnosis, and Research feed and focus, as well as circuitry to generate an early form of quantitation of cervical smears [23]. In 1957, the RCA television pioneer, Vladimir Zworykin, published in Science a description of an ultraviolet television color-translating microscope [24]. In this system, an orthicon television camera was used to capture false color images of a specimen s multispectral ultraviolet absorption. Preston, in 1961, perfected the CELL- SCAN system which enabled the machine recognition of leukocytes [25]. This development and the work by Mendelsohn, Mayall, and Prewitt gave impetus to the commercial development of white blood cell differential count devices [26]. During the 1960s, we have for the first time the combination of a microscope and a digital computer. This combination enables digital image transformation, automated object detection, segmentation, feature extraction, and cell classification. The 1960s also saw the introduction of computer-controlled high-speed, precision scanning stages capable of submicron accuracy. At the close of the decade, the acquisition of high-resolution digital images of cells and tissues was possible, recorded online to a dedicated laboratory computer. At the end of the 1970s, image data acquisition became dominated by video technology. A typical example of this type of system is the combination of a microcomputer and a simple video system as described by Jarvis [27,28]. However, a next generation image-sensing device was already on the horizon: CCD image sensors, today ubiquitous, were first developed in the early 1970s. In the 1980s, Bartels, Shack, and colleagues at the University of Arizona developed the ultrafast laser scanning microscope [29]. This instrument incorporated a number of advanced features such as an air-bearing rotating polygon mirror that scanned the laser beam across the biological specimen. The microscope objective had a very large field of view of 2 mm in diameter and a numerical aperture of NA = 0.8. This system demonstrated that images with very high signal-to-noise ratio could be obtained with a dwell time as short as 20 nanoseconds per pixel. The development of imageunderstanding systems also dates to the late 1980s and early 1990s [30]. This development signals the increasing significance of the software component of a scanning microscope. The development of the personal computer (PC) is also traced to this decade. The period from then to now is marked by tremendous advances in PC technology and functionality and falling costs of random-access memory and magnetic data storage. For instance, processor clock speeds have increased one hundred-fold since the early 1990s. Concurrently, largeformat solid-state image sensors such as CCDs and CMOS devices have become commonplace. While a scanning microscope has undergone dramatic changes related to detection of light or computing, its basic optical system has remained largely unaltered. For instance, scintillation tubes gave way to television cameras, which have been replaced by CCD and most recently CMOS solidstate image sensors. Racks full of computer equipment, which dwarfed the microscope proper, have been replaced by desktop PCs. Matrices of transmission values of a specimen printed on paper have given way to multi-terabyte data-storage systems. However, the microscope optics with their principal limitation, i.e., the tradeoff of field of view versus resolution, remain to this day a standard component of a digital (scanning) microscope. DMetrix s breakthrough array-microscope solution eliminates this tradeoff as described in the next section. 16

11 2067_C002.fm Page 17 Wednesday, May 11, :02 AM Reinvention of Light Microscopy The Array Microscope The array microscope concept was developed by DMetrix and the University of Arizona [32 34]. An early description of the concept appeared in a mini-symposium on telepathology in the journal Human Pathology in 2001 [2]. The array microscope is an entirely new microscope configuration, unanticipated in the 400 years since the first appearance of a compound microscope [17] (Figure 2.2). As mentioned already, the foremost advantage of the array-microscope concept is the decoupling of the trade-off between large field of view and high resolution. In the case of the array microscope, the field of view can be extended by the addition of more miniature microscopes while the resolution remains constant and high. A further benefit of the array microscope is the massively parallel form of image acquisition. For instance, in the DMetrix DX-40 instrument, 80 microscopes simultaneously image the biological specimen, permitting much faster virtual-slide acquisition than is possible with a virtual slide processor based on conventional microscope optics. Implementation of the array microscope required advances in real-time computing, optics manufacturing, and image sensor technology. Image data acquired by the 80 microscopes are compressed in dedicated integrated circuitry (IC) while the glass slide is being scanned, instead of later, after the scan is finished. The optical designs that DMetrix has discovered for use in the array microscope require unprecedented precision in manufacturing and testing. For example, DMetrix s optics-testing methods are used elsewhere only for oneof-a-kind scientific instruments such as the two 8.4-meter diameter mirror large binocular telescopes (LBT) being constructed on Mt. Graham in Arizona [36]. 9 mm Figure 2.2 Upper: Futuristic concept rendering of an array microscope-based digital image acquisition system mounted on a glass slide. The rows of tiny microscopes are staggered so that the entire slide can be digitized with a single sweep of the array across the surface of the slide. Each miniature microscope views the successive images in a 250 micrometer track on the long axis of the glass slide. The array is shown without its image capture device. The large arrow indicates the direction of the movement of the array during the scanning process. Lower: Exploded view of a single miniaturized microscope imaging unit. Actual lens shapes are not illustrated nor are their relative positions in the miniaturized microscope shown to scale. (Adapted from Weinstein, R.S. et al., Hum. Pathol., 32, 1283, 2001.) 17

12 2067_C002.fm Page 18 Wednesday, May 11, :02 AM Virtual Microscopy and Virtual Slides in Teaching, Diagnosis, and Research Finally, relative to the age of the microscope, CCD image sensors are a very recent newcomer, having been developed in the early 1970s [37]. Image sensor designs and technology suitable for use in array microscopy did not become available until the last decade [31]. The fact that the concept of an array microscope originated in part at the University of Arizona Optical Sciences Center, one of the top ranked optical sciences research centers in the world, is not surprising. The Arizona Optical Sciences Center is a leader in space-faring instrument design and fabrication, photolithography, nanoscale optical data storage, and telescopemirror design and fabrication. 2.5 THE DMETRIX ULTRARAPID VIRTUAL SLIDE SYSTEM The following description provides an overview of components of the DMetrix DX-40 system ( the first commercial realization of arraymicroscope technology Overview The array microscope s imaging engine consists of an ensemble of 80 miniature microscope objectives and an image sensor embedded in a custom digital camera. In the DX-40 instrument, each miniature microscope objective consists of three A B C Microscope Slide D E Figure 2.3 DMetrix array microscope components. (A) Single lenslet array. (B) Photomicrograph of a DMetrix lenslet array by transillumination. This 10 8 array consists of staggered rows of lenses. (C) Three lenslet arrays constitute a lenslet array ensemble. The arrow indicates the optical pathway of what constitutes a single objective lens. (D) Photomicrograph of a surface of the bottom-tier lenslet array of the lenslet array ensemble by oblique illumination. The lenses are precisely positioned and uniform in shape. Individual lens convexities, at this surface, are 1.5 mm in diameter. (E) Schematic illustration of the position of the lenslet array at the beginning of the scanning process. For purposes of illustration, the lenses are in a different configuration than the lenses in an actual lenslet array, as shown in Figure 2.3B (Figure 2.3B shows the lenses collapsed into a more compact configuration). The lenslet array moves in the direction of the long axis of the glass slide. (From Weinstein, R.S. et al., Hum. Pathol., 35:1303, With permission.) 18

13 2067_C002.fm Page 19 Wednesday, May 11, :02 AM Reinvention of Light Microscopy high-precision, aspheric lenslet elements. Each of the three lenslet elements is replicated 80 times on a separate plate. The plates are precisely assembled, one on top of another, to form a lenslet array ensemble (Figure 2.3). A custom-engineered sensor/ camera combination is attached to the top of the lenslet array ensemble (Figure 2.4). The sensor captures images from each of the objectives as the lenslet array ensemble glides above the glass slide. Each tiny microscope and the overlying sensor/ camera digitize a unique field of view that is complementary to the areas imaged by the remaining microscopes in the array (Figure 2.3E). The 80 objectives collectively produce a seamless, two-dimensional image of the specimen, i.e., a virtual slide (Figure 2.5) DMetrix Microscope Optics DMetrix s microscope optics are designed in-house and meet stringent image-quality criteria dictated ultimately by the laws of optics. DMetrix relies extensively on the most advanced microscopeobjective design concepts, including the use of aspheric lenses mentioned previously. Historically, lenses and mirrors have been fabricated with spherical surfaces, i.e., surfaces that are part of a sphere. The reason for this preference is associated with ease of fabrication and testing of spherical Sensor/Camera Lenslet Array Ensemble Glass Slide 2.54 cm (1 inch) (a) Figure 2.4 (See color insert following page 242.) DMetrix ultrarapid virtual slide scanner and array microscope. (a) DMetrix ultrarapid rapid virtual slide scanner cabinet. The slide scanning apparatus is positioned at the level of a standard laboratory bench top. The lower cabinet houses on-board computers and environment control equipment. (b) Concept rendering of the DMetrix array microscope. This consists of a three-tier lenslet array ensemble and a sensor/camera. The array is mounted above a glass slide, shown schematically. The optical pathway is approximately 1.0 cm in length. The illumination system is not shown. The optics and camera combined have approximately the same dimensions as a stack of 6 U.S. quarter coins (not shown). (Modified from Weinstein, R.S. et al., Hum. Pathol., 25:1303, 2004.) (b) 19

14 2067_C002.fm Page 20 Wednesday, May 11, :02 AM Virtual Microscopy and Virtual Slides in Teaching, Diagnosis, and Research Figure 2.5 Ultra rapidly scanned virtual slide of a human breast fibroadenoma. The figure illustrates the seamless integration of images from multiple miniaturized microscopes into a large composite image. Hematoxylin and eosin staining (H&E staining). 3 magnification. (Modified from Weinstein, R.S. et al., Hum. Pathol., 35:1303, 2004.) surfaces. However, the natural shape of lens elements in a microscope objective, for instance, wants to be aspheric. A conventional microscope-objective design is the result of a best approximation, using spherical lenses, of the aspheric-lens design. Aspheric lens shapes are uncommon in optical systems such as microscope objectives due to the difficulty inherent in their manufacturing and their testing. DMetrix has developed its own proprietary techniques for fabrication of such lens shapes and now uses aspheric lenses extensively in the design of tiny microscope objectives. For example, a three-aspheric-element microscope objective that is simple in appearance features a numerical aperture (NA) of 0.65 and diffraction-limited image quality (i.e., the best that it can be) across a field of view that is a significant fraction of the objective s diameter (Figure 2.6). A complete array of such objectives, as included in one DX-40 scanner, contains 480 optical surfaces (Figure 2.7). For comparison, consider that a conventional microscope objective of similar numerical aperture would contain about 10 optical surfaces. The individual lenses that make up an array microscope can be fabricated from glass, crystalline materials (e.g., fluorite), or plastics. The choice of material is made depending on the role that each lens plays within the entire microscope objective Sensor/Camera Ultrarapid slide processing requires a digital camera that can swiftly move image data out of the sensor in preparation for the next exposure. A further requirement 20

15 2067_C002.fm Page 21 Wednesday, May 11, :02 AM Reinvention of Light Microscopy Sensor/Camera Lens Lenslet Carrier #3 1 cm Baffle Lens Lenslet Carrier #2 Lens Baffle Lenslet Carrier #1 Coverslip Figure 2.6 Layout of a single miniaturized microscope objective. (From Liang, C., Miniature microscope objective lens, US Patent Application, US 2004/ With permission.) on any camera suitable for array microscopy is that it captures the images formed by all 80 objectives in the array microscope. DMetrix engineers searched exhaustively for such a combination. In the end, the decision was made to develop a custom image sensor and a specialized digital camera. Image capture in the DX-40 instrument is accomplished by means of a scientificgrade CMOS image sensor (Model DM5760). CMOS sensors are not new to microscopy and have been used in that application since the mid-1990s [39,40]. Before being used in a DMetrix camera, a candidate DM5760 image sensor must undergo a battery of stringent functional and performance tests. For example, the response to light of every single pixel is tested individually and only those sensors with 100% acceptable pixels continue toward integration in an array microscope. The DM5760 sensor is designed with 10 separate image data outputs. Thanks to such a high level of parallel data flow, the DM5760 image sensor can sustain a frame rate of 3000 frames per second. This frame rate is 100 times faster than typical video or what is frequently referred to as realtime imaging. The DM5760 has two additional performance-enhancing features. First, it can be windowed, meaning that any region smaller than the full format of the sensor may be accessed for read-out. This feature is particularly useful with small-area specimens. In that case, windowing results in 21

16 2067_C002.fm Page 22 Wednesday, May 11, :02 AM Virtual Microscopy and Virtual Slides in Teaching, Diagnosis, and Research Figure 2.7 Scanning electron micrograph of the upper surface of lenslet carrier #1, showing staggered rows consisting of 10 precision lenses each. At this surface, each lens is approximately 1.5 mm in diameter (see Figure 2.6). Original photograph, 23 magnification. (Courtesy of Dr. Stuart K. Williams.) a significant reduction of scan time, by a factor of two or more. Second, the DM5760 features a noise suppression technology known as correlated double sampling. This is a unique feature found usually only on advanced, scientific-grade CCD image sensors. Finally, the DMetrix image sensor is unique due to its postage-stamp size format and the world s smallest pixels. Based on pixel density, the DMetrix image sensor is equivalent to a 24 megapixel device. When combined with the array-microscope optics of the DX-40 model, each pixel projects to a 0.47-micron size at the microscope slide Illumination Illumination in a microscope requires a separate optical system that typically consists of a light source, a collector lens, and a condenser lens. The purpose of the illumination system is to provide uniform illumination across the field of view of a microscope objective while at the same time filling with light the aperture stop of 22

17 2067_C002.fm Page 23 Wednesday, May 11, :02 AM Reinvention of Light Microscopy the microscope objective. The most common illumination system in a microscope was introduced by Köhler in 1893 [41]. Incidentally, Köhler illumination appears in many everyday, non-microscopy optical systems such as overhead projectors. The same illumination requirements apply in the case of the array microscope. The field of view of each of the 80 miniature microscope objectives must be uniformly illuminated and the numerical aperture of each miniature microscope objective needs to be filled. DMetrix attacked this problem by extensive computer simulation, using the same types of software tools as are employed to design space telescopes and photonic circuits. The eventual solution is an innovative giant-lens illumination system that achieves the properties envisioned by Köhler simultaneously at all 80 microscope objectives [41]. Finally, the DMetrix DX-40 instrument uses a high-brightness light source that is further enhanced by means of proprietary reflective optical elements. These reflective elements direct more efficiently the available light toward the microscope slide being scanned, ensuring a high signal level at the sensor Whole Slide Scanning A virtual slide can be viewed over the Internet immediately after scanning with the DX-40 instrument is completed (Figure 2.8). Virtual slide processing begins when a histopathology slide is inserted into the instrument, either manually or using an automated slide loader. During scanning, the array microscope optics are positioned to span the width of a glass slide (Figure 2.3E). The glass slide is scanned simultaneously by all 80 objectives of the array microscope as the glass slide is borne by an ultra-precision stage. During scanning, each miniaturized microscope captures an image complementary to the other 79 microscopes in the array. Glass slides travel at rates of up to 3 mm/sec during scanning. Even faster scan rates are planned for future models of array microscopes. Such faster rates will support, among other applications, three-dimensional (3-D) imaging of biological specimens on glass N. America Europe Internet Asia S. America Africa Australia Figure 2.8 The DMetrix DX-40 scanner is linked to the Internet. Virtual slides are accessible simultaneously throughout the world. 23

18 2067_C002.fm Page 24 Wednesday, May 11, :02 AM Virtual Microscopy and Virtual Slides in Teaching, Diagnosis, and Research slides while maintaining a high throughput measured in slides/hour. The DX-40 system automatically determines optimal focus for each of the miniature objectives in the array. With its array of closely packed microscopes distributed over a 1 cm 2 cm area of the specimen, the DX-40 instrument is not confused by open areas in or around a specimen since at least several of the 80 microscopes are typically overlying biological materials on the glass slide. The result of best-focus determination is a calculated trajectory that the objectives follow during scanning. This trajectory consists of continuous minute rolling, pitching, and vertical movements of the imaging engine, which are executed as a function of position along the length of the glass slide. The process is precisely orchestrated and monitored by the instrument s software. The trajectory is calculated automatically without human intervention. In operation, the ultrarapid virtual slide processor scans each slide in full color, producing a 24-bit color image. At the conclusion of the scan, i.e., at lights-out, the completed virtual slide is stored on the on-board image server and is ready for immediate viewing. Full-color scanning, image processing, i.e., on-board compression, and image storage time for a 2.25 cm 2 tissue section, is 58 seconds. Using an automated slide loader, a throughput of up to 40 virtual slides per hour can be reached without an operator s intervention. Throughput does, of course, depend on the average area and shape of the region of interest on each glass slide Image Processing The DX-40 system performs a minimum of image processing, thus reducing the time from glass slide loading into the scanner to viewing the captured virtual slide. For example, with array microscope technology, there is no need to repeatedly estimate field of view overlap or to merge image fields with cross-correlation techniques. The only form of image processing that occurs in the DX-40 system is image compression which is executed in realtime, concurrently with image acquisition by the imaging engine. The parallel imaging characteristic of the array microscope naturally gives rise to parallel electronic image acquisition and parallel processing of acquired image data (e.g., real-time image compression). By simultaneously executing imaging and processing tasks, high-speed image capture and therefore high throughput of virtual slides are achieved Data Storage and Retrieval One image captured with the array microscope requires between 200 MB to 1 GB of storage space, after compression. Large numbers of such images may be stored on a RAID that is part of the DX- 40 instrument. Alternately, files can be placed in on-demand storage linked to the DX-40 system via a network connection. Images may be accessed using a webbrowser viewer or a native Windows application installed on an end-user s computer. The capability to deliver image data and metadata to multiple clients is built into the system (Figure 2.8) DMetrix Virtual Slide Viewer The virtual slide images captured by the DX-40 system are viewable using DMetrix s digitaleyepiece software. This viewing software has a familiar look and feel because it shares many features with digital-photography software applications. DMetrix s software allows a viewer to access 24

19 2067_C002.fm Page 25 Wednesday, May 11, :02 AM Reinvention of Light Microscopy multiple image servers using a username and password combination for each server. The digitaleyepiece software provides controls for previewing a slide in a thumbnail as well as navigating the slide either by selecting a region of interest on the thumbnail or by click-and-drag controls as part of the full display (Figure 2.9). Zoom in and zoom out controls allow for viewing at a broad range of magnifications, a digital analog to rotating a microscope objective turret. The digitaleyepiece software also incorporates a set of digital-photography tools such as color balance or brightness and contrast adjustments. The digitaleyepiece software also offers features that have no equivalent on a conventional microscope. For example, end users can outline regions of a virtual slide and associate with them time and userstamped text entries. Later, the annotated regions can be quickly revisited using the Annotation Manager tool in digitaleyepiece. Features on the specimen can be measured in terms of linear distance, ratios of distances, and area. Additionally, these measurements may be recorded for further statistical analysis in a spreadsheet program. 2.6 VALIDATION OF VIDEO MICROSCOPY AND VIRTUAL MICROSCOPY Video microscopy was the forerunner of virtual microscopy and, conceptually the largest break with the tradition of, Figure 2.9 Control panel of the DMetrix ultra-rapid virtual slide scanner. Three separate breast pathology cases are seen in the slide manager windows in the upper right corner. The bottom slide, from a breast core biopsy, has been selected for viewing in detail on the main screen. (Modified from Weinstein, R.S. et al., Hum. Pathol., 25:1303, 2004.) 25

20 2067_C002.fm Page 26 Wednesday, May 11, :02 AM Virtual Microscopy and Virtual Slides in Teaching, Diagnosis, and Research pathologists using conventional light microscopy for diagnostic pathology. Video microscopy was an enabling technology for the first telepathology systems as well as for today s virtual microscopy systems. Results of the first validation study of video microscopy are relevant since they became the benchmark for subsequent diagnostic accuracy studies. The first formal study of the diagnostic accuracy of surgical pathologists using video microscopy for surgical pathology was carried out in the Department of Pathology at Rush Presbyterian St. Luke s Medical Center in Chicago, in 1986 (Table 2.4). For the study, six surgical pathologists examined 115 breast frozen section cases by both video microscopy and light microscopy under highly controlled conditions. For the video microscopy sessions, a Sony Model SHR-10 video camera (1050 scan lines) was mounted on a microscope without eye pieces, and the images were displayed on a Sony Trinitron 950-line monitor. In total, 1380 diagnostic decisions were rendered. By light microscopy, the pathologists made 97.7% true-positive decisions and 1.7% false-positive decisions. By video microscopy, they made 95.1% true-positive decisions and 2.7% false-positive decisions. The differences were not statistically significant. The kappa value for light microscopy equals.80 (SD =.07) whereas the kappa value for video microscopy equals.77 (SD =.09). The Z-test for differences equals.75, which was not significant. The results of this initial study have been discussed in several publications [8,11,42 44]. The conclusion was that video microscopy can be substituted for conventional light microscopy in the practice of surgical pathology. Since then, numerous studies on the diagnostic accuracy of video microscopy and telepathology have been published and TABLE 2.4 Video Microscopy Study (1986). Frequency of Pooled Pathologists Diagnoses vs. Baseline Truth Diagnoses a Pooled Data Baseline Truth Diagnosis Light Microscopy b Positive Negative Equivocal Total Positive 381 d Negative Equivocal Total Video Microscopy c Positive Negative Equivocal Total Positive Negative Equivocal Total a For pathologist s diagnosis: positive = 4 or 5 confidence rating; negative = 1 or 2 confidence rating; and equivocal = 3 confidence rating (nonstringent criteria) b Kappa value for light microscopy =.80 (SD =.07) c Kappa value for video microscopy =.77 (SD =.09) d Z-test for differences =.75 (not significant) Source: Reproduced from Krupinski, E.A., et al., Adv. Pathol. Lab. Med., 6, 63,

21 2067_C002.fm Page 27 Wednesday, May 11, :02 AM Reinvention of Light Microscopy many have shown comparable results [45-65]. The diagnostic accuracy of pathologists using the DMetrix technology and a virtual slide processor using conventional optics (the Aperio Technologies ScanScope ) was evaluated in two separate studies: the 2004 Weinstein study and the Barker study carried out in 2003, respectively [34,66]. The same four pathologists rendered diagnoses on breast surgical pathology cases in the two studies. Using the DMetrix system, there was a strong correlation between array microscopy vs. truth diagnoses based on surgical pathology reports. The kappa statistic for the array microscopy vs. truth was 0.95, which is highly significant (z = 10.33, p < 0.001) and there was no statistically significant difference between rates of agreement with truth between array microscopy and light microscopy (z = 0.134, p > 0.05). Pathologists rated over 95 percent of array microscopy virtual slide images as good or excellent. The mean viewing time for a DMetrix virtual slide was 1.16 minutes, which is essentially the same viewing times for experienced telepathologists using video microscopy or dynamic robotic telepathology [2,43,55]. The results of the Weinstein study and the Barker study indicate that the diagnostic accuracy for the evaluated DMetrix system and the virtual slide processor using conventional optics was statistically the same (Table 2.5). 2.7 APPLICATIONS OF VIRTUAL SLIDES IN TELEPATHOLOGY The DMetrix virtual slide images are of high quality and are suitable for diagnostic pathology, second opinions, expert opinions, clinical trials, education, and research (Figure 2.10 and Figure 2.11). When incorporated into telepathology practices and electronic health records on a larger scale, virtual microscopy will become part of the mainstream of the field of diagnostic pathology. The DMetrix scanner is designed to handle the daily glass slide output of many pathology departments. With a 1000 slide per day throughput for histopathology slides, the ultrarapid virtual slide scanner can serve as the digital image input device for pathology picture archiving and communications systems (pathology PACS). A pathology PACS has the potential to expedite the work flow of surgical pathology cases and cytopathology cases in hospitals and reference laboratories. In addition to the telepathology applications described in Section 2.3 and above, it is anticipated that new types of TABLE 2.5 Comparison of the Performance of Virtual Slide Processors Weinstein study (DMetrix DX- 40) Barker study (Aperio Technologies) p Value Diagnostic Accuracy 97.5% 96.3% p>0.05 Image Quality Rating a ± ± p = Scanning Plus Image Processing Time 1 to 3 minutes 36 minutes p<0.05 a Based on a scale of 1 (poor) to 4 (excellent) Source: Adapted from Weinstein, R.S. et al., Hum. Pathol., 35:1303, 2004, and Barker, G., Ph.D. dissertation, Kennedy-Western University, Thousand Oaks, CA , 2004.) 27

22 2067_C002.fm Page 28 Wednesday, May 11, :02 AM Virtual Microscopy and Virtual Slides in Teaching, Diagnosis, and Research A B C D Figure 2.10 Ultrarapid processed virtual slides for pathology teaching. (A) Rheumatoid arthritis, H&E staining. Reduced from 3 magnification. (B) Tuberculosis with a cluster of Langhans giant cells. H&E staining. Reduced from 150 magnification. (C) Renal biopsy from a patient with minimal change disease. The arrow points to a capillary loop with a thin basement membrane. Jones silver stain. Reduced from 150 magnification. (D) Human placenta with positive immunostaining for cytokeratin. Reduced from150 magnification. (Modified from Weinstein, R.S. et al., Hum. Pathol., 35:1303, 2004.) healthcare services might be introduced as pathology PACS telepathology systems become available. For example, faculty at the University of Arizona College of Medicine and the Arizona Cancer Center in Tucson, are planning a novel rapid throughput breast clinic (Figure 2.12a). Planning was initiated after the faculty established a new standard of care for breast digital mammography on an Indian reservation, using teleradiology. In 2001, the Arizona Telemedicine Program, located at the Arizona Health Sciences Center, in Tucson, participated in implementing a digital telemammography service on an Indian reservation, 300 miles to the north of the College of Medicine. Receiving mammography imaging studies from a distant city over a DS3 telecommunications link, Arizona College of Medicine radiologists achieved 45-minute turnaround times for reading out digital mammograms, including the time needed to fax a hard copy of reports back to the referring physicians [3]. Over 3000 digital mammography cases have been diagnosed with these rapid turnaround times. The benefits of this rapid turnaround service included an increase in patient compliance with respect to follow-up visits. The plan for the rapid throughput breast clinic at the University of Arizona is to use a combination of digital telemammography, on-site laboratory automation coupled with telepathology for immediate readout of cases by telepathologists as stained slides come off the automated immunostainers, and the use of teleoncology for the initial encounter of the patient with an oncologist by video conferencing as soon as the diagnostic components of the case work up are completed (Figure 2.12b). An ultrarapid virtual slide scanner will be used to create virtual slides of rapidly processed specimens. Virtual slides will be stored on an Internet-accessible 28

23 2067_C002.fm Page 29 Wednesday, May 11, :02 AM Reinvention of Light Microscopy Figure 2.11 Human colonic polyp, H&E stained. Reduced from 200 magnification. server. This will permit off-site telepathologists to read out slides as soon as they are available on the Internet and thus expedite the carrying out of the next step in the pathology diagnostic workup. The plan calls for reducing the time it takes a woman with a breast lesion to obtain a tissue diagnosis and to see an oncologist for initial therapeutic planning from the current 21 to 30 days down to one day. The goals of the program are to reduce patient anxiety, increase quality of care, and decrease costs, among others [Weinstein, Barker and Lopez, unpublished data]. As another example of the use of a pathology PACS telepathology system, the efficiency of cancer clinical trials could be improved by pre-qualifying patients, by telepathology, for admission into specific clinical trials through the use of up-front validation of local pathology reports by study referee pathologists before the initiation of therapy [34]. This would be a paradigm shift since validation of diagnoses 29