Digital Imaging Outline

Transcription

1 Digital Microscopy and Imaging Update Michael Feldman, MD, PhD Associate Professor of Pathology Assistant Dean for Information Technology University of Pennsylvania School of Medicine Digital Imaging Outline Digital Imaging What is it? Why use it? How do you use it (What are you trying to achieve)? Integrate into clinical practice Education Archive Research Hardware Camera s (Gross and microscopic) Server s for storage Computer to manage, manipulate, display and utilize data Whole slide (virtual) digital slide vs static Software Storage LIS Integration Manipulation How to make it look better Web Education Why are we here? What do we hope to learn today? Who is using digital imaging? How and where is it being used in your practice? Why are you using digital imaging technology? Any burning questions out there about this technology that we can discuss today?

2 Digital Imaging (What are we talking about?) What is digital imaging? Use of digital imaging technologies in place of analog systems to capture our daily work for archive, publication, education and research Implicit in this definition is a need for pathologists to understand (or hire folks who do) hardware and software as well as provide users with technology that enables this technology Define Use case for imaging: Why use imaging, what are the goals, what are the needs, what is the cost Why switch from the Good Old Days 35mm slide workhorse Everyone knows how but not everyone does it well Folks already have large collections Cost is ~$4-5K for 35mm camera set up for a microscope Cost for gross images <$1K for good 35 mm camera Pathology is a Visual Discipline Areas with Primary Image Data: Gross Pathology Autopsy and Surgical Pathology Microscopic Imaging Autopsy, Surgical Pathology, Cytopathology, Hemepath, Microbiology Electron Microscopy Immunofluorescence Microscopy Molecular Diagnosis gels, microarray, SNP

3 Current Paradigm Sinard et al, Human Pathology 2002 Pathology Workstation Sinard et al, Human Pathology 2002 So, why switch Possible Use Cases Cost ROI for digital imaging is very positive Gross room ROI 3-4 months Microscope imaging 1-2 years Convenience Immediately available and easily modifiable Reusable content Shareable content among faculty and housestaff Usability Education Image retrieval, Online education, Content based Image Retrieval Publication require electronic formats Research Quantitive Higher throughput Collaboration and consultation Online

4 Digital Imaging Outline Digital Imaging What is it? Why use it? How do you use it (What are you trying to achieve)? Integrate into clinical practice Education Archive Research Hardware Camera s (Gross and microscopic) Server s for storage Computer to manage, manipulate, display and utilize data Whole slide (virtual) digital slide vs static Software Storage LIS Integration Manipulation How to make it look better Web Education What are you trying to accomplish? Do not underestimate this statement! Your hardware and software needs will vary depending on what you want to accomplish as will the cost of the project How are you planning on using the digital images? Integrate into report (LIS integration vs custom solution) Education Med school (UME), residents (GME), attending (CME)? Publications which formats Consultation real time vs static image transfer Research archive vs image process vs CAD vs Histocytometry Clinical support case conferences, tumor boards, etc What areas are amenable to Digital Images? Gross Room Autopsy and Surgical Pathology Excellent place to start Large volume and good case mix PA s can assist with technology Easier to enforce standards within a small group Education - Images are great for clinical conferences, publications, education of medical students and house staff Cost can be low consumer level digital camera at least 3 megapixel or better will do very nicely in these areas 3 Megapixel will do a great job on images printed at up to 5x7. If you think you will need 8x10 then go for a 5 megapixel camera

5 Size Really Does Matter Pixels basic unit of digital camera sensor Image is derived from red, green and blue information at each pixel Most cameras produce 8 bits (256 shades) of information at each color (red, green and blue) to create a total of 24 million color variations 8 bits = 1 byte so Since each pixel contains 8 bits of data for red, green and blue, a single pixel with 24 bits of color information takes up 3 bytes of data Size Megabyte (million), Gigabyte (billion) and Terabyte (Trillion) A single image from a digital camera will occupy 3 bytes of data, 2 megapixel camera = 6 Megabyte image 3 mgeapixel = 9 Megabytes image 5 megapxiel = 15 Megabyte image 12 megapixel = 36 Megabyte image These camera s can produce large files! File formats The image produced by camera is stored as a file, of which there are many different file types. Loss less formats: RAW pure binary data from the camera sensor. Each manufacturer has it s own raw format (images are big) TIFF standard established years ago. May be used without compression for lossless image (images are big) Lossy Formats JPEG (Joint Photographic Experts Group ) Again, an older standard which uses a mathematical algorithm to reduce the image size (throws out some data). Variable amount of compression can be used from very little to a whole lot. Personal experience suggests 10-20:1 compression produces so little change that it is imperceptible (Photoshop setting of quality = 75 in a jpeg file produces ~15:1 compression JPEG2000 newer version of jpeg uses wavelet compression. More compression with less degradation of image quality. Not widely adopted yet but could be interesting in future. Also does lossless compression Autopsy Pathology

6 Built In Macro for Very Close Work Pseudo Melanosis of Esophagus Small lymphoid aggregate with anthracotic pigment Lymphoid aggregates are < 1 mm in size Tiff vs JPEG (2X) with Leica D480 TIFF JPEG 20:1 Tiff vs JPEG (10X) with Leica D480 TIFF JPEG 20:1

7 Tiff vs JPEG (40X) with Leica D480 TIFF JPEG 20:1 Gross Imaging 3.3 Megapixel camera (This is minimum requirement) Generates uncompressed 2048x1536 pixel images 9 Mb uncompressed Images displayed as well as the printouts are jpeg compressed to Kb (15-20:1 compression ratio). Compression is done in the camera Key features: PA s are a constant in the laboratory and can help educate other users Yearly conference to teach how to use the camera Monthly gross image conference for residents to use the images for teaching gross pathology (helps folks realize benefit from their efforts) Integrated into daily work for resident s, PA s Images transferred and stored in central image database server archive and indexing Resulting images are used in web based education materials for housestaff: Web based Interesting cases Web based Board review Web based support for Clinical conferences (Breast, Liver, ENT pathology) Web based image query for search and retrieval Microscopic Imaging More challenging than gross photography Technical Issues: Static vs wide field full digital slides Camera s Which type (consumer camera, dedicated microscope camera, single chip, progreesive scan) How many pixels is enough? S/N ratio and dynamic range of cameras End User Issues: Ease of use how many seconds does it take to make a good picture? This is more important than the technical stuff Good way to test this is to get system in house and play with it for a few weeks. Alternately, go visit someone with the setup you are interested in buying Know the end user match the system to the person Always extended test before purchasing!

8 Image and Acquisition Microscope with glass slide: Start with good scope and slide - garbage in, garbage out... Your eye can resolve very small differences. No camera, digital or otherwise is currently better. Color film captures information at approximately 6000 dpi Film 6000 dpi High end digital camera 3000 dpi Intermediate digital camera 1500 dpi Low resolution digital camera 600 dpi How many pixels do you need? Depends on what you are doing with the image? Resolution and Imaging Slides Dirk Soenksen CEO Aperio technology Camera s s and Technology Standard Photograph - gold standard Digital Camera Single chip Triple chip Progressive scanning Whole slide scanning (Digital or Virtual Slide)

9 Digital Camera - Single chip CCD CCD - Single chip Digital information Each pixel measures a color intensity. Filter for RGB Microscope Lens A filter is used to split the signal into Red, Green and Blue (RGB) signals at each pixel (Bayer = GRGB). The information is then combined in software to create final image which is not true color SPOT, Leica, Olympus 3 Images taken with filter switching RGB at each pixel. Final image is true color. SPOT 3 shot Histology Slide Digital Camera - Triple chip CCD CCD - Three chip Digital information Each pixel measures a color intensity. Prism to split light A prism and filters are used to split the signal into Red, Green and Blue (RGB) signals for each of the three CCD arrays. The information is the combined to give a true color at each pixel in the CCD array Microscope Lens Final image is true color at each pixel In general used for high end analog video Histology Slide Digital Camera - Progressive scanning Progressive scanning Digital information CCD Array is moved across field to capture information between pixels that a stationary CCD would miss Each pixel measures a color intensity. A filter is used to split the signal into Red, Green and Blue (RGB) signals at each pixel. The information is the combined to give a true color at each pixel in the CCD array. Microscope Lens megapixel cameras Slow acquisition, vibration, small stepping motors Histology Slide Leica, Zeiss, Nikon

10 Virtual slides (Stitching) Microbrightfield BLISS Bacus Labs Digital Slide (Virtual) - A 1 cm 2 piece of tissue imaged at 40X (0.25 mm FOV) - Creates a 40x40 grid = 1600 Images - Each Image is 1 Mb - Total Image Tiled together is 1.6 Gb uncompressed - With compression Mb - We view 250,000 Pieces of glass/year - If we scanned 33% of our glass (80,000 pieces of tissue) at 200 Mb/slide = 16 Terabytes/year! (That is 23, 73 GB HD s) Whole slide scanner (ScanScope( from Aperio Inc.)

11 Digital Slides (Wide field whole slide) DMetrix s s array-microscope technology Problem with current microscopy paradigm: High detail means small field of view; and Small field of view means long time for imaging Solution: Parallel imaging with the equivalent of 80 or more microscopes in one instrument Result: High throughput True if: only one image per slide, and True if: multi-plane, multi-color, multi-modality image set per slide Virtual slide scanning Many vendors in space today Dmetrix microsocope array Aperio linear scanner Olympus/Baccus - tiles Hammatsu - tiles Zeiss/HistoRx - tiles Bioimagene - tiles Clarient/Trestle - tiles

12 Digital Imaging Outline Digital Imaging What is it? Why use it? How do you use it (What are you trying to achieve)? Integrate into clinical practice Education Archive Research Hardware Camera s (Gross and microscopic) Server s for storage Computer to manage, manipulate, display and utilize data Whole slide (virtual) digital slide vs static Software Storage LIS Integration Manipulation How to make it look better Web Education Now that you have the image, what next Image Storage: Department server with 200 GB RAID 5 array and tape backup Database Client Server (ThumbsPlus from running against Microsoft SQL 7 or 2000 Autopsy images are 0.3 MB (300 KB) each. We capture images/year. That will require GB space/year Surg Path gross images are small 0.3 MB (300 KB). We capture images/month GB space/year 17 Faculty and resident s with digital microscope cameras 15-20,000 images/year at 0.3 MB = 5-6 GB/space/year Combined, we will use < 10 GB of space each year. Carefully think through and consider your needs! Image Database ThumbsPlus from Cerious Software Scalable Single user ($89.95/seat) runs Access database Multiuser Scalable to enterprise level using Microsoft SQL database Added cost of central file server and database server and software (5-7K) Integrates with cameras and scanner s TWAIN Customizable user fields, annotation, keywords Controlled vocabulary Gallery view work s like digital light box Catalog offline disks (CD, DVD )

13 Screen Shot ThumbsPlus Preview/Task Explorer View Thumbnail Area Image Properties Web Search Page Simple client for web enabled image search Easier to understand than thumbs client search function Great to be able to distribute light weight thin search client To Do: Shopping cart metaphor being adopted to help folks collect multiple images as they shop for data Keep pushing folks to carefully label and annotate images

14 Web based Search page Board Review Answer s Medical School Case Authoring

15 Med School Case Authoring Client side viewing What software is used to pull images from server Java Applet from web page (Aurora, Microbright, Bacus) cross platform ActiveX (Zoomify, Bacus, Aperio) integrate into web or custom application (windows) Stand alone application (Aperio) windows based HTML (Aperio) using zoomify format Quicktime (Zoomify) Flash (Zoomify) Advantages of Applet Whether Flash, Java, Quicktime Cross platform viewing Mac and PC Easier to support across enterprise for software updates Portable to devices including (PDA s) Lightweight (<100 Kb)

16 Annotation Serving up standardized microscope quality high resolution images is start Uniformity No bad slides, cracked, not representative No slide handling and management Slides always available Even better if we can author content on slides and with the slides into educational lessons Annotation Cont d Ability to annotate image can be used in multiple manners Simply label slide simple answer Prior to lesson - Author material with questions and no answer s After lesson provide same images with answers and pointers Allow students to label images with questions independent of faculty annotations Questions Interactive like a threaded discussion that embeds digital slides Which fit s your pedagogy? How do you teach with glass vs. digital? Glass: Search and identify with you assisting and facilitating in a small group Digital: Same Digital: Compare and contrast multiple slides at once Digital: Before and after scenario Digital: support s asynchronous learning, threaded discussion Student to student Student to faculty Digital: expert learning system with intelligent tutor (Rebecca Crowley, University Pittsburgh) Digital: distributed multiheaded microscope Digital: share content

17 Usage Data from Penn Pilot Usage stats from one course section (4 labs): 1.6 million hits (each time move slide registers as hit Only 12 manually tiled slides used Avg 22 minutes per visit More 50% viewed outside campus 150 students viewed slides from > 200 computers Pushed > 2 Gb data during these 4 small group sessions Feedback from Pilot Uniformity of image Always there to study Supported group study Image quality excellent Student s liked it, more comfortable than search and find on microscope Wanted more of it New Value in Going Digital Image Processing CAD (computer assisted diagnosis) Automated and quantitative IHC single color Spectral imaging multicolor immunostaining Histocytometry or flow cytometry for your slide CBIR content based image retrieval Distance consultation/collaboration

18 Color and Spectra Light has no color. Color is an observer interpretation. These two yellows appear identical to the human eye, yet they have very different spectral components. MSI vs. Traditional Color Imaging Spectroscopy captures the entire spectrum (light intensity as a function of wavelength). Intensity Spectral But with RGB-based instrumentation, this complex spectrum will be described using only 3 values (bins) averaged over large spectral regions. Intensity I blue = 112 I green = 203 I red = 253 RGB 3. Image processing to resolve individual stains based on spectra teach computer the spectral profile of chromogen/fluorophore A, B, C extract spectra of individual stains A B C

19 6. Computational assignment of immunostains to each nucleus breast tumor stained for p-erk (DAB) & hematoxylin segmentation of nuclei (based on hematoxylin) p-erk+ tumor cells (yellow) p-erk- stromal cells (grey) Data display & Analysis: Frequency histogram of intensity of p-erk p staining of stromal and tumor cell nuclei in a breast tumor tumor cell nuclei stromal cell nuclei

20 A B C D Fraction of nucleii A B C D Ki67 E F G H 150 perk Ki I J K L perk Ki67 M N O P perk Ki67 50 perk Ki67 0 perk Don t hesitate to ask if you are uncertain