Technical Aspects in Digital Pathology

Technical Aspects in Digital Pathology Yukako Yagi, PhD yyagi@mgh.harvard.edu Director of the MGH Pathology Imaging & Communication Technology Center Assistant Professor of Pathology, Harvard Medical School Affiliate Faculty, Wellman Center for Photomedicine, MGH

Today s topics: Contents How we use image analysis in Pathology :examples Issues in current WSI for image analysis Importance of Standardized Quality in Digital Pathology Summary of slide, image quality and color quality evaluation Data accuracy

Automation Plan of Histology Laboratory Image Analysis

Imaging Failure Pre Capture Slide Breakage Robot Breakage Slide/Specimen ID Error Cover slip Error Staining Error Tissue Processing Error Bubbles Tissue Tears Tissue Folds Tissue Thickness Error Processing Speed Image Capture Tissue Finding Errors Focusing Errors Focus Plane Errors Optical Image Formation Inappropriate Sampling Period Dynamic Range / Contrast Capture Speed Post Capture Slide Breakage Scanner Breakage Compression Image ID Error Storage Failure Display Errors Network Failure Data Integration Failure Pathologist Interpretation

Standardized Quality in Digital Pathology is important

Challenges in Digital Pathology Management Required Quality is different per situation or equipment. Consistent not necessary the best quality through out the network. It includes Image data accuracy, Image quality, color consistency, color accuracy, speed and etc.

How to manage Standardized Quality? Evaluate each aspect

Evaluate Aspects Data Quality/Accuracy Slide Quality Evaluation Image Quality Evaluation Stain Standardization Color Standardization Human Interface Network (system structure)

Data accuracy

Data accuracy To producing accurate data (size, color, figure, measurement) is must in WSI application. Otherwise, analyzed data and developed decision support system cannot be useful. It is very difficult to notice for the users. It might be difficult to notice for the vendors too(?) It is difficult to know which one or who is causing the error scanner? viewer? software (image analysis software or browser)? user? Viewer Accuracy/reliability

Measurement & Appearance A part of same WSIs viewed by different viewers provided by a scanner vendor. Both 40x (clicked 40x) view. Both squares show 400um2. Size is also a little bit different.

Scanner A Scanner B

A slide was scanned by different 2 scanners. Resolutions are is 0.23um/pixel and 0.25um/pixel at 40x. 40x images was viewed on same monitor and a viewer provided each vendor. part of same WSIs viewed by different viewers provided by each scanner vendor. Possible causes are: User changed magnification accidentally by rolling the mouse wheel Scanner s resolution viewer

Evaluate Aspects Data Quality/Accuracy Slide Quality Evaluation Image Quality Evaluation Stain Standardization Color Standardization Human Interface Network (system structure)

Tissue artifacts 50% or more of the tissue slides we deal contains fold Whole slide image Tissue fold Blurry areas Tissue fold The presence of tissue artifacts such as folds or bubbles may produce blur or unfocused areas in the scanned images especially around the artifact areas. Could we still have a good image quality even in the presence of tissue artifacts such as folds????

Image quality is affected by tissue artifacts in most cases

Slide quality evaluation Slide quality control

image @ 40x in original Results After the adjustment We can see more details in image scanned by manual mode

Tissue artifacts Detection for Application K-means clustering plus morphological filtering Manual segmentation αs-βv method (New) Result of the proposed method is closer to the result of manual segmentation

Fold segmentation by the proposed method αs-βv #1 #2 #3 fold fold #4 #5 #6 fold The folds were segmented appropriately Some of the nuclei areas are labeled as folds, especially those which are very dark and lumped together fold fold #7 #8 #9 fold fold fold fold Mislabeling of nuclei as folds explains the lower specificity of the proposed method compared to clustering

Tissue Sectioning: Manual vs Automation

If we can create Thin, uniform thickness across entire specimen.. Could improve image quality of WSI Could reduce # of points for auto focus Could reduce scanning time

Automated Sectioning System

Manual Automation

Thickness of Specimen & Staining Thicker sections are stained more by the automated staining machine 3um 3um 3um 6um 3um 7um 5um 8um 7um 6um 5um 4um

Tissue Sectioning: Manual vs Automation The automated sectioning machine could section tissues with better consistency and quality for most of types of organs. However, there are still some issues for clinical usage such as speed and workflow integration. We have been using automated sectioning machine for all research materials such as for 3D Imaging. * Film/tape type of automated cover slipper

Thicker ->more noise

9um 2um

Sakura Smart Section Dainippon Seiki AS410

Image Quality Scanner 1 Scanner 2 Scanner 3 Scanner 4 Scanner 5

Commercial Slides Available test target slides are not for WSI or too expensive Target: too easy Thickness of slide: does not fit to some of scanners

Image Quality Evaluation Algorithm Image Quality Multiple regression analysis Definitive evaluation index, is calculated by q s n q,, are derived from training data.

Original Image Image Quality Evaluation Algorithm Result of Evaluation Image Quality problem were detected in dark regions. Darker->lower quality

Real Time Image Quality Assessment for WSI Yukako Yagi, PhD & Noriaki Hashimoto, PhD After we experimented evaluating following 10 parameters, we have decided 4 parameters were most efficient (1), (3), (7), (8) (1) Average width of edge [1] (2) Average ratio of height to width of edge [2] (3) Independence metric from surrounding pixels [1] (4) Low frequency component in DFT image (5) HH component in DWT image (6) HL and LH component in DWT image (7) Blockiness factor [3] (8) Difference within block [3] (9) Zero-crossing rate [3] (10) Ratio of blockiness factor to subtraction within block

Intensity Canny edge detection Suppress noises and detect only edges Gaussian filtering for noise reduction Sharpness evaluation Edge detection by differential Sharpness evaluation Width of detected edge is calculated The average value in the all edges is defined as an evaluation index for sharpness s If s is large, the image is blurred Assumption: Noise appears independently 250 200 150 39 100 Pixel

Application of the method to WSI Algorithm is processed for every 512x512 pixel block Visualized as a pseudo-color image

Comparison of images Images from excellent and poor regions Image quality = 4.66 Image quality = 1.96 41

Application to decide threshold Worked with pathologists to decide, we have decided what score to reject and accept

WSI Quality Evaluation Algorithm

Application to decide threshold: already used with pathologists Worked with pathologists to decide, we have decided what score to reject and accept

Color

The causes of color variation in WSI Thickness of Specimen Staining Scanner or Scanning process Viewer Software Display Display is important for annotation and diagnosis

Color Standardization in WSI: From Staining to Display Multispectral Imaging application Staining Scanning Viewer software Display

Staining color standardization

Scanning Color Standardization in WSI: Scanner Review Display The Imaging web site has the colors of the Calibration slide. Compare the displayed colors of the calibration slide to their actual colors to understand the difference VS The Imaging web site has Calibration slide.

Polynomial transformation Color of the patches as produced by a particular scanner Reference color of the color patches Color transformation matrix will be stored for used in color standardization Each scanner will have its own Color transformation matrix MEDICAL SCHOOL

WSI scanners and Color Imaging Whole slide scanner 1 (WSI 1) Whole slide scanner 2 (WSI 2) The same tissue slide was scanned with different scanners. Use the mouse embryo slide to confirm the effectiveness of the color transformation matrix H&E stained images

Liver Scanner A Scanner B Without color correction

blue channel Scanner A Scanner B Liver 240 220 200 180 160 140 120 100 Nu (Orig) A 50 80 110140170200230 Red channel Without color correction

Liver Scanner A Scanner B Result of color correction

Blue channel Scanner A Scanner B Liver 240 220 200 180 160 140 120 100 Nu (Corrected) A B 50 80 110 140 170 200 230 Red channel Result of color correction

Slide Quality can be evaluated to improve WSI quality Slide Quality can be improved Image Quality can be evaluated if it appropriate for diagnosis, Image analysis, or/and Education. Color of WSIs can be standardized.

Quality Control of WSI system Image Quality & color Color Optics Using the two calibration slides, the evaluation of image quality and color standardization could be done. Monitor

Tissue Processing Accurate Results require Good Images, Good images require good slide, good slides require good block Paraffin block Summary Scanning Digital Pathology can improve the patient care globally. Digital Pathology has unlimited possibility. We just to need to think what we want to do.. Sectioning Staining Cover glass Digital Stains Stained slides

Thank you!