THEORY AND APPROACHES TO AUTOMATED IMAGE ANALYSIS IN DIGITAL PATHOLOGY

Transcription

1 THEORY AND APPROACHES TO AUTOMATED IMAGE ANALYSIS IN DIGITAL PATHOLOGY Kyle Takayama, MS Charles River Laboratories EVERY STEP OF THE WAY EVERY STEP OF THE WAY

2 MORPHOMETRY Measurements or counts performed on 2-dimensional tissue sections Clinical and research applications Medical diagnosis (biopsy analysis) Applied research (pharmacology, toxicology) Basic research (natural sciences) Performed manually or via automated image analysis Manual analyses are subjective and vulnerable to human error Automated analyses are objective and more efficient 2 EVERY STEP OF THE WAY

3 DIGITAL IMAGE ANALYSIS Advantages of automated image analysis: 1. Improved workflow speed and capacity 2. Reduce human error 3. Fit-for-purpose processing 4. Generate and format study data 3 EVERY STEP OF THE WAY

4 DIGITAL IMAGE ANALYSIS Advantages of automated image analysis: 1. Improved workflow speed and capacity 2. Reduce human error 3. Fit-for-purpose processing 4. Generate and format study data 4 EVERY STEP OF THE WAY

5 DIGITAL SCANNING Slide-scanning microscopes generate high-resolution images of whole tissue sections Fast, accurate, and complete tissue images Reduces sampling bias and user error 5 EVERY STEP OF THE WAY

6 DIGITAL IMAGE ANALYSIS AUTOMATED DECISION MAKING How many total nuclei are shown in this image? How many nuclei appear to be DAB-positive? Ki-67 (brown), hematoxylin counterstain 6 EVERY STEP OF THE WAY

7 DIGITAL IMAGE ANALYSIS Increase workflow speed and efficiency zoom out 7 EVERY STEP OF THE WAY

8 DIGITAL IMAGE ANALYSIS Increase workflow speed and efficiency zoom out, again! 8 EVERY STEP OF THE WAY

9 DIGITAL IMAGE ANALYSIS Increase workflow speed and efficiency now, repeat 9 EVERY STEP OF THE WAY

10 DIGITAL IMAGE ANALYSIS Increase workflow speed and efficiency and repeat!!! 10 EVERY STEP OF THE WAY

11 DIGITAL PATHOLOGY Increased workflow speed and capacity Availability of automated slide scanners creates the supply and the demand for digital image analysis The relative efficiency of automated image analysis increases as the size and scope of the study expands. Batch processing reduces tedious swapping of slides/images and increases the speed of analysis Apply systematic uniform random sampling techniques 11 EVERY STEP OF THE WAY

12 12 EVERY STEP OF THE WAY SURS APPLIED TO MICROSCOPIC FIELD SELECTION

13 DIGITAL IMAGE ANALYSIS Advantages of automated image analysis: 1. Improved workflow speed and capacity 2. Reduce human error 3. Fit-for-purpose processing 4. Generate and format study data 13 EVERY STEP OF THE WAY

14 DIGITAL IMAGE ANALYSIS Manual Pathology Scoring as the Gold-Standard 1 Pattern detection is a highly evolved feature of human vision Superior detection of complex features compared to simple computerbased algorithms Pathologists are trained to evaluate a range of species, tissues, lesions, and biomarkers Critical thinking and acquired experience improves efficiency and accuracy of assessment 14 EVERY STEP OF THE WAY 1 Aeffner, Famke, et al. Arch Pathol Lab Med Sep;141(9):

15 Bias in manual scoring DIGITAL IMAGE ANALYSIS Visual bias 1 Perception of size, brightness, and texture can be influenced by adjacent features (illusions) Limited sensitivity to changes in density/number Poor detection of minor changes in brightness 15 EVERY STEP OF THE WAY 1 Aeffner, Famke, et al. Arch Pathol Lab Med Sep;141(9):

16 Bias in manual scoring DIGITAL IMAGE ANALYSIS Ebbinghaus illusion Hermann grid illusion 16 EVERY STEP OF THE WAY

17 DIGITAL IMAGE ANALYSIS Bias in manual scoring Cognitive bias 1 Variability between users limits reproducibility of results Confirmation/context bias Diagnostic drift: subtle but consistent variation over time Gambler s Fallacy: failure to treat individual events independently 17 EVERY STEP OF THE WAY 1 Aeffner, Famke, et al. Arch Pathol Lab Med Sep;141(9):

18 A PICTURE IS WORTH A THOUSAND WORDS...? Perception (physiology) and interpretation (psychology) of an image is inherently subjective Rabbit-duck Yellow Mountain - Yuan Zuo 18 EVERY STEP OF THE WAY

19 DIGITAL IMAGE ANALYSIS Reduce human error Provides quantitative data to support qualitative evaluation Programmed configurations are objectively applied, parameters are consistent from sample to sample Greater reproducibility of results 19 EVERY STEP OF THE WAY

20 DIGITAL IMAGE ANALYSIS Advantages of automated image analysis: 1. Improved workflow speed and capacity 2. Reduce human error 3. Fit-for-purpose processing 4. Generate and format study data 20 EVERY STEP OF THE WAY

21 DIGITAL IMAGE ANALYSIS Broad range of quantitative end points Counting Labeling index (positive/negative) Vascular density Linear/Distance Dermal thickness Epithelial height / crypt depth Area fraction Percent staining (positive/negative) Tissue composition Signal intensity Immunofluorescence In situ hybridization 21 EVERY STEP OF THE WAY

22 LABELING INDEX Original image Processed image 22 EVERY STEP OF THE WAY

23 23 EVERY STEP OF THE WAY LINEAR MEASUREMENTS

24 AREA FRACTION Original image Processed image 24 EVERY STEP OF THE WAY

25 TISSUE COMPOSITION Original image Processed image 25 EVERY STEP OF THE WAY

26 SIGNAL INTENSITY In Vitro GFP Expression 26 EVERY STEP OF THE WAY

27 DIGITAL IMAGE ANALYSIS Fit-for-purpose processing Algorithms can be customized to suit a variety of: Variables (counts, distance, area) Species/tissues Stains (common, specialty, IHC) Microscopy (high/low resolution, brightfield vs. fluorescent) Sampling techniques (comprehensive, systematic uniform random sampling) 27 EVERY STEP OF THE WAY

28 DIGITAL IMAGE ANALYSIS Advantages of automated image analysis: 1. Improve workflow speed and capacity 2. Reduce human error 3. Fit-for-purpose processing 4. Generate and format study data 28 EVERY STEP OF THE WAY

29 Generate and format study data DIGITAL PATHOLOGY Reduces manual data entry, formatting, and calculations Accuracy can be validated for GLP studies 29 EVERY STEP OF THE WAY

30 PRACTICAL APPROACH IN IMAGE ANALYSIS... What percent of tissue is composed of β cells? What percent of β-cells are positive for Ki67? Pancreas: Insulin (red), Ki-67 (brown), hematoxylin counterstain 30 EVERY STEP OF THE WAY

31 WHAT IS A PIXEL? Coding and storage of the visual environment Merriam-Webster: (noun) any of the small discrete elements that together constitute an image digital representation of reality 31 EVERY STEP OF THE WAY

32 WHAT IS A PIXEL? Coding, storage, and display of the visual environment Grayscale 8-bit grayscale 2 8 = 2x2x2x2x2x2x2x2 = 256 possible values 24-bit RBG (2 8 ) 3 = (256) 3 = 16,777,216 possible values RGB 32 EVERY STEP OF THE WAY

33 FILTERS Converting RGB images to grayscale Original Red filter Blue filter Green filter 33 EVERY STEP OF THE WAY

34 FILTERS Converting RGB images to grayscale Original Red filter Blue filter Green filter 34 EVERY STEP OF THE WAY

35 DIGITAL IMAGE ANALYSIS Three stages of automated image analysis: 1. Preprocessing exploring RGB values to define individual pixels as positive or negative/counter staining 2. Classification detection and labeling of stains in digital image files 3. Post-processing inclusion/exclusion criteria for individual labels based on size/shape/proximity/etc 35 EVERY STEP OF THE WAY

36 PRACTICAL APPROACH IN IMAGE ANALYSIS... What percent of tissue is composed of β cells? What percent of β-cells are positive for Ki67? Pancreas: Insulin (red), Ki-67 (brown), hematoxylin counterstain 36 EVERY STEP OF THE WAY

37 PRACTICAL APPROACH IN IMAGE ANALYSIS... Pre-processing Original image Filter (red chromaticity) 37 EVERY STEP OF THE WAY

38 PRACTICAL APPROACH IN IMAGE ANALYSIS... Classification Filter (red chromaticity) Pixel Classification 38 EVERY STEP OF THE WAY

39 PRACTICAL APPROACH IN IMAGE ANALYSIS... Classification Filter with Classification Overlay Original with Classification Overlay 39 EVERY STEP OF THE WAY

40 PRACTICAL APPROACH IN IMAGE ANALYSIS... Post-processing Original with Classification Overlay Fill Function 40 EVERY STEP OF THE WAY

41 PRACTICAL APPROACH IN IMAGE ANALYSIS... Post-processing Original with Processed Overlay Exclude by Area 41 EVERY STEP OF THE WAY

42 PRACTICAL APPROACH IN IMAGE ANALYSIS... Post-processing Original with Processed Overlay Dilate and Erode 42 EVERY STEP OF THE WAY

43 PRACTICAL APPROACH IN IMAGE ANALYSIS... Calculate Original with Processed Overlay Calculate Area of Overlay 43 EVERY STEP OF THE WAY

44 PRACTICAL APPROACH IN IMAGE ANALYSIS... Further investigation 44 EVERY STEP OF THE WAY Original with β cell ROI Hematoxylin vs. DAB Detection

45 SELECTING A METHODOLOGY 1 45 EVERY STEP OF THE WAY 1 Aeffner, Famke, et al. Arch Pathol Lab Med Sep;141(9):

46 Questions? CONTACT US Kyle Takayama, MS Associate Research Scientist Pathology Associates Charles River Laboratories 4025 Stirrup Creek Drive, Suite 150 Durham, NC Website: Phone: