Digital Pathology and Tissue-based Diagnosis. How do they differ?

Transcription

1 Digital Pathology and Tissue-based Diagnosis. How do they differ? P. Hufnagl Institute of Pathology (Rudolf-Virchow-Haus). Humboldt University, Berlin?

2 Structure of the talk Possible workflow routine diagnostic diagnostic support Which scanner? Quantification / image analysis Conclusions

3 Structure of the talk Possible workflow routine diagnostic diagnostic support Which scanner can we trust? Quantification / image analysis Conclusions

4 The Conventional Workflow Tissue Lab: Cutting, Staining, Coverslip Clinical request HIS PLIMS Report Physician: Dictation, Images, Physician: Assignment, diagnosing, Annotation consultation

5 The Digital Workflow Tissue Lab: Cutting, Staining, Coverslip Slide Scanner: Registration, Digitalisation PACS: Storage, Image streaming Clinical request HIS PLIMS Report Physician: Dictation, Images, Physician: Assignment, diagnosing, Image analysis: marker quantification Annotation consultation

6 The Digital Workflow Missing link between cover slipper and scanner Tissue Lab: Cutting, Staining, Coverslip Missing Slide Scanner: link between Registration, scanner Digitalisation and PLIS PACS: Storage, Image streaming Clinical request HIS PLIMS Report Physician: Dictation, Images, Physician: Assignment, diagnosing, Image analysis: marker quantification Annotation consultation

7 Advantages of Virtual Microscopy User Viewer Slide Server No glass transportation, no glass archive (?) Previous slides are always available Microscopic diagnostic - anytime anywhere - Parallel viewing of different stainings, positions Viewing and handling parallel at different locations Facilitated second opinion - online Quantification and image analysis just in time Annotations are simple to be handled much more.. Slide Scanner

8 Problems of the digital workflow Disadvantages Additional process step: Digitalisation Huge and continuous hardware and software investments Training of personnel Diagnosis on the monitor is unfamiliar for pathologists Legal Problems

9 Strategy Start with a less critical application Biobanking Introduce VM to applications with most effects Tumour board Second opinion in-house Solve all problems along the workflow LIMS integration Sure barcode identification Establishing of continuous testing

10 ZEBANC CHARITÉ BIOBANK CVK CCM CBF

11 CURRENTLY OFFERED SERVICES Aliquotation Quality documentation (SPREC) WSI generation TMA generation and WSI* based analysis Automatic DNA- extraction DNA Sequencing... *whole slide image 11

12 VALUE OF A SAMPLE Sample Clinical Data Whole Slide Images Quantification of Morphology V A L U E Next Generation Sequencing Data Data generated during experiments...

13 ZEBANC AND VM Digitalisation established for samples from frozen section labs Technical quality control implemented Infrastructure for block-centric navigation established Medical quality control in use (prototype tumor area detection) TMA as sample array in CentraXX integrated Several quantification procedures implemented to generate additional sample features Virtual studies on virtual slides instead of real samples Virtual microscopy has a huge potential for services in the context of a biomaterial bank

14 Clinical Pathology Second Opinion, Studies, Marker Quantification Medical Workstation - All Information in One View

15 Workflow of Tumor Board Meetings Set bookmarks and make annotations Browse slides and move to annotations Before meeting On meeting

16 Quantification

17 Structure of the talk Possible workflow routine diagnostic diagnostic support Which scanner? Quantification / image analysis Conclusions

18 Requirements on Slide Scanning Correct slide information is present Completeness of tissue Image sharpness Color fidelity

19 VIRTUAL MICROSCOPY SCANNER CONTEST P. Hufnagl 1,2, N.Zerbe 2 1 University of Applied Science Berlin, Berlin, German 2 Institute for Pathology, Charité Berlin, Germany 2 nd International Scanner Contest technology meets pathology

20 MISSION 2 nd International Scanner Contest technology meets pathology Determination of the state of the art in slide scanning Support of pathologists and scientists to find the appropriate scanner for their applications Support of vendors to understand the needs of pathologists Determination of quality of WSI within the context of pathology Development of a set of standard features for the characterization and comparison of scanning devices

21 DISCIPLINES 2 nd International Scanner Contest technology meets pathology High Throughput Quality Fluorescence Image Analysis Technical

22 2 nd International Scanner Contest QUALITY EVALUATION TERMINALS technology meets pathology

23 VENDORS. 2 nd International Scanner Contest technology meets pathology

24 2 nd International Scanner Contest TECHNICAL COLOUR & GEOMETRY AIMS & MATERIAL technology meets pathology Aims: Measurement of colour fidelity and colour resolution of devices Determination of true effective pixel size Detection of image distortions Material: IT8.7/1 Colour Target mounted on glas slide 264 colour & 24 grey value fields known absorption spectra and colourimetric coordinates Grid pattern glas slide overall size: 1 x 3 (25mm x 75mm) image area: 20mm x 50mm clear aperture: 8.5 µm² opaque lines: 1.5 µm² pitch: 10 µm

25 2 nd International Scanner Contest TECHNICAL COLOUR & GEOMETRY TASK technology meets pathology General Conditions: All participants had to scan the same slide Any manual interaction was allowed Rescan of slide was allowed 1h time limit Evaluation: Colour difference calculation to CIEDE2000 average inside middle 50% of each field low-resolution scan Measurements inside whole slide images Inside sensor field (no stitching) 9 sensor fields 18 measurements each

26 2 nd International Scanner Contest technology meets pathology TECHNICAL COLOUR TEST METHOD Fidelity test: average de over 144 fields de avg = Σ de(c * mes, c ref ) /144 mix-colours matrix for fidelity test Colour difference calculation to CIEDE2000 colour step wedges

27 2 nd International Scanner Contest TECHNICAL COLOUR SOFTWARE technology meets pathology

28 COMPLETENESS OF SCAN 2 nd International Scanner Contest technology meets pathology

29 2 nd International Scanner Contest technology meets pathology HIGH THROUGHPUT AIMS & MATERIAL 29

30 2 nd International Scanner Contest technology meets pathology HIGH THROUGHPUT AIMS & MATERIAL 30

31 2 nd International Scanner Contest DIGITALISATION - SCANMASTER technology meets pathology Sample identifikation (barcode / OCR) Sharpness assessment + Completeness of particles

32 2 nd International Scanner Contest FOCUS QUALITY ASSESSMENT technology meets pathology Green: quality sufficiant Red: not sharp, possible artefacts

33

34 2 nd International Scanner Contest technology meets pathology

35 2 nd International Scanner Contest technology meets pathology VIEWING ON A VIRTUAL MICROSCOPE

36 RESOLUTION 2 nd International Scanner Contest technology meets pathology MAGNIFICATION IS NOT RESOLUTION AND OPTICAL RESOLUTION IS NOT DIGITAL RESOLUTION!

37 Several Positions in Test Leap Motion over the table Leap Motion under acryl glass pane

38 Leap Motion Stereo imaging based on infrared cameras (

39 Handling Possible workflow

40 Gesture Control Next/ previous slide Zoom in/ out

41 Histological Image Registration Goal Inter-Modal Registration (Stain-To-Stain) Applications WSI Navigation Support Virtual Staining 3D Reconstruction Approach Intensity Based Multi-Resolution Similarity Measure Mutual Information Transformation Models Rigid: Rotation + Translation Affine: Linear Transformation + Translation Free Form Deformation: B-Splines Optimization Gradient Descent

42 Registration Of Renal Images: Reference Image (H&E) Template Image (SFOG)

43 Reference Image (H&E)

44 Template Image (SFOG)

45 Coarse to Fine Image Registration: Rigid, Affine, Free Form Model

46 Rigid Registration

47 Affine Registration

48 Free Form Registration

49 Reference Image

50 Structure of the talk Possible workflow routine diagnostic diagnostic support Which scanner? Quantification / image analysis Conclusion

51 The strong reputation of pathology is becoming weaker.. if trust is gone, you almost never get it back

52 Standardized Quantification in Tumor Pathology Nat J Inst., preprint November

53 . At the St. Gallen cutoff of 13.5 % there are 32.3 % high Ki67 by Lab A while Lab B would call the same cases low Ki

54 Optical Illusions Not always funny, sometimes really critical

55 Human brain: Square a is lighter than square b! Reality: Both are identical

56 Simulated Ki67 15%

57 Estimation of variation of Ki67 scoring

58 FEATURE BASED MULTIRESOLUTION CORRESPONDENCE Combined tumor annotations Individual tumor annotations Annotated as tumor by 10 pathologists By 4 pathologists By 2 pathologists tumor non-tumor by no pathologist

59 CLASSIFICATION RESULTS Gastric Cancer (HER2) Transmission to HE Learning Sample 59

60 Structure of the talk Possible workflow routine diagnostic diagnostic support Which scanner? Quantification / image analysis Conclusions

61 Summary on Relevance of VM Workflow routine diagnostic diagnostic support Which scanner? Quantification / image analysis In routine path not yet active on a broad level Excellent instruments exist, but they have to be integrated properly Will become very important in personalized medicine Clinical-pathological tumorconferences (tumor board) Is already very important Biobanking Is important in research institutions

62 Most important requirements on VM Clinical data (LIMS) are correctly connected to WSI Completeness of tissue Image sharpness Color fidelity Compression is adequate Resolution and quality of the monitor is sufficiant Test continously!

63 Let s go virtual 13th European Conference on Digital Pathology Berlin May 25. /

64 Acknowledgement Team Norman Zerbe, Karsten Schlüns, Sebastian Lohmann, Mario Domhardt, Björn Lindequist, Daniel Heim, Stephan Wienert, Kai Saeger, Thorsten Knape, Arend Müller, Wolfram Schädel, Uwe Brunner, Thomas Schrader, Manfred Dietel 2 nd International Scanner Contest technology meets pathology