DeepScope: Nonintrusive whole slide saliency annotation and prediction from pathologists at the microscope

Andrew J. Schaumberg, S. Joseph Sirintrapun, Hikmat A. Al-Ahmadie, Peter J. Schüffler, Thomas J. Fuchs

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

6 Scopus citations

Abstract

Modern digital pathology departments have grown to produce whole-slide image data at petabyte scale, an unprecedented treasure chest for medical machine learning tasks. Unfortunately, most digital slides are not annotated at the image level, hindering large-scale application of supervised learning. Manual labeling is prohibitive, requiring pathologists with decades of training and outstanding clinical service responsibilities. This problem is further aggravated by the United States Food and Drug Administration’s ruling that primary diagnosis must come from a glass slide rather than a digital image. We present the first end-to-end framework to overcome this problem, gathering annotations in a nonintrusive manner during a pathologist’s routine clinical work: (i) microscope-specific 3D-printed commodity camera mounts are used to video record the glass-slide-based clinical diagnosis process; (ii) after routine scanning of the whole slide, the video frames are registered to the digital slide; (iii) motion and observation time are estimated to generate a spatial and temporal saliency map of the whole slide. Demonstrating the utility of these annotations, we train a convolutional neural network that detects diagnosis-relevant salient regions, then report accuracy of 85.15% in bladder and 91.40% in prostate, with 75.00% accuracy when training on prostate but predicting in bladder, despite different pathologists examining the different tissues. When training on one patient but testing on another, AUROC in bladder is 0.79 ± 0.11 and in prostate is 0.96 ± 0.04. Our tool is available at https://bitbucket.org/aschaumberg/deepscope.

Original languageEnglish
Title of host publicationComputational Intelligence Methods for Bioinformatics and Biostatistics - 13th International Meeting, CIBB 2016, Revised Selected Papers
EditorsGiulio Caravagna, Roberto Tagliaferri, David Gilbert, Andrea Bracciali
PublisherSpringer Verlag
Pages42-58
Number of pages17
ISBN (Print)9783319678337
DOIs
StatePublished - 2017
Externally publishedYes
Event13th International Meeting on Computational Intelligence Methods for Bioinformatics and Biostatistics, CIBB 2016 - Stirling, United Kingdom
Duration: 1 Sep 20163 Sep 2016

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10477 LNBI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference13th International Meeting on Computational Intelligence Methods for Bioinformatics and Biostatistics, CIBB 2016
Country/TerritoryUnited Kingdom
CityStirling
Period1/09/163/09/16

Fingerprint

Dive into the research topics of 'DeepScope: Nonintrusive whole slide saliency annotation and prediction from pathologists at the microscope'. Together they form a unique fingerprint.

Cite this