Radiological Reports Improve Pre-training for Localized Imaging Tasks on Chest X-Rays

Philip Müller, Georgios Kaissis, Congyu Zou, Daniel Rueckert

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

5 Scopus citations

Abstract

Self-supervised pre-training on unlabeled images has shown promising results in the medical domain. Recently, methods using text-supervision from companion text like radiological reports improved upon these results even further. However, most works in the medical domain focus on image classification downstream tasks and do not study more localized tasks like semantic segmentation or object detection. We therefore propose a novel evaluation framework consisting of 18 localized tasks, including semantic segmentation and object detection, on five public chest radiography datasets. Using our proposed evaluation framework, we study the effectiveness of existing text-supervised methods and compare them with image-only self-supervised methods and transfer from classification in more than 1200 evaluation runs. Our experiments show that text-supervised methods outperform all other methods on 13 out of 18 tasks making them the preferred method. In conclusion, image-only contrastive methods provide a strong baseline if no reports are available while transfer from classification, even in-domain, does not perform well in pre-training for localized tasks.

Original languageEnglish
Title of host publicationMedical Image Computing and Computer Assisted Intervention – MICCAI 2022 - 25th International Conference, Proceedings
EditorsLinwei Wang, Qi Dou, P. Thomas Fletcher, Stefanie Speidel, Shuo Li
PublisherSpringer Science and Business Media Deutschland GmbH
Pages647-657
Number of pages11
ISBN (Print)9783031164422
DOIs
StatePublished - 2022
Event25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022 - Singapore, Singapore
Duration: 18 Sep 202222 Sep 2022

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13435 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022
Country/TerritorySingapore
CitySingapore
Period18/09/2222/09/22

Keywords

  • Radiology reports
  • Self-supervised representation learning

Fingerprint

Dive into the research topics of 'Radiological Reports Improve Pre-training for Localized Imaging Tasks on Chest X-Rays'. Together they form a unique fingerprint.

Cite this