Multitask Weakly Supervised Generative Network for MR-US Registration

Mohammad Farid Azampour, Kristina Mach, Emad Fatemizadeh, Beatrice Demiray, Kay Westenfelder, Katja Steiger, Matthias Eiber, Thomas Wendler, Bernhard Kainz, Nassir Navab

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

Registering pre-operative modalities, such as magnetic resonance imaging or computed tomography, to ultrasound images is crucial for guiding clinicians during surgeries and biopsies. Recently, deep-learning approaches have been proposed to increase the speed and accuracy of this registration problem. However, all of these approaches need expensive supervision from the ultrasound domain. In this work, we propose a multitask generative framework that needs weak supervision only from the pre-operative imaging domain during training. To perform a deformable registration, the proposed framework translates a magnetic resonance image to the ultrasound domain while preserving the structural content. To demonstrate the efficacy of the proposed method, we tackle the registration problem of pre-operative 3D MR to transrectal ultrasonography images as necessary for targeted prostate biopsies. We use an in-house dataset of 600 patients, divided into 540 for training, 30 for validation, and the remaining for testing. An expert manually segmented the prostate in both modalities for validation and test sets to assess the performance of our framework. The proposed framework achieves a 3.58 mm target registration error on the expert-selected landmarks, 89.2% in the Dice score, and 1.81 mm 95th percentile Hausdorff distance on the prostate masks in the test set. Our experiments demonstrate that the proposed generative model successfully translates magnetic resonance images into the ultrasound domain. The translated image contains the structural content and fine details due to an ultrasound-specific two-path design of the generative model. The proposed framework enables training learning-based registration methods while only weak supervision from the pre-operative domain is available.

Original languageEnglish
Pages (from-to)1
Number of pages1
JournalIEEE Transactions on Medical Imaging
DOIs
StateAccepted/In press - 2024
Externally publishedYes

Keywords

  • Biopsy
  • Convolutional neural network
  • Deep learning
  • Image segmentation
  • Measurement
  • Medical image registration
  • Multi-task learning
  • Multitasking
  • Training
  • Ultrasonic imaging
  • Unpaired image-to-image translation
  • US-MR registration

Fingerprint

Dive into the research topics of 'Multitask Weakly Supervised Generative Network for MR-US Registration'. Together they form a unique fingerprint.

Cite this