TY - GEN
T1 - General Vision Encoder Features as Guidance in Medical Image Registration
AU - Kögl, Fryderyk
AU - Reithmeir, Anna
AU - Sideri-Lampretsa, Vasiliki
AU - Machado, Ines
AU - Braren, Rickmer
AU - Rueckert, Daniel
AU - Schnabel, Julia A.
AU - Zimmer, Veronika A.
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.
PY - 2024
Y1 - 2024
N2 - General vision encoders like DINOv2 and SAM have recently transformed computer vision. Even though they are trained on natural images, such encoder models have excelled in medical imaging, e.g., in classification, segmentation, and registration. However, no in-depth comparison of different state-of-the-art general vision encoders for medical registration is available. In this work, we investigate how well general vision encoder features can be used in the dissimilarity metrics for medical image registration. We explore two encoders that were trained on natural images as well as one that was fine-tuned on medical data. We apply the features within the well-established B-spline FFD registration framework. In extensive experiments on cardiac cine MRI data, we find that using features as additional guidance for conventional metrics improves the registration quality. The code is available at github.com/compai-lab/2024-miccai-koegl.
AB - General vision encoders like DINOv2 and SAM have recently transformed computer vision. Even though they are trained on natural images, such encoder models have excelled in medical imaging, e.g., in classification, segmentation, and registration. However, no in-depth comparison of different state-of-the-art general vision encoders for medical registration is available. In this work, we investigate how well general vision encoder features can be used in the dissimilarity metrics for medical image registration. We explore two encoders that were trained on natural images as well as one that was fine-tuned on medical data. We apply the features within the well-established B-spline FFD registration framework. In extensive experiments on cardiac cine MRI data, we find that using features as additional guidance for conventional metrics improves the registration quality. The code is available at github.com/compai-lab/2024-miccai-koegl.
KW - feature-based distance measures
KW - Foundation models
UR - http://www.scopus.com/inward/record.url?scp=85206946945&partnerID=8YFLogxK
U2 - 10.1007/978-3-031-73480-9_21
DO - 10.1007/978-3-031-73480-9_21
M3 - Conference contribution
AN - SCOPUS:85206946945
SN - 9783031734793
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 265
EP - 279
BT - Biomedical Image Registration - 11th International Workshop, WBIR 2024, Held in Conjunction with MICCAI 2024, Proceedings
A2 - Modat, Marc
A2 - Špiclin, Žiga
A2 - Hering, Alessa
A2 - Simpson, Ivor
A2 - Bastiaansen, Wietske
A2 - Mok, Tony C. W.
PB - Springer Science and Business Media Deutschland GmbH
T2 - 11th International Workshop on Biomedical Image Registration, WBIR 2024, held in conjunction with the 27th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2024
Y2 - 6 October 2024 through 6 October 2024
ER -