TY - GEN
T1 - Prior-RadGraphFormer
T2 - 5th Workshop on GRaphs in biomedicAl Image anaLysis Satellite event at MICCAI, GRAIL 2023 and 1st Cell Detection from Cell-Tissue Interaction challenge in MICCAI, OCELOT 2023 Held in Conjunction with International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023
AU - Xiong, Yiheng
AU - Liu, Jingsong
AU - Zaripova, Kamilia
AU - Sharifzadeh, Sahand
AU - Keicher, Matthias
AU - Navab, Nassir
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.
PY - 2024
Y1 - 2024
N2 - The extraction of structured clinical information from free-text radiology reports in the form of radiology graphs has been demonstrated to be a valuable approach for evaluating the clinical correctness of report-generation methods. However, the direct generation of radiology graphs from chest X-ray (CXR) images has not been attempted. To address this gap, we propose a novel approach called Prior-RadGraphFormer that utilizes a transformer model with prior knowledge in the form of a probabilistic knowledge graph (PKG) to generate radiology graphs directly from CXR images. The PKG models the statistical relationship between radiology entities, including anatomical structures and medical observations. This additional contextual information enhances the accuracy of entity and relation extraction. The generated radiology graphs can be applied to various downstream tasks, such as free-text or structured reports generation and multi-label classification of pathologies. Our approach represents a promising method for generating radiology graphs directly from CXR images, and has significant potential for improving medical image analysis and clinical decision-making. Our code is open sourced at https://github.com/xiongyiheng/Prior-RadGraphFormer.
AB - The extraction of structured clinical information from free-text radiology reports in the form of radiology graphs has been demonstrated to be a valuable approach for evaluating the clinical correctness of report-generation methods. However, the direct generation of radiology graphs from chest X-ray (CXR) images has not been attempted. To address this gap, we propose a novel approach called Prior-RadGraphFormer that utilizes a transformer model with prior knowledge in the form of a probabilistic knowledge graph (PKG) to generate radiology graphs directly from CXR images. The PKG models the statistical relationship between radiology entities, including anatomical structures and medical observations. This additional contextual information enhances the accuracy of entity and relation extraction. The generated radiology graphs can be applied to various downstream tasks, such as free-text or structured reports generation and multi-label classification of pathologies. Our approach represents a promising method for generating radiology graphs directly from CXR images, and has significant potential for improving medical image analysis and clinical decision-making. Our code is open sourced at https://github.com/xiongyiheng/Prior-RadGraphFormer.
KW - Prior Knowledge
KW - Radiology Graph Generation
KW - Transformer
UR - http://www.scopus.com/inward/record.url?scp=85188682527&partnerID=8YFLogxK
U2 - 10.1007/978-3-031-55088-1_5
DO - 10.1007/978-3-031-55088-1_5
M3 - Conference contribution
AN - SCOPUS:85188682527
SN - 9783031550874
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 54
EP - 63
BT - Graphs in Biomedical Image Analysis, and Overlapped Cell on Tissue Dataset for Histopathology - 5th MICCAI Workshop, GRAIL 2023 and 1st MICCAI Challenge, OCELOT 2023, Held in Conjunction with MICCAI 2023, Proceedings
A2 - Ahmadi, Seyed-Ahmad
A2 - Pereira, Sérgio
PB - Springer Science and Business Media Deutschland GmbH
Y2 - 8 October 2023 through 12 October 2023
ER -