Generating X-ray Images from Point Clouds Using Conditional Generative Adversarial Networks

Mustafa Haiderbhai, Sergio Ledesma, Nassir Navab, Pascal Fallavollita

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Scopus citations

Abstract

Simulating medical images such as X-rays is of key interest to reduce radiation in non-diagnostic visualization scenarios. Past state of the art methods utilize ray tracing, which is reliant on 3D models. To our knowledge, no approach exists for cases where point clouds from depth cameras and other sensors are the only input modality. We propose a method for estimating an X-ray image from a generic point cloud using a conditional generative adversarial network (CGAN). We train a CGAN pix2pix to translate point cloud images into X-ray images using a dataset created inside our custom synthetic data generator. Additionally, point clouds of multiple densities are examined to determine the effect of density on the image translation problem. The results from the CGAN show that this type of network can predict X-ray images from points clouds. Higher point cloud densities outperformed the two lowest point cloud densities. However, the networks trained with high-density point clouds did not exhibit a significant difference when compared with the networks trained with medium densities. We prove that CGANs can be applied to image translation problems in the medical domain and show the feasibility of using this approach when 3D models are not available. Further work includes overcoming the occlusion and quality limitations of the generic approach and applying CGANs to other medical image translation problems.

Original languageEnglish
Title of host publication42nd Annual International Conferences of the IEEE Engineering in Medicine and Biology Society
Subtitle of host publicationEnabling Innovative Technologies for Global Healthcare, EMBC 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1588-1591
Number of pages4
ISBN (Electronic)9781728119908
DOIs
StatePublished - Jul 2020
Event42nd Annual International Conferences of the IEEE Engineering in Medicine and Biology Society, EMBC 2020 - Montreal, Canada
Duration: 20 Jul 202024 Jul 2020

Publication series

NameProceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS
Volume2020-July
ISSN (Print)1557-170X

Conference

Conference42nd Annual International Conferences of the IEEE Engineering in Medicine and Biology Society, EMBC 2020
Country/TerritoryCanada
CityMontreal
Period20/07/2024/07/20

Fingerprint

Dive into the research topics of 'Generating X-ray Images from Point Clouds Using Conditional Generative Adversarial Networks'. Together they form a unique fingerprint.

Cite this