Relevance-based visualization to improve surgeon perception

Olivier Pauly; Benoît Diotte; Séverine Habert; Simon Weidert; Ekkehard Euler; Pascal Fallavollita; Nassir Navab

doi:10.1007/978-3-319-07521-1_19

Relevance-based visualization to improve surgeon perception

Olivier Pauly, Benoît Diotte, Séverine Habert, Simon Weidert, Ekkehard Euler, Pascal Fallavollita, Nassir Navab

Informatics 16 - Chair of Computer Aided Medical Procedures

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

13 Scopus citations

Abstract

In computer-aided interventions, the visual feedback of the doctor is vital. Enhancing the relevant object will help for the perception of this feedback. In this paper, we present a learning-based labeling of the surgical scene using a depth camera (comprised of RGB and depth range sensors). The depth sensor is used for background extraction and Random Forests are used for segmenting color images. The end result is a labeled scene consisting of surgeon hands, surgical instruments and background labels. We evaluated the method by conducting 10 simulated surgeries with 5 clinicians and demonstrated that the approach provides surgeons a dissected surgical scene, enhanced visualization, and upgraded depth perception.

Original language	English
Title of host publication	Information Processing in Computer-Assisted Interventions - 5th International Conference, IPCAI 2014, Proceedings
Publisher	Springer Verlag
Pages	178-185
Number of pages	8
ISBN (Print)	9783319075204
DOIs	https://doi.org/10.1007/978-3-319-07521-1_19
State	Published - 2014
Event	5th International Conference on Information Processing in Computer-Assisted Interventions, IPCAI 2014 - Fukuoka, Japan Duration: 28 Jun 2014 → 28 Jun 2014

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	8498 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	5th International Conference on Information Processing in Computer-Assisted Interventions, IPCAI 2014
Country/Territory	Japan
City	Fukuoka
Period	28/06/14 → 28/06/14

Keywords

machine learning
medical augmented reality
multimodal image fusion
operating room
visualization

Access to Document

10.1007/978-3-319-07521-1_19

Cite this

Pauly, O., Diotte, B., Habert, S., Weidert, S., Euler, E., Fallavollita, P., & Navab, N. (2014). Relevance-based visualization to improve surgeon perception. In Information Processing in Computer-Assisted Interventions - 5th International Conference, IPCAI 2014, Proceedings (pp. 178-185). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8498 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-07521-1_19

Pauly, Olivier ; Diotte, Benoît ; Habert, Séverine et al. / Relevance-based visualization to improve surgeon perception. Information Processing in Computer-Assisted Interventions - 5th International Conference, IPCAI 2014, Proceedings. Springer Verlag, 2014. pp. 178-185 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{9be4ddc6a6b44184bf1ae460e07d8ca7,

title = "Relevance-based visualization to improve surgeon perception",

abstract = "In computer-aided interventions, the visual feedback of the doctor is vital. Enhancing the relevant object will help for the perception of this feedback. In this paper, we present a learning-based labeling of the surgical scene using a depth camera (comprised of RGB and depth range sensors). The depth sensor is used for background extraction and Random Forests are used for segmenting color images. The end result is a labeled scene consisting of surgeon hands, surgical instruments and background labels. We evaluated the method by conducting 10 simulated surgeries with 5 clinicians and demonstrated that the approach provides surgeons a dissected surgical scene, enhanced visualization, and upgraded depth perception.",

keywords = "machine learning, medical augmented reality, multimodal image fusion, operating room, visualization",

author = "Olivier Pauly and Beno{\^i}t Diotte and S{\'e}verine Habert and Simon Weidert and Ekkehard Euler and Pascal Fallavollita and Nassir Navab",

year = "2014",

doi = "10.1007/978-3-319-07521-1_19",

language = "English",

isbn = "9783319075204",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "178--185",

booktitle = "Information Processing in Computer-Assisted Interventions - 5th International Conference, IPCAI 2014, Proceedings",

note = "5th International Conference on Information Processing in Computer-Assisted Interventions, IPCAI 2014 ; Conference date: 28-06-2014 Through 28-06-2014",

}

Pauly, O, Diotte, B, Habert, S, Weidert, S, Euler, E, Fallavollita, P & Navab, N 2014, Relevance-based visualization to improve surgeon perception. in Information Processing in Computer-Assisted Interventions - 5th International Conference, IPCAI 2014, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 8498 LNCS, Springer Verlag, pp. 178-185, 5th International Conference on Information Processing in Computer-Assisted Interventions, IPCAI 2014, Fukuoka, Japan, 28/06/14. https://doi.org/10.1007/978-3-319-07521-1_19

Relevance-based visualization to improve surgeon perception. / Pauly, Olivier; Diotte, Benoît; Habert, Séverine et al.
Information Processing in Computer-Assisted Interventions - 5th International Conference, IPCAI 2014, Proceedings. Springer Verlag, 2014. p. 178-185 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8498 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Relevance-based visualization to improve surgeon perception

AU - Pauly, Olivier

AU - Diotte, Benoît

AU - Habert, Séverine

AU - Weidert, Simon

AU - Euler, Ekkehard

AU - Fallavollita, Pascal

AU - Navab, Nassir

PY - 2014

Y1 - 2014

N2 - In computer-aided interventions, the visual feedback of the doctor is vital. Enhancing the relevant object will help for the perception of this feedback. In this paper, we present a learning-based labeling of the surgical scene using a depth camera (comprised of RGB and depth range sensors). The depth sensor is used for background extraction and Random Forests are used for segmenting color images. The end result is a labeled scene consisting of surgeon hands, surgical instruments and background labels. We evaluated the method by conducting 10 simulated surgeries with 5 clinicians and demonstrated that the approach provides surgeons a dissected surgical scene, enhanced visualization, and upgraded depth perception.

AB - In computer-aided interventions, the visual feedback of the doctor is vital. Enhancing the relevant object will help for the perception of this feedback. In this paper, we present a learning-based labeling of the surgical scene using a depth camera (comprised of RGB and depth range sensors). The depth sensor is used for background extraction and Random Forests are used for segmenting color images. The end result is a labeled scene consisting of surgeon hands, surgical instruments and background labels. We evaluated the method by conducting 10 simulated surgeries with 5 clinicians and demonstrated that the approach provides surgeons a dissected surgical scene, enhanced visualization, and upgraded depth perception.

KW - machine learning

KW - medical augmented reality

KW - multimodal image fusion

KW - operating room

KW - visualization

UR - http://www.scopus.com/inward/record.url?scp=84958540974&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-07521-1_19

DO - 10.1007/978-3-319-07521-1_19

M3 - Conference contribution

AN - SCOPUS:84958540974

SN - 9783319075204

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 178

EP - 185

BT - Information Processing in Computer-Assisted Interventions - 5th International Conference, IPCAI 2014, Proceedings

PB - Springer Verlag

T2 - 5th International Conference on Information Processing in Computer-Assisted Interventions, IPCAI 2014

Y2 - 28 June 2014 through 28 June 2014

ER -

Pauly O, Diotte B, Habert S, Weidert S, Euler E, Fallavollita P et al. Relevance-based visualization to improve surgeon perception. In Information Processing in Computer-Assisted Interventions - 5th International Conference, IPCAI 2014, Proceedings. Springer Verlag. 2014. p. 178-185. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-07521-1_19

Relevance-based visualization to improve surgeon perception

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this