TeCNO: Surgical Phase Recognition with Multi-stage Temporal Convolutional Networks

Tobias Czempiel; Magdalini Paschali; Matthias Keicher; Walter Simson; Hubertus Feussner; Seong Tae Kim; Nassir Navab

doi:10.1007/978-3-030-59716-0_33

TeCNO: Surgical Phase Recognition with Multi-stage Temporal Convolutional Networks

Tobias Czempiel
, Magdalini Paschali
, Matthias Keicher
, Walter Simson
, Hubertus Feussner
, Seong Tae Kim
, Nassir Navab

Technical University of Munich
Johns Hopkins University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

219 Scopus citations

Abstract

Automatic surgical phase recognition is a challenging and crucial task with the potential to improve patient safety and become an integral part of intra-operative decision-support systems. In this paper, we propose, for the first time in workflow analysis, a Multi-Stage Temporal Convolutional Network (MS-TCN) that performs hierarchical prediction refinement for surgical phase recognition. Causal, dilated convolutions allow for a large receptive field and online inference with smooth predictions even during ambiguous transitions. Our method is thoroughly evaluated on two datasets of laparoscopic cholecystectomy videos with and without the use of additional surgical tool information. Outperforming various state-of-the-art LSTM approaches, we verify the suitability of the proposed causal MS-TCN for surgical phase recognition.

Original language	English
Title of host publication	Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings
Editors	Anne L. Martel, Purang Abolmaesumi, Danail Stoyanov, Diana Mateus, Maria A. Zuluaga, S. Kevin Zhou, Daniel Racoceanu, Leo Joskowicz
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	343-352
Number of pages	10
ISBN (Print)	9783030597153
DOIs	https://doi.org/10.1007/978-3-030-59716-0_33
State	Published - 2020
Event	23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020 - Lima, Peru Duration: 4 Oct 2020 → 8 Oct 2020

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12263 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020
Country/Territory	Peru
City	Lima
Period	4/10/20 → 8/10/20

Keywords

Cholecystectomy
Endoscopic videos
Surgical phase recognition
Surgical workflow
Temporal Convolutional Networks

Access to Document

10.1007/978-3-030-59716-0_33

Cite this

Czempiel, T., Paschali, M., Keicher, M., Simson, W., Feussner, H., Kim, S. T., & Navab, N. (2020). TeCNO: Surgical Phase Recognition with Multi-stage Temporal Convolutional Networks. In A. L. Martel, P. Abolmaesumi, D. Stoyanov, D. Mateus, M. A. Zuluaga, S. K. Zhou, D. Racoceanu, & L. Joskowicz (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings (pp. 343-352). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12263 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-59716-0_33

Czempiel, Tobias ; Paschali, Magdalini ; Keicher, Matthias et al. / TeCNO : Surgical Phase Recognition with Multi-stage Temporal Convolutional Networks. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings. editor / Anne L. Martel ; Purang Abolmaesumi ; Danail Stoyanov ; Diana Mateus ; Maria A. Zuluaga ; S. Kevin Zhou ; Daniel Racoceanu ; Leo Joskowicz. Springer Science and Business Media Deutschland GmbH, 2020. pp. 343-352 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{8d797e894b8a4cea9ae420fd291331ec,

title = "TeCNO: Surgical Phase Recognition with Multi-stage Temporal Convolutional Networks",

abstract = "Automatic surgical phase recognition is a challenging and crucial task with the potential to improve patient safety and become an integral part of intra-operative decision-support systems. In this paper, we propose, for the first time in workflow analysis, a Multi-Stage Temporal Convolutional Network (MS-TCN) that performs hierarchical prediction refinement for surgical phase recognition. Causal, dilated convolutions allow for a large receptive field and online inference with smooth predictions even during ambiguous transitions. Our method is thoroughly evaluated on two datasets of laparoscopic cholecystectomy videos with and without the use of additional surgical tool information. Outperforming various state-of-the-art LSTM approaches, we verify the suitability of the proposed causal MS-TCN for surgical phase recognition.",

keywords = "Cholecystectomy, Endoscopic videos, Surgical phase recognition, Surgical workflow, Temporal Convolutional Networks",

author = "Tobias Czempiel and Magdalini Paschali and Matthias Keicher and Walter Simson and Hubertus Feussner and Kim, \{Seong Tae\} and Nassir Navab",

note = "Publisher Copyright: {\textcopyright} 2020, Springer Nature Switzerland AG.; 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020 ; Conference date: 04-10-2020 Through 08-10-2020",

year = "2020",

doi = "10.1007/978-3-030-59716-0\_33",

language = "English",

isbn = "9783030597153",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "343--352",

editor = "Martel, \{Anne L.\} and Purang Abolmaesumi and Danail Stoyanov and Diana Mateus and Zuluaga, \{Maria A.\} and Zhou, \{S. Kevin\} and Daniel Racoceanu and Leo Joskowicz",

booktitle = "Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings",

}

Czempiel, T, Paschali, M, Keicher, M, Simson, W, Feussner, H, Kim, ST & Navab, N 2020, TeCNO: Surgical Phase Recognition with Multi-stage Temporal Convolutional Networks. in AL Martel, P Abolmaesumi, D Stoyanov, D Mateus, MA Zuluaga, SK Zhou, D Racoceanu & L Joskowicz (eds), Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12263 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 343-352, 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020, Lima, Peru, 4/10/20. https://doi.org/10.1007/978-3-030-59716-0_33

TeCNO: Surgical Phase Recognition with Multi-stage Temporal Convolutional Networks. / Czempiel, Tobias; Paschali, Magdalini; Keicher, Matthias et al.
Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings. ed. / Anne L. Martel; Purang Abolmaesumi; Danail Stoyanov; Diana Mateus; Maria A. Zuluaga; S. Kevin Zhou; Daniel Racoceanu; Leo Joskowicz. Springer Science and Business Media Deutschland GmbH, 2020. p. 343-352 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12263 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - TeCNO

T2 - 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020

AU - Czempiel, Tobias

AU - Paschali, Magdalini

AU - Keicher, Matthias

AU - Simson, Walter

AU - Feussner, Hubertus

AU - Kim, Seong Tae

AU - Navab, Nassir

PY - 2020

Y1 - 2020

N2 - Automatic surgical phase recognition is a challenging and crucial task with the potential to improve patient safety and become an integral part of intra-operative decision-support systems. In this paper, we propose, for the first time in workflow analysis, a Multi-Stage Temporal Convolutional Network (MS-TCN) that performs hierarchical prediction refinement for surgical phase recognition. Causal, dilated convolutions allow for a large receptive field and online inference with smooth predictions even during ambiguous transitions. Our method is thoroughly evaluated on two datasets of laparoscopic cholecystectomy videos with and without the use of additional surgical tool information. Outperforming various state-of-the-art LSTM approaches, we verify the suitability of the proposed causal MS-TCN for surgical phase recognition.

AB - Automatic surgical phase recognition is a challenging and crucial task with the potential to improve patient safety and become an integral part of intra-operative decision-support systems. In this paper, we propose, for the first time in workflow analysis, a Multi-Stage Temporal Convolutional Network (MS-TCN) that performs hierarchical prediction refinement for surgical phase recognition. Causal, dilated convolutions allow for a large receptive field and online inference with smooth predictions even during ambiguous transitions. Our method is thoroughly evaluated on two datasets of laparoscopic cholecystectomy videos with and without the use of additional surgical tool information. Outperforming various state-of-the-art LSTM approaches, we verify the suitability of the proposed causal MS-TCN for surgical phase recognition.

KW - Cholecystectomy

KW - Endoscopic videos

KW - Surgical phase recognition

KW - Surgical workflow

KW - Temporal Convolutional Networks

UR - https://www.scopus.com/pages/publications/85092704071

U2 - 10.1007/978-3-030-59716-0_33

DO - 10.1007/978-3-030-59716-0_33

M3 - Conference contribution

AN - SCOPUS:85092704071

SN - 9783030597153

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 343

EP - 352

BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings

A2 - Martel, Anne L.

A2 - Abolmaesumi, Purang

A2 - Stoyanov, Danail

A2 - Mateus, Diana

A2 - Zuluaga, Maria A.

A2 - Zhou, S. Kevin

A2 - Racoceanu, Daniel

A2 - Joskowicz, Leo

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 4 October 2020 through 8 October 2020

ER -

Czempiel T, Paschali M, Keicher M, Simson W, Feussner H, Kim ST et al. TeCNO: Surgical Phase Recognition with Multi-stage Temporal Convolutional Networks. In Martel AL, Abolmaesumi P, Stoyanov D, Mateus D, Zuluaga MA, Zhou SK, Racoceanu D, Joskowicz L, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings. Springer Science and Business Media Deutschland GmbH. 2020. p. 343-352. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-59716-0_33

TeCNO: Surgical Phase Recognition with Multi-stage Temporal Convolutional Networks

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this