Remote Interactive Surgery Platform (RISP): Proof of Concept for an Augmented-Reality-Based Platform for Surgical Telementoring

Yannik Kalbas; Hoijoon Jung; John Ricklin; Ge Jin; Mingjian Li; Thomas Rauer; Shervin Dehghani; Nassir Navab; Jinman Kim; Hans Christoph Pape; Sandro Michael Heining

doi:10.3390/jimaging9030056

Remote Interactive Surgery Platform (RISP): Proof of Concept for an Augmented-Reality-Based Platform for Surgical Telementoring

Yannik Kalbas
, Hoijoon Jung
, John Ricklin
, Ge Jin
, Mingjian Li
, Thomas Rauer
, Shervin Dehghani
, Nassir Navab
, Jinman Kim
, Hans Christoph Pape
, Sandro Michael Heining

Informatics 16 - Chair of Computer Aided Medical Procedures

University Hospital Zurich
University of Sydney
Technical University of Munich

Research output: Contribution to journal › Article › peer-review

11 Scopus citations

Abstract

The “Remote Interactive Surgery Platform” (RISP) is an augmented reality (AR)-based platform for surgical telementoring. It builds upon recent advances of mixed reality head-mounted displays (MR-HMD) and associated immersive visualization technologies to assist the surgeon during an operation. It enables an interactive, real-time collaboration with a remote consultant by sharing the operating surgeon’s field of view through the Microsoft (MS) HoloLens2 (HL2). Development of the RISP started during the Medical Augmented Reality Summer School 2021 and is currently still ongoing. It currently includes features such as three-dimensional annotations, bidirectional voice communication and interactive windows to display radiographs within the sterile field. This manuscript provides an overview of the RISP and preliminary results regarding its annotation accuracy and user experience measured with ten participants.

Original language	English
Article number	56
Journal	Journal of Imaging
Volume	9
Issue number	3
DOIs	https://doi.org/10.3390/jimaging9030056
State	Published - Mar 2023

Keywords

AR-assisted surgery
HoloLens2
MR-HMD
RISP
augmented reality
remote interactive surgery platform
telementoring

Access to Document

10.3390/jimaging9030056

Cite this

@article{cc1f1dc05e434d0f80995c793293a683,

title = "Remote Interactive Surgery Platform (RISP): Proof of Concept for an Augmented-Reality-Based Platform for Surgical Telementoring",

abstract = "The {\textquotedblleft}Remote Interactive Surgery Platform{\textquotedblright} (RISP) is an augmented reality (AR)-based platform for surgical telementoring. It builds upon recent advances of mixed reality head-mounted displays (MR-HMD) and associated immersive visualization technologies to assist the surgeon during an operation. It enables an interactive, real-time collaboration with a remote consultant by sharing the operating surgeon{\textquoteright}s field of view through the Microsoft (MS) HoloLens2 (HL2). Development of the RISP started during the Medical Augmented Reality Summer School 2021 and is currently still ongoing. It currently includes features such as three-dimensional annotations, bidirectional voice communication and interactive windows to display radiographs within the sterile field. This manuscript provides an overview of the RISP and preliminary results regarding its annotation accuracy and user experience measured with ten participants.",

keywords = "AR-assisted surgery, HoloLens2, MR-HMD, RISP, augmented reality, remote interactive surgery platform, telementoring",

author = "Yannik Kalbas and Hoijoon Jung and John Ricklin and Ge Jin and Mingjian Li and Thomas Rauer and Shervin Dehghani and Nassir Navab and Jinman Kim and Pape, \{Hans Christoph\} and Heining, \{Sandro Michael\}",

note = "Publisher Copyright: {\textcopyright} 2023 by the authors.",

year = "2023",

month = mar,

doi = "10.3390/jimaging9030056",

language = "English",

volume = "9",

journal = "Journal of Imaging",

issn = "2313-433X",

publisher = "Multidisciplinary Digital Publishing Institute (MDPI)",

number = "3",

}

TY - JOUR

T1 - Remote Interactive Surgery Platform (RISP)

T2 - Proof of Concept for an Augmented-Reality-Based Platform for Surgical Telementoring

AU - Kalbas, Yannik

AU - Jung, Hoijoon

AU - Ricklin, John

AU - Jin, Ge

AU - Li, Mingjian

AU - Rauer, Thomas

AU - Dehghani, Shervin

AU - Navab, Nassir

AU - Kim, Jinman

AU - Pape, Hans Christoph

AU - Heining, Sandro Michael

PY - 2023/3

Y1 - 2023/3

N2 - The “Remote Interactive Surgery Platform” (RISP) is an augmented reality (AR)-based platform for surgical telementoring. It builds upon recent advances of mixed reality head-mounted displays (MR-HMD) and associated immersive visualization technologies to assist the surgeon during an operation. It enables an interactive, real-time collaboration with a remote consultant by sharing the operating surgeon’s field of view through the Microsoft (MS) HoloLens2 (HL2). Development of the RISP started during the Medical Augmented Reality Summer School 2021 and is currently still ongoing. It currently includes features such as three-dimensional annotations, bidirectional voice communication and interactive windows to display radiographs within the sterile field. This manuscript provides an overview of the RISP and preliminary results regarding its annotation accuracy and user experience measured with ten participants.

AB - The “Remote Interactive Surgery Platform” (RISP) is an augmented reality (AR)-based platform for surgical telementoring. It builds upon recent advances of mixed reality head-mounted displays (MR-HMD) and associated immersive visualization technologies to assist the surgeon during an operation. It enables an interactive, real-time collaboration with a remote consultant by sharing the operating surgeon’s field of view through the Microsoft (MS) HoloLens2 (HL2). Development of the RISP started during the Medical Augmented Reality Summer School 2021 and is currently still ongoing. It currently includes features such as three-dimensional annotations, bidirectional voice communication and interactive windows to display radiographs within the sterile field. This manuscript provides an overview of the RISP and preliminary results regarding its annotation accuracy and user experience measured with ten participants.

KW - AR-assisted surgery

KW - HoloLens2

KW - MR-HMD

KW - RISP

KW - augmented reality

KW - remote interactive surgery platform

KW - telementoring

UR - https://www.scopus.com/pages/publications/85150961023

U2 - 10.3390/jimaging9030056

DO - 10.3390/jimaging9030056

M3 - Article

AN - SCOPUS:85150961023

SN - 2313-433X

VL - 9

JO - Journal of Imaging

JF - Journal of Imaging

IS - 3

M1 - 56

ER -

Remote Interactive Surgery Platform (RISP): Proof of Concept for an Augmented-Reality-Based Platform for Surgical Telementoring

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this