Spatially-Aware Displays for Computer Assisted Interventions

We present a novel display and visual interaction paradigm, which aims at reducing the complexity of understanding the spatial transformations between the surgeon’s viewpoint, the patient, the pre- or intra-operative 2D and 3D data, and surgical tools during computer assisted interventions. To the best of our knowledge, this is the first work in which the traditional interventional display, for example in surgical navigation systems, is registered both to the patient and to the surgeon’s view. The closest concept is that of traditional Augmented Reality windows in which a semitransparent or video see-through display is positioned between the surgeon and the patient. In such cases, the system was providing an AR view into the patient. In the new concept introduced here, the surgeon keeps his/her own direct view to the patient without any need for additional display or direct view augmentation, but the monitor used in the operating room is now registered to the patient and surgeon’s viewpoint. The display could act as fixed viewing frustum or as a mirror frustum relative to the surgeon’s view. This allows the physicians to effortlessly relate their view of tools and patient to the virtual representation of the patient data. In this paper, the first realization and implementation of such a concept is presented and three clinical partners have tested the system and their first feedback is discussed in detail. They unanimously believe that this concept opens the path for facilitating interactive exploration of data and more intuitive navigation guidance in computer assisted interventions.