A method to introduce evaluate motion parallax with stereo for medical AR/MR

Megha Kalia, Nassir Navab, Sidney Fels, Tim Salcudean

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Scopus citations

Abstract

Incorrect depth perception and lack of good evaluation systems are major barriers in clinical translation of augmented and mixed reality AR/MR. Thus, a systematic study of depth cues is necessary. Therefore, in the current paper we present a method to introduce the quantitative depth cue Motion Parallax (MP) in surgical scenes and study its effect on depth perception when combined with binocular disparity. In addition to this we present an innovative virtual tool method to evaluate depth. To introduce MP, we reconstructed the tissue surface using structure from motion technique. Then to get accurate absolute scale of the reconstructed surface stereo-triangulation was used. The simulated tumor was rendered beneath the reconstructed point-cloud by rendering a hole for 'X-ray' like vision. The MP was introduced by rotating the entire scene from side-to-side with a tumor-surface-point as pivot for maximum impact. Finally for evaluation, we used a virtual surgical tool rendered using real-time da Vinci surgical API's forward kinematics data. In total, 12 subjects participated in a within-subjects-experiment design to study four cases, i.e., Stereo + MP (\text{S}+\text{MP}), Mono + MP (\text{M}+\text{MP}), Stereo + No MP (\text{S}+\text{N}-\text{MP}) and Mono + No MP (\text{M}+\text{N}-\text{MP}). The subjects significantly overestimated Judged Percentage of True Distance in \text{M}+\text{MP} when compared to \text{M}+\text{N}-\text{MP} (probability (\text{p})=0.000, Number of Samples (\text{N}))=120) and \text{S}+\text{N}-\text{MP} cases (\text{p}=0.001, \text{N}= 120). Furthermore, the observed VariableError was less in \text{S}+\text{MP} and \text{S}+\text{N}-\text{MP} cases when compared to \text{M}+\text{MP} and \text{M}+\text{N}-\text{MP} cases. The use of Motion Parallax in console interfaces for surgical robotics showed overestimation of judged distance. But to our knowledge it is the first work studying the effect of motion parallax and stereo in the surgical context. Therefore, its further study is warranted.

Original languageEnglish
Title of host publication26th IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1755-1759
Number of pages5
ISBN (Electronic)9781728113777
DOIs
StatePublished - Mar 2019
Event26th IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019 - Osaka, Japan
Duration: 23 Mar 201927 Mar 2019

Publication series

Name26th IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019 - Proceedings

Conference

Conference26th IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019
Country/TerritoryJapan
CityOsaka
Period23/03/1927/03/19

Keywords

  • Augmented reality
  • Depth perception
  • Evaluation method
  • Medical augmented reality
  • Mixed reality
  • Motion parallax

Fingerprint

Dive into the research topics of 'A method to introduce evaluate motion parallax with stereo for medical AR/MR'. Together they form a unique fingerprint.

Cite this