Measuring auditory attention effort in virtual audio-visual environments using pupillometry

Bernhard U. Seeber, Tim Wackler

Research output: Contribution to journalConference articlepeer-review

Abstract

Pupillometry is a common approach to measure cognitive effort due to attending auditory targets in noise. The enhanced attention to sound sources required in difficult listening situations is associated with cognitive effort, which is linked with an increased pupil diameter. However, pupil diameter is far more affected by the luminance of the focused visual object. Hence, pupillometry is done with low illumination or by focusing on a region with constant luminance. In ecologically valid audio-visual environments, visual objects appear interactively, dynamically in view and luminance changes constantly. We present a method that compensates for these 'environmental luminance effects' such that the measurement of attention through pupil dilation is possible. Using a head-mounted world-camera and an eye-tracker, luminance in the focused area is measured. A system of differential equations is solved to dynamically predict a baseline pupil diameter as a function of momentary luminance. The approach is verified in a listening experiment with cued attention to one of two spatialized talkers and varying difficulty, while luminance on a video projection screen is varied. Results show that the large effect of luminance on pupil diameter can predicted with sufficient accuracy in order to measure the residual small effect of attentional effort. This paves the way for dynamic attention measurements in virtual communication scenarios.

Original languageEnglish
JournalProceedings of the International Congress on Acoustics
StatePublished - 2022
Event24th International Congress on Acoustics, ICA 2022 - Gyeongju, Korea, Republic of
Duration: 24 Oct 202228 Oct 2022

Keywords

  • communication
  • eye-tracking
  • Spatial attention

Fingerprint

Dive into the research topics of 'Measuring auditory attention effort in virtual audio-visual environments using pupillometry'. Together they form a unique fingerprint.

Cite this