V-ir-Net: A Novel Neural Network for Pupil and Corneal Reflection Detection trained on Simulated Light Distributions

Virmarie Maquiling, Sean Anthony Byrne, Marcus Nyström, Enkelejda Kasneci, Diederick C. Niehorster

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

Deep learning has shown promise for gaze estimation in Virtual Reality (VR) and other head-mounted applications, but such models are hard to train due to lack of available data. Here we introduce a novel method to train neural networks for gaze estimation using synthetic images that model the light distributions captured in a P-CR setup. We tested our model on a dataset of real eye images from a VR setup, achieving 76% accuracy which is close to the state-of-the-art model which was trained on the dataset itself. The localization error for CRs was 1.56 pixels and 2.02 pixels for the pupil, which is on par with state-of-the-art. Our approach allowed inference on the whole dataset without sacrificing data for model training. Our method provides a cost-efficient and lightweight training alternative, eliminating the need for hand-labeled data. It offers flexible customization, e.g. adapting to different illuminator configurations, with minimal code changes.

Original languageEnglish
Title of host publicationProceedings of the 25th International Conference on Mobile Human-Computer Interaction, MobileHCI 2023 Companion
EditorsAndreas Komninos, Carmen Santoro, Damianos Gavalas, Johannes Schoening, Maristella Matera, Luis A. Leiva
PublisherAssociation for Computing Machinery, Inc
ISBN (Electronic)9781450399241
DOIs
StatePublished - 26 Sep 2023
Event25th International Conference on Mobile Human-Computer Interaction, MobileHCI 2023 Companion - Athens, Greece
Duration: 26 Sep 202329 Sep 2023

Publication series

NameProceedings of the 25th International Conference on Mobile Human-Computer Interaction, MobileHCI 2023 Companion

Conference

Conference25th International Conference on Mobile Human-Computer Interaction, MobileHCI 2023 Companion
Country/TerritoryGreece
CityAthens
Period26/09/2329/09/23

Keywords

  • corneal reflections
  • eye tracking
  • gaze estimation
  • neural networks
  • p-cr
  • simulations

Fingerprint

Dive into the research topics of 'V-ir-Net: A Novel Neural Network for Pupil and Corneal Reflection Detection trained on Simulated Light Distributions'. Together they form a unique fingerprint.

Cite this