Encodji: Encoding gaze data into emoji space for an amusing scanpath classification approach

Wolfgang Fuhl, Efe Bozkir, Benedikt Hosp, Nora Castner, David Geisler, Thiago C. Santini, Enkelejda Kasneci

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

18 Scopus citations

Abstract

To this day, a variety of information has been obtained from human eye movements, which holds an imense potential to understand and classify cognitive processes and states – e.g., through scanpath classification. In this work, we explore the task of scanpath classification through a combination of unsupervised feature learning and convolutional neural networks. As an amusement factor, we use an Emoji space representation as feature space. This representation is achieved by training generative adversarial networks (GANs) for unpaired scanpath-to-Emoji translation with a cyclic loss. The resulting Emojis are then used to train a convolutional neural network for stimulus prediciton, showing an accuracy improvement of more than five percentual points compared to the same network trained using solely the scanpath data. As a side effect, we also obtain novel unique Emojis representing each unique scanpath. Our goal is to demonstrate the applicability and potential of unsupervised feature learning to scanpath classification in a humorous and entertaining way.

Original languageEnglish
Title of host publicationProceedings - ETRA 2019
Subtitle of host publication2019 ACM Symposium On Eye Tracking Research and Applications
EditorsStephen N. Spencer
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450367097
DOIs
StatePublished - 25 Jun 2019
Externally publishedYes
Event11th ACM Symposium on Eye Tracking Research and Applications, ETRA 2019 - Denver, United States
Duration: 25 Jun 201928 Jun 2019

Publication series

NameEye Tracking Research and Applications Symposium (ETRA)

Conference

Conference11th ACM Symposium on Eye Tracking Research and Applications, ETRA 2019
Country/TerritoryUnited States
CityDenver
Period25/06/1928/06/19

Keywords

  • Emoji
  • Eye Tracking
  • Generative Adversarial Networks
  • Image Generation
  • Scanpath

Fingerprint

Dive into the research topics of 'Encodji: Encoding gaze data into emoji space for an amusing scanpath classification approach'. Together they form a unique fingerprint.

Cite this