Zero-Shot Segmentation of Eye Features Using the Segment Anything Model (SAM)

Virmarie Maquiling, Sean Anthony Byrne, Diederick C. Niehorster, Marcus Nyström, Enkelejda Kasneci

Publikation: Beitrag in FachzeitschriftArtikelBegutachtung

1 Zitat (Scopus)

Abstract

The advent of foundation models signals a new era in artificial intelligence. The Segment Anything Model (SAM) is the first foundation model for image segmentation. In this study, we evaluate SAM's ability to segment features from eye images recorded in virtual reality setups. The increasing requirement for annotated eye-image datasets presents a significant opportunity for SAM to redefine the landscape of data annotation in gaze estimation. Our investigation centers on SAM's zero-shot learning abilities and the effectiveness of prompts like bounding boxes or point clicks. Our results are consistent with studies in other domains, demonstrating that SAM's segmentation effectiveness can be on-par with specialized models depending on the feature, with prompts improving its performance, evidenced by an IoU of 93.34% for pupil segmentation in one dataset. Foundation models like SAM could revolutionize gaze estimation by enabling quick and easy image segmentation, reducing reliance on specialized models and extensive manual annotation.

OriginalspracheEnglisch
Aufsatznummer26
FachzeitschriftProceedings of the ACM on Computer Graphics and Interactive Techniques
Jahrgang7
Ausgabenummer2
DOIs
PublikationsstatusVeröffentlicht - 17 Mai 2024

Fingerprint

Untersuchen Sie die Forschungsthemen von „Zero-Shot Segmentation of Eye Features Using the Segment Anything Model (SAM)“. Zusammen bilden sie einen einzigartigen Fingerprint.

Dieses zitieren