Comparing Objective and Subjective Measures of Usability in a Human-Robot Dialogue System

Mary Ellen Foster, Manuel Giuliani, Alois Knoll

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

19 Scopus citations

Abstract

We present a human-robot dialogue system that enables a robot to work together with a human user to build wooden construction toys. We then describe a study in which naïve subjects interacted with this system under a range of conditions and then completed a user-satisfaction questionnaire. The results of this study provide a wide range of subjective and objective measures of the quality of the interactions. To assess which aspects of the interaction had the greatest impact on the users’ opinions of the system, we used a method based on the PARADISE evaluation framework (Walker et al., 1997) to derive a performance function from our data. The major contributors to user satisfaction were the number of repetition requests (which had a negative effect on satisfaction), the dialogue length, and the users’ recall of the system instructions (both of which contributed positively).

Original languageEnglish
Title of host publicationACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf.
PublisherAssociation for Computational Linguistics (ACL)
Pages879-887
Number of pages9
ISBN (Print)9781617382581
StatePublished - 2009
EventJoint Conference of the 47th Annual Meeting of the Association for Computational Linguistics and 4th International Joint Conference on Natural Language Processing of the AFNLP, ACL-IJCNLP 2009 - Suntec, Singapore
Duration: 2 Aug 20097 Aug 2009

Publication series

NameACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf.

Conference

ConferenceJoint Conference of the 47th Annual Meeting of the Association for Computational Linguistics and 4th International Joint Conference on Natural Language Processing of the AFNLP, ACL-IJCNLP 2009
Country/TerritorySingapore
CitySuntec
Period2/08/097/08/09

Fingerprint

Dive into the research topics of 'Comparing Objective and Subjective Measures of Usability in a Human-Robot Dialogue System'. Together they form a unique fingerprint.

Cite this