TY - GEN
T1 - Comparing Objective and Subjective Measures of Usability in a Human-Robot Dialogue System
AU - Foster, Mary Ellen
AU - Giuliani, Manuel
AU - Knoll, Alois
N1 - Publisher Copyright:
© 2009 ACL and AFNLP.
PY - 2009
Y1 - 2009
N2 - We present a human-robot dialogue system that enables a robot to work together with a human user to build wooden construction toys. We then describe a study in which naïve subjects interacted with this system under a range of conditions and then completed a user-satisfaction questionnaire. The results of this study provide a wide range of subjective and objective measures of the quality of the interactions. To assess which aspects of the interaction had the greatest impact on the users’ opinions of the system, we used a method based on the PARADISE evaluation framework (Walker et al., 1997) to derive a performance function from our data. The major contributors to user satisfaction were the number of repetition requests (which had a negative effect on satisfaction), the dialogue length, and the users’ recall of the system instructions (both of which contributed positively).
AB - We present a human-robot dialogue system that enables a robot to work together with a human user to build wooden construction toys. We then describe a study in which naïve subjects interacted with this system under a range of conditions and then completed a user-satisfaction questionnaire. The results of this study provide a wide range of subjective and objective measures of the quality of the interactions. To assess which aspects of the interaction had the greatest impact on the users’ opinions of the system, we used a method based on the PARADISE evaluation framework (Walker et al., 1997) to derive a performance function from our data. The major contributors to user satisfaction were the number of repetition requests (which had a negative effect on satisfaction), the dialogue length, and the users’ recall of the system instructions (both of which contributed positively).
UR - http://www.scopus.com/inward/record.url?scp=85130574008&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85130574008
SN - 9781617382581
T3 - ACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf.
SP - 879
EP - 887
BT - ACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf.
PB - Association for Computational Linguistics (ACL)
T2 - Joint Conference of the 47th Annual Meeting of the Association for Computational Linguistics and 4th International Joint Conference on Natural Language Processing of the AFNLP, ACL-IJCNLP 2009
Y2 - 2 August 2009 through 7 August 2009
ER -