Situated reference in a hybrid human-robot interaction system

Manuel Giuliani, Mary Ellen Foster, Amy Isard, Colin Matheson, Jon Oberlander, Alois Knoll

Research output: Contribution to conferencePaperpeer-review

16 Scopus citations


We present the situated reference generation module of a hybrid human-robot interaction system that collaborates with a human user in assembling target objects from a wooden toy construction set. The system contains a sub-symbolic goal inference system which is able to detect the goals and errors of humans by analysing their verbal and non-verbal behaviour. The dialogue manager and reference generation components then use situated references to explain the errors to the human users and provide solution strategies. We describe a user study comparing the results from subjects who heard constant references to those who heard references generated by an adaptive process. There was no difference in the objective results across the two groups, but the subjects in the adaptive condition gave higher subjective ratings to the robot's abilities as a conversational partner. An analysis of the objective and subjective results found that the main predictors of subjective user satisfaction were the user's performance at the assembly task and the number of times they had to ask for instructions to be repeated.

Original languageEnglish
Number of pages9
StatePublished - 2010
Event6th International Natural Language Generation Conference, INLG 2010 - Trim, Co. Meath, Ireland
Duration: 7 Jul 20109 Jul 2010


Conference6th International Natural Language Generation Conference, INLG 2010
CityTrim, Co. Meath


Dive into the research topics of 'Situated reference in a hybrid human-robot interaction system'. Together they form a unique fingerprint.

Cite this