Event analysis and interpretation of human activity for augmented reality-based assistant systems

In this paper a concept and its implementation of an ergonomic cognitive assistant system for supporting human workers at complex assembly tasks in industrial environments is introduced. Depending on the level of the user's product knowledge this mixed-initiative system follows and gains knowledge from the human worker's construction steps while it is also able to automatically give hints and instruct the worker whenever needed. The presented agent bases on a closed human-machine interaction loop consisting of the multimodal perception of the worker's action, the comparison with the system's knowledge about the production task, and the displaying of the adequate next assembly instruction step. First experimental results of the assistant system are demonstrated on a simplified use case with the construction of a small toy car using augmented reality display techniques.