HOIsim: Synthesizing realistic 3D human-object interaction data for human activity recognition

Marsil Zakour, Alaeddine Mellouli, Rahul Chaudhari

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

6 Scopus citations

Abstract

Correct understanding of human activities is critical for meaningful assistance by robots in daily life. The development of perception algorithms and Deep Learning models of human activity requires large-scale sensor datasets. Good real-world activity data is, however, difficult and time- consuming to acquire. Several precisely calibrated and time- synchronized sensors are required, and the annotation and labeling of the collected sensor data is extremely labor intensive.To address these challenges, we present a 3D activity simulator, "HOIsim", focusing on Human-Object Interactions (HOIs). Using HOIsim, we provide a procedurally generated synthetic dataset of two sample daily life activities "lunch"and "breakfast". The dataset contains out-of-the-box ground truth annotations in the form of human and object poses, as well as ground truth activity labels. Furthermore, we introduce methods to meaningfully randomize activity flows and the environment topology. This allows us to generate a large number of random variants of these activities in very less time.Based on an abstraction of the low-level pose data in the form of spatiotemporal graphs of HOIs, we evaluate the generated Lunch dataset only with two Deep Learning models for activity recognition. The first model, based on recurrent neural networks achieves an accuracy of 87%, whereas the other, based on transformers, achieves an accuracy of 94.7%.

Original languageEnglish
Title of host publication2021 30th IEEE International Conference on Robot and Human Interactive Communication, RO-MAN 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1124-1131
Number of pages8
ISBN (Electronic)9781665404921
DOIs
StatePublished - 8 Aug 2021
Event30th IEEE International Conference on Robot and Human Interactive Communication, RO-MAN 2021 - Virtual, Vancouver, Canada
Duration: 8 Aug 202112 Aug 2021

Publication series

Name2021 30th IEEE International Conference on Robot and Human Interactive Communication, RO-MAN 2021

Conference

Conference30th IEEE International Conference on Robot and Human Interactive Communication, RO-MAN 2021
Country/TerritoryCanada
CityVirtual, Vancouver
Period8/08/2112/08/21

Fingerprint

Dive into the research topics of 'HOIsim: Synthesizing realistic 3D human-object interaction data for human activity recognition'. Together they form a unique fingerprint.

Cite this