Enhancing human action recognition through spatio-temporal feature learning and semantic rules

Karinne Ramirez-Amaro, Eun Sol Kim, Jiseob Kim, Byoung Tak Zhang, Michael Beetz, Gordon Cheng

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

9 Scopus citations

Abstract

In this paper, we present a two-stage framework that deal with the problem of automatically extract human activities from videos. First, for action recognition we employ an unsupervised state-of-the-art learning algorithm based on Independent Subspace Analysis (ISA). This learning algorithm extracts spatio-temporal features directly from video data and it is computationally more efficient and robust than other unsupervised methods. Nevertheless, when applying this one-stage state-of-the-art action recognition technique on the observations of human everyday activities, it can only reach an accuracy rate of approximately 25%. Hence, we propose to enhance this process with a second stage, which define a new method to automatically generate semantic rules that can reason about human activities. The obtained semantic rules enhance the human activity recognition by reducing the complexity of the perception system and they allow the possibility of domain change, which can great improve the synthesis of robot behaviors. The proposed method was evaluated under two complex and challenging scenarios: making a pancake and making a sandwich. The difficulty of these scenarios is that they contain finer and more complex activities than the well known data sets (Hollywood2, KTH, etc). The results show benefits of two stages method, the accuracy of action recognition was significantly improved compared to a single-stage method (above 87% compared to human expert). This indicates the improvement of the framework using the reasoning engine for the automatic extraction of human activities from observations, thus, providing a rich mechanism for transferring a wide range of human skills to humanoid robots.

Original languageEnglish
Title of host publication2013 13th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2013
PublisherIEEE Computer Society
Pages456-461
Number of pages6
EditionFebruary
ISBN (Electronic)9781479926176
DOIs
StatePublished - 3 Feb 2015
Event2013 13th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2013 - Atlanta, United States
Duration: 15 Oct 201317 Oct 2013

Publication series

NameIEEE-RAS International Conference on Humanoid Robots
NumberFebruary
Volume2015-February
ISSN (Print)2164-0572
ISSN (Electronic)2164-0580

Conference

Conference2013 13th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2013
Country/TerritoryUnited States
CityAtlanta
Period15/10/1317/10/13

Fingerprint

Dive into the research topics of 'Enhancing human action recognition through spatio-temporal feature learning and semantic rules'. Together they form a unique fingerprint.

Cite this