General recognition models capable of integrating multiple sensors for different domains

Allowing robots to recognize activities through different sensors and re-using its previous experiences is a prominent way to program robots. For this, a recognition method needs to be proposed such that is transferable toward different domains independently of the used input sources. One key component for such generalization is the definition of common representations. In this paper, we present a flexible system to extract symbolic representations of the perceived scenario which adapts to different sensors, such as cameras, multi-modal skin, and robot joint data. These symbolic representations are used to generate a semantic reasoning engine to transfer the obtained models among different domains. To validate our system, first, our robot learns basic activities from observing a video for the task cutting bread. The extracted symbolic representations are later used as previous experiences to the robot, to allow on-line segmentation and recognition of the Kinesthetically demonstrated activities for the new packing oranges scenario with an average accuracy of 83%, thus demonstrating the generalization of our method.