Deep Active Cross-Modal Visuo-Tactile Transfer Learning for Robotic Object Recognition

Prajval Kumar Murali, Cong Wang, Dongheui Lee, Ravinder Dahiya, Mohsen Kaboli

Research output: Contribution to journalArticlepeer-review

14 Scopus citations

Abstract

We propose for the first time, a novel deep active visuo-tactile cross-modal full-fledged framework for object recognition by autonomous robotic systems. Our proposed network xAVTNet is actively trained with labelled point clouds from a vision sensor with one robot and tested with an active tactile perception strategy to recognise objects never touched before using another robot. We propose a novel visuo-tactile loss (VTLoss) to minimise the discrepancy between the visual and tactile domains for unsupervised domain adaptation. Our framework leverages the strengths of deep neural networks for cross-modal recognition along with active perception and active learning strategies for increased efficiency by minimising redundant data collection. Our method is extensively evaluated on a real robotic system and compared against baselines and other state-of-art approaches. We demonstrate clear outperformance in recognition accuracy compared to the state-of-art visuo-tactile cross-modal recognition method.

Original languageEnglish
Pages (from-to)9557-9564
Number of pages8
JournalIEEE Robotics and Automation Letters
Volume7
Issue number4
DOIs
StatePublished - 1 Oct 2022
Externally publishedYes

Keywords

  • Active visuo-tactile object recognition
  • perception for grasping and manipulation
  • transfer learning
  • visuo-tactile cross-modal learning

Fingerprint

Dive into the research topics of 'Deep Active Cross-Modal Visuo-Tactile Transfer Learning for Robotic Object Recognition'. Together they form a unique fingerprint.

Cite this