Enabling Versatility and Dexterity of the Dual-Arm Manipulators: A General Framework toward Universal Cooperative Manipulation

Yi Ren, Zhehua Zhou, Ziwei Xu, Yang Yang, Guangyao Zhai, Marion Leibold, Fenglei Ni, Zhengyou Zhang, Martin Buss, Yu Zheng

Research output: Contribution to journalArticlepeer-review


Grasping and manipulating various kinds of objects cooperatively is the core skill of a dual-arm robot when deployed as an autonomous agent in a human-centered environment. This requires fully exploiting the robot's versatility and dexterity. In this work, we propose a general framework for dual-arm manipulators that contains two correlative modules. The learning-based dexterity-reachability-aware perception module deals with vision-based bimanual grasping. It employs an end-to-end evaluation network and probabilistic modeling of the robot's reachability to deliver feasible and dexterity-optimum grasp pairs for unseen objects. The optimization-based versatility-oriented control module addresses the online cooperative manipulation control by using a hierarchical quadratic programming formulation. Self-collision avoidance and dual-arm manipulability ellipsoid tracking with high reliability and fidelity are simultaneously achieved based on a learned lightweight distance proxy function and a speed-level tracking technique on Riemannian manifold. Intrinsic system safety is guaranteed, and a novel interface for skill transfer is enabled. A long-horizon rearrangement experiment, a bimanual turnover manipulation, and multiple comparative performance evaluation verify the effectiveness of the proposed framework.

Original languageEnglish
Pages (from-to)1-20
Number of pages20
JournalIEEE Transactions on Robotics
StateAccepted/In press - 2024


  • Bimanual grasping
  • Collision avoidance
  • dual-arm manipulators
  • Grasping
  • machine learning
  • Planning
  • Robot kinematics
  • Robot sensing systems
  • Robots
  • self-collision avoidance
  • Task analysis


Dive into the research topics of 'Enabling Versatility and Dexterity of the Dual-Arm Manipulators: A General Framework toward Universal Cooperative Manipulation'. Together they form a unique fingerprint.

Cite this