Robot Policy Improvement With Natural Evolution Strategies for Stable Nonlinear Dynamical System

Yingbai Hu, Guang Chen, Zhijun Li, Alois Knoll

Research output: Contribution to journalArticlepeer-review

33 Scopus citations

Abstract

Robot learning through kinesthetic teaching is a promising way of cloning human behaviors, but it has its limits in the performance of complex tasks with small amounts of data, due to compounding errors. In order to improve the robustness and adaptability of imitation learning, a hierarchical learning strategy is proposed: low-level learning comprises only behavioral cloning with supervised learning, and high-level learning constitutes policy improvement. First, the Gaussian mixture model (GMM)-based dynamical system is formulated to encode a motion from the demonstration. We then derive the sufficient conditions of the GMM parameters that guarantee the global stability of the dynamical system from any initial state, using the Lyapunov stability theorem. Generally, imitation learning should reason about the motion well into the future for a wide range of tasks; it is significant to improve the adaptability of the learning method by policy improvement. Finally, a method based on exponential natural evolution strategies is proposed to optimize the parameters of the dynamical system associated with the stiffness of variable impedance control, in which the exploration noise is subject to stability conditions of the dynamical system in the exploration space, thus guaranteeing the global stability. Empirical evaluations are conducted on manipulators for different scenarios, including motion planning with obstacle avoidance and stiffness learning.

Original languageEnglish
Pages (from-to)4002-4014
Number of pages13
JournalIEEE Transactions on Cybernetics
Volume53
Issue number6
DOIs
StatePublished - 1 Jun 2023

Keywords

  • Dynamical system
  • exponential natural evolution strategies (NESs)
  • imitation learning
  • policy improvement of robustness and adaptability

Fingerprint

Dive into the research topics of 'Robot Policy Improvement With Natural Evolution Strategies for Stable Nonlinear Dynamical System'. Together they form a unique fingerprint.

Cite this