Learning a Low-Dimensional Representation of a Safe Region for Safe Reinforcement Learning on Dynamical Systems

Zhehua Zhou, Ozgur S. Oguz, Marion Leibold, Martin Buss

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

For the safe application of reinforcement learning algorithms to high-dimensional nonlinear dynamical systems, a simplified system model is used to formulate a safe reinforcement learning (SRL) framework. Based on the simplified system model, a low-dimensional representation of the safe region is identified and used to provide safety estimates for learning algorithms. However, finding a satisfying simplified system model for complex dynamical systems usually requires a considerable amount of effort. To overcome this limitation, we propose a general data-driven approach that is able to efficiently learn a low-dimensional representation of the safe region. By employing an online adaptation method, the low-dimensional representation is updated using the feedback data to obtain more accurate safety estimates. The performance of the proposed approach for identifying the low-dimensional representation of the safe region is illustrated using the example of a quadcopter. The results demonstrate a more reliable and representative low-dimensional representation of the safe region compared with previous works, which extends the applicability of the SRL framework.

Original languageEnglish
Pages (from-to)2513-2527
Number of pages15
JournalIEEE Transactions on Neural Networks and Learning Systems
Volume34
Issue number5
DOIs
StatePublished - 1 May 2023

Keywords

  • Data-driven model order reduction
  • deep learning in robotics and automation
  • learning and adaptive systems
  • safe reinforcement learning (SRL)

Fingerprint

Dive into the research topics of 'Learning a Low-Dimensional Representation of a Safe Region for Safe Reinforcement Learning on Dynamical Systems'. Together they form a unique fingerprint.

Cite this