Safe Reinforcement Learning via Confidence-Based Filters

Sebastian Curi, Armin Lederer, Sandra Hirche, Andreas Krause

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations


Ensuring safety is a crucial challenge when deploying reinforcement learning (RL) to real-world systems. We develop confidence-based safety filters, a control-theoretic approach for certifying state safety constraints for nominal policies learnt via standard RL techniques, based on probabilistic dynamics models. Our approach is based on a reformulation of state constraints in terms of cost functions, reducing safety verification to a standard RL task. By exploiting the concept of hallucinating inputs, we extend this formulation to determine a "backup"policy which is safe for the unknown system with high probability. The nominal policy is minimally adjusted at every time step during a roll-out towards the backup policy, such that safe recovery can be guaranteed afterwards. We provide formal safety guarantees, and empirically demonstrate the effectiveness of our approach.

Original languageEnglish
Title of host publication2022 IEEE 61st Conference on Decision and Control, CDC 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages7
ISBN (Electronic)9781665467612
StatePublished - 2022
Event61st IEEE Conference on Decision and Control, CDC 2022 - Cancun, Mexico
Duration: 6 Dec 20229 Dec 2022

Publication series

NameProceedings of the IEEE Conference on Decision and Control
ISSN (Print)0743-1546
ISSN (Electronic)2576-2370


Conference61st IEEE Conference on Decision and Control, CDC 2022


Dive into the research topics of 'Safe Reinforcement Learning via Confidence-Based Filters'. Together they form a unique fingerprint.

Cite this