Hierarchical Reinforcement Learning for Waypoint-based Exploration in Robotic Devices

Jonas Zinn, Birgit Vogel-Heuser, Fabian Schuhmann, Luis Alberto Cruz Salazar

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

4 Scopus citations

Abstract

The training of Deep Reinforcement Learning algorithms on robotic devices is challenging due to their large number of actuators and limited number of feasible action sequences. This paper addresses this challenge by extending and transferring existing approaches for waypoint-based exploration with Hierarchical Reinforcement Learning to the domain of robotic devices. The resulting algorithm utilizes a top-level policy, which suggests waypoints to a bottom-level policy that controls the system actuators. The waypoints can either be provided to the top-level policy as domain knowledge or be learned from scratch. The algorithm explicitly accounts for the low number of feasible waypoints and waypoint transitions that are characteristic of robotic devices. The effectiveness of the approach is evaluated on the simulation of a research demonstrator, and a separate ablation study proves the importance of its components.

Original languageEnglish
Title of host publicationProceedings - 2021 IEEE 19th International Conference on Industrial Informatics, INDIN 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728143958
DOIs
StatePublished - 2021
Event19th IEEE International Conference on Industrial Informatics, INDIN 2021 - Mallorca, Spain
Duration: 21 Jul 202123 Jul 2021

Publication series

NameIEEE International Conference on Industrial Informatics (INDIN)
Volume2021-July
ISSN (Print)1935-4576

Conference

Conference19th IEEE International Conference on Industrial Informatics, INDIN 2021
Country/TerritorySpain
CityMallorca
Period21/07/2123/07/21

Keywords

  • Fault tolerant design
  • Intelligent systems
  • Mechatronics and robotics

Fingerprint

Dive into the research topics of 'Hierarchical Reinforcement Learning for Waypoint-based Exploration in Robotic Devices'. Together they form a unique fingerprint.

Cite this