Deep Q-learning for the Control of PLC-based Automated Production Systems

Jonas Zinn, Birgit Vogel-Heuser, Paulina Ockier

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

6 Scopus citations

Abstract

This paper evaluates the use of Deep Reinforcement Learning to control Programmable Logic Controller-based automated Production Systems, which are characterized by multiple end-effectors that are actuated in only one or two axes. Due to the large number of actuators of which only a few affect the processing of a workpiece at a given time, these systems are challenging to learn. In this paper, Deep Q-learning is applied to a small use case focusing on sorting workpieces by color in a simulation of such a production system. The basic algorithm is hereby compared to four commonly used extensions: Double Q-learning, Dueling Networks, Prioritized Experience Replay, and Hindsight Experience Replay. For the scope of this paper, simplifications are applied to the state and action space. While the baseline implementation of Deep Q-learning is able to correctly sort 30 previously seen workpiece combinations, it does not reliably generalize to unseen ones within 45,000 training episodes. In contrast, the algorithm using all four considered extensions is able to reliably generalize to all 81 possible workpiece combinations.

Original languageEnglish
Title of host publication2020 IEEE 16th International Conference on Automation Science and Engineering, CASE 2020
PublisherIEEE Computer Society
Pages1434-1440
Number of pages7
ISBN (Electronic)9781728169040
DOIs
StatePublished - Aug 2020
Event16th IEEE International Conference on Automation Science and Engineering, CASE 2020 - Hong Kong, Hong Kong
Duration: 20 Aug 202021 Aug 2020

Publication series

NameIEEE International Conference on Automation Science and Engineering
Volume2020-August
ISSN (Print)2161-8070
ISSN (Electronic)2161-8089

Conference

Conference16th IEEE International Conference on Automation Science and Engineering, CASE 2020
Country/TerritoryHong Kong
CityHong Kong
Period20/08/2021/08/20

Fingerprint

Dive into the research topics of 'Deep Q-learning for the Control of PLC-based Automated Production Systems'. Together they form a unique fingerprint.

Cite this