Modular Production Control with Multi-Agent Deep Q-Learning

The automotive industry is increasingly focusing on product customization. The concept of Modular Production addresses this issue by providing more flexibility in production with Automated Guided Vehicles transporting products between modular workstations. The added complexity of Modular Production Control calls for approaches that can handle the scheduling complexity while also minimizing production costs. As a result, literature has focused on two promising approaches: Deep Reinforcement Learning and Multi-Agent Systems. Both approaches have their advantages. Especially in complex, large-scale production environments with random breakdowns, those two fields have been seldomly combined, though. As a result, this article aims to fill that research gap by introducing a Deep Reinforcement Learning Multi-Agent System approach for Modular Production Control. We introduce a reward design incentivizing agents to achieve maximal throughput. In addition, we show that the method learns optimal behavior even in a large-scale production environment with random machine breakdowns. Lastly, we compare the Multi-Agent System to a single-agent implementation of the Deep Reinforcement Learning approach and conclude that the Multi-Agent Deep Reinforcement Learning method learns and solves the Modular Production Control problem with the same solution quality as the single agent. Hence, the approach allows to foster MAS benefits such as robustness without losses in the solution quality.