Cost optimization at early stages of design using deep reinforcement learning

With the increase in the complexity of the modern System on Chips (SoCs) and the demand for a lower time-to-market, automation becomes essential in hardware design. This is particularly relevant in complex/time-consuming tasks, as the optimization of design cost for a hardware component. Design cost, in fact, may depend on several objectives, as for the hardware-software trade-off. Given the complexity of this task, the designer often has no means to perform a fast and effective optimization-in particular for larger and complex designs. In this paper, we introduce Deep Reinforcement Learning (DRL) for design cost optimization at the early stages of the design process. We first show that DRL is a perfectly suitable solution for the problem at hand. Afterwards, by means of a Pointer Network, a neural network specifically applied for combinatorial problems, we benchmark three DRL algorithms towards the selected problem. Results obtained in different settings show the improvements achieved by DRL algorithms compared to conventional optimization methods. Additionally, by using reward redistribution proposed in the recently introduced RUDDER method, we obtain significant improvements in complex designs.