QoS-Aware Dynamic Frequency Scaling for Mixed-Critical Systems based on Shielded Reinforcement Learning

In recent years, several approaches have proposed dynamic voltage-frequency scaling (DVFS) controllers for power management in multiprocessor system-on-chips based on reinforcement learning (RL) to cope with changing system dynamics. Those implemented in software cannot respond to short-term changes in processor load, which occur at a sub-millisecond scale. On the other hand, modern processors contain various built-in hardware features, like prefetchers, reacting to short-term variations to improve system performance. Previously, we integrated such an RL-based DVFS algorithm into the hardware and applied it to optimize the performance of best-effort tasks. In this work, we augment it with preemptive shielding and eligibility traces to employ it on deadline-bound quality of service (QoS) tasks. The algorithm takes advantage of short-term variations to minimize power usage while ensuring the application's QoS. Additionally, our algorithm complies with the power constraints necessary in mixed-critical systems unlike state-of-the-art algorithms. We demonstrate the features of our algorithm in a hardware-in-the-loop simulation by running LLVM's single-source benchmarks on SparcV8 processors.