NCSbench: Reproducible Benchmarking Platform for Networked Control Systems

The evolution of the Internet of Things accelerated the development of Cyber-Physical Systems. Among them, Networked Control Systems (NCS) gained notable attention thanks to their application to industrial operations. Experimental NCS require expertise from control, computation, and communication disciplines. This requirement, together with the fragmentation of implementation platforms and experimental investigations, represents a challenge for the reproducibility and comparison of research results. In this paper, we tackle this problem by proposing a novel NCS benchmarking methodology that aids the reproducibility of NCS experiments. Relying on a novel approach to model the architectural elements and the delays of NCS, the methodology defines the experiment parameters and the relevant Key Performance Indicators (KPIs) that need to be observed during its execution. Furthermore, we detail the implementation of the first reproducible benchmarking platform for NCS. The proposed platform is open-source and designed to be easily reproducible and extensible by anyone. Finally, we replicate and evaluate the platform following the proposed NCS benchmarking methodology. The experimental results evaluate and compare the KPIs during the execution of the platform in different benchmarking scenarios, proving the validity of the proposed benchmarking methodology.