DDS Implementations as Real-Time Middleware - A Systematic Evaluation

Publish-subscribe messaging has seen increased adoption in the context of timing critical applications, with multiple frameworks integrating publish-subscribe middleware into their ecosystem. The Data Distribution Service (DDS) is a standard for pub-sub systems that has gained traction, through e.g. the adoption in ROS2, with multiple vendors distributing their implementations of the standard. However, while DDS is being used in a real-time context, the examined implementations are not strictly real-time capable and therefore cannot provide hard timing guarantees to the application. Still, users are looking to take advantage of the flexibility offered by DDS in close-to real-time use cases, raising the question of how well DDS implementations can provide soft real-time reliability assurances. We use DDS-Perf, a novel cross-vendor benchmarking tool for impartial performance analysis of DDS implementations, to examine how reliably four vendors (OpenDDS, RTI Connext, Fast-DDS and CycloneDDS) can deliver real-time-like performance under different scenarios. From a typical out-of-the-box setup, we offer a guide to users for tuning performance/reliability and examine problems users might encounter trying to satisfy real-time constraints. The vendor implementations are tested against a range of experiments to evaluate operating performance under favorable and adverse conditions. Overall, we find that OpenDDS has the worst performance out-of-the-box and after calibration, while FastDDS and CycloneDDS offer the best performance, which is comparable to user-space networking technologies in the average case but with a much higher worst-case latency bound.