Evaluate and control service and transaction dependability of complex IoT systems

Observing and controlling the dependability of service provision of complex IoT systems is challenging. In practice, many organizations struggle to derive consumer needs related to quality and to observe and quantify the service provision in the context of the dynamic behavior of a complex distributed system. In this paper, we present an approach to define and evaluate the dependability of complex IoT systems. Our approach is an adaptation of the ISO/IEC 25040, an international standard for the evaluation process for system and software quality, which is part of the systems and software quality requirements and evaluation (SQuaRE) series. Our approach was designed and evaluated with action research in an industrial study at Robert Bosch GmbH. Based on the framework of the SQuaRE series, we integrated different elements of site reliability engineering (SRE) and combined them with distributed tracing as a promising measurement method. Our approach introduces the IoT transaction concept to reduce modeling and observation efforts while increasing operationalization to measure performance against dependability targets. Our adaption was effectively applied, consumer-centricity along different system stakeholders were enhanced, and negative consequences of organizational silos were reduced. This has improved the dependability evaluation of service provision to enable fast feedback cycles for service performance control and improvement.