TY - GEN
T1 - Unreliable Test Infrastructures in Automotive Testing Setups
AU - Jordan, Claudius
AU - Foth, Philipp
AU - Pretschner, Alexander
AU - Fruth, Matthias
N1 - Publisher Copyright:
© 2022 IEEE.
PY - 2022
Y1 - 2022
N2 - During system testing of automotive electrical control units various reasons can lead to invalid test failures, e.g., non-responding components, faulty simulation models, faulty test case implementations, or hardware or software misconfigurations. To determine whether a test failure is invalid and what the underlying cause was, the test executions have to be analyzed manually, which is tedious and therefore costly. In this work, we report the magnitude of the problem of invalid test failures with four system testing projects from the automotive domain. We find that up to 91% of failed test executions are considered invalid. An oftentimes overlooked challenge are unreliable test infrastructures which deteriorate the validity of the test runs. In the studied projects already between 27% and 53% of failed test executions are linked to unreliable test infrastructures.
AB - During system testing of automotive electrical control units various reasons can lead to invalid test failures, e.g., non-responding components, faulty simulation models, faulty test case implementations, or hardware or software misconfigurations. To determine whether a test failure is invalid and what the underlying cause was, the test executions have to be analyzed manually, which is tedious and therefore costly. In this work, we report the magnitude of the problem of invalid test failures with four system testing projects from the automotive domain. We find that up to 91% of failed test executions are considered invalid. An oftentimes overlooked challenge are unreliable test infrastructures which deteriorate the validity of the test runs. In the studied projects already between 27% and 53% of failed test executions are linked to unreliable test infrastructures.
UR - http://www.scopus.com/inward/record.url?scp=85132822030&partnerID=8YFLogxK
U2 - 10.1109/ICSE-SEIP55303.2022.9793982
DO - 10.1109/ICSE-SEIP55303.2022.9793982
M3 - Conference contribution
AN - SCOPUS:85132822030
T3 - Proceedings - International Conference on Software Engineering
SP - 307
EP - 308
BT - Proceedings - 2022 ACM/IEEE 44th International Conference on Software Engineering
PB - IEEE Computer Society
T2 - 44th ACM/IEEE International Conference on Software Engineering: Software Engineering in Practice, ICSE-SEIP 2022
Y2 - 22 May 2022 through 27 May 2022
ER -