Speed up BDD for safety verification in agile development: A partially replicated controlled experiment

Context: Agile development has aroused interest in developing safety-critical systems. Problem statement: However, an increasingly changing architecture makes traditional safety assurance, especially safety analysis and verification, difficult to work. Recently, we proposed STPA-BDD (System-Theoretic Process Analysis-Behaviou Driven Development) for safety analysis and verification in agile development. It showed a good communication effectiveness in a controlled experiment with 44 undergraduate students. However, productivity, test thoroughness and fault detection effectiveness show no statistically significant difference in comparison with UAT (User Acceptance Testing). Objective: In this article, we aim to speed up STPA-BDD through improving productivity, test thoroughness and fault detection effectiveness of STPA-BDD. We develop a semi-automated tool to speed up BDD for safety verification. Method: We evaluate the semi-automated tool concerning productivity, test thoroughness and fault detection effectiveness in a replicated experiment with 11 additional undergraduate students by comparing them with the results from 22 students in the original experiment. Results: The results turn out that productivity is more than 7 times greater, while test thoroughness is more than 1.5 times greater and fault detection effectiveness is more than 2 times greater by using this semi-automated tool than without using it. Conclusion: BDD for safety verification has beed sped up by using this semi-automated tool. Using professionals instead of students as subjects might provide a clearer impact.