StellaUAV: A Tool for Testing the Safe Behavior of UAVs with Scenario-Based Testing (Tools and Artifact Track)

When we allow Unmanned Aerial Vehicles (UAVs) to perform their missions autonomously in the near future, we need to ensure their safe behavior. To generate relevant test cases that can reveal potential faults in the tested UAVs, we propose to leverage scenario-based testing from the automotive domain. For a systematic application of this methodology, we present StellaUAV, a tool for testing the safe behavior of UAVs with scenario-based testing. With our proposed tool, we can describe relevant test situations, generate test cases for these situations that can reveal potential faults in the tested UAV, and evaluate the performance of different optimization algorithms and their combinations. To demonstrate its applicability, we apply StellaUAV to generate test cases for various situations and discover several safety distance violations of the tested exemplary UAV in the presence of dynamic obstacles. These experimental results indicate that the given system under test can handle situations with only static obstacles rather well, while it encounters problems when facing dynamic ones. Further, we detect that a combination of optimization algorithms can find safety distance violations for a logical scenario that the widely used algorithm NSGAII deemed safe for the tested system. Overall, our results show that StellaUAV can effectively detect potential faults in the tested UAV.