Scenario-based Validation for Autonomous Vehicles with Different Fidelity Levels

To successfully adopt Autonomous Vehicles (AV) in the real world, it is crucial to rigorously evaluate their safety and performance. Simulation plays a vital role in achieving this objective. Simulators, which vary in fidelity levels, are utilized to determine safety-critical scenarios and improve the performance of AVs. In this study, we conduct 15000 simulations to investigate the performance and reliability of different fidelity-level simulations in a desired scenario space defined for an autonomous shuttle in operation. This simulation includes safety evaluation and optimization attempts to enhance vehicle performance. The study further demonstrates the reliability of each fidelity platform's optimization results by running highly realistic simulations. Our study shows that although low-fidelity simulations can be executed faster and with less effort. However, they are unable to uncover hidden details that result in errors in more realistic simulations. Furthermore, the study reveals that optimizations performed within the low-fidelity simulation have limited transferability to high-fidelity simulations.