Misbehaviour prediction for autonomous driving systems

Andrea Stocco, Michael Weiss, Marco Calzana, Paolo Tonella

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

92 Scopus citations


Deep Neural Networks (DNNs) are the core component of modern autonomous driving systems. To date, it is still unrealistic that a DNN will generalize correctly to all driving conditions. Current testing techniques consist of offline solutions that identify adversarial or corner cases for improving the training phase. In this paper, we address the problem of estimating the confidence of DNNs in response to unexpected execution contexts with the purpose of predicting potential safety-critical misbehaviours and enabling online healing of DNN-based vehicles. Our approach SelfOracle is based on a novel concept of self-assessment oracle, which monitors the DNN confidence at runtime, to predict unsupported driving scenarios in advance. SelfOracle uses autoencoderand time series-based anomaly detection to reconstruct the driving scenarios seen by the car, and to determine the confidence boundary between normal and unsupported conditions. In our empirical assessment, we evaluated the effectiveness of different variants of SelfOracle at predicting injected anomalous driving contexts, using DNN models and simulation environment from Udacity. Results show that, overall, SelfOracle can predict 77% misbehaviours, up to six seconds in advance, outperforming the online input validation approach of DeepRoad.

Original languageEnglish
Title of host publicationProceedings - 2020 ACM/IEEE 42nd International Conference on Software Engineering, ICSE 2020
PublisherIEEE Computer Society
Number of pages13
ISBN (Electronic)9781450371216
StatePublished - 27 Jun 2020
Externally publishedYes
Event42nd ACM/IEEE International Conference on Software Engineering, ICSE 2020 - Virtual, Online, Korea, Republic of
Duration: 27 Jun 202019 Jul 2020

Publication series

NameProceedings - International Conference on Software Engineering
ISSN (Print)0270-5257


Conference42nd ACM/IEEE International Conference on Software Engineering, ICSE 2020
Country/TerritoryKorea, Republic of
CityVirtual, Online


  • Anomaly detection
  • Deep learning
  • Misbehaviour prediction
  • Testing


Dive into the research topics of 'Misbehaviour prediction for autonomous driving systems'. Together they form a unique fingerprint.

Cite this