Spatiotemporal representation of driving scenarios and classification using neural networks

Large scale fleet tests of autonomous vehicles lead to the availability of massive recorded datasets, offering significant potential for the generation of realistic virtual test drives, for the development and training of machine learning based functions, and facilitated performance analysis. Automated scenario classification and data labeling is necessary to maximize the utility of these massive datasets and make them fully accessible and searchable for developers. In this paper we present and compare several spatiotemporal representations of recorded driving scenarios and analyze the impact of the representation type on the results of the subsequent automated scenario classification with deep neural networks. Built on a fused list of objects that combines data from several sensor types, we create and annotate datasets for each of the representations and train the classification algorithm. The best classification results were achieved with the presented Stacked Velocity Grid, which includes temporal information.