SceneDiffusion: Conditioned Latent Diffusion Models for Traffic Scene Prediction

Predicting the future motion of traffic participants is one of the crucial topics to be addressed for safe autonomous driving. Deep learning methods have shown remarkable success in recent years for the task of scene prediction. Most of the work considers the scene prediction problem as a classification and regression tasks. In contrast to such approaches, in this work, it is shown how conditional latent diffusion with a temporal constraint can be used for scene prediction. This is one of the first works to use latent diffusion with a temporal constraint for the purpose of predicting the motion of vehicles in a traffic scenario. The main goal is to show what architectural changes are necessary in order to use latent diffusion models with a temporal constraint to address the challenge of scene prediction. A major advantage of using the proposed architecture for scene prediction is the possibility to extend the temporal constraint with spacial constraints, such as goal points, acceleration conditions, etc. The proposed scene diffusion model can be used in the conditional mode as a scene predictor and in the unconditional mode as a scene initialiser. The experiments show that diffusion models are a promising method to tackle the challenges of scene prediction.