Unlocking Past Information: Temporal Embeddings in Cooperative Bird's Eye View Prediction

Dominik Rößle, Jeremias Gerner, Klaus Bogenberger, Daniel Cremers, Stefanie Schmidtner, Torsten Schön

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Accurate and comprehensive Bird's Eye View (BEV) semantic segmentation is essential for ensuring safe and proactive navigation in autonomous driving. Although cooperative perception has exceeded the detection capabilities of single-agent systems, prevalent camera-based algorithms in cooperative perception neglect valuable information derived from historical observations. This limitation becomes critical during sensor failures or communication issues as cooperative perception reverts to single-agent perception, leading to degraded performance and incomplete BEV segmentation maps. This paper introduces TempCoBEV, a temporal module designed to incorporate historical cues into current observations, thereby improving the quality and reliability of BEV map segmentations. We propose an importance-guided attention architecture to effectively integrate temporal information that prioritizes relevant properties for BEV map segmentation. TempCoBEV is an independent temporal module that seamlessly integrates into state-of-the-art camera-based cooperative perception models. We demonstrate through extensive experiments on the OPV2V dataset that TempCoBEV performs better than non-temporal models in predicting current and future BEV map segmentations, particularly in scenarios involving communication failures. We show the efficacy of TempCoBEV and its capability to integrate historical cues into the current BEV map, improving predictions under optimal communication conditions by up to 2% and under communication failures by up to 19%. The code is available at https://github.com/cvims/TempCoBEV.

Original languageEnglish
Title of host publication35th IEEE Intelligent Vehicles Symposium, IV 2024
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2220-2225
Number of pages6
ISBN (Electronic)9798350348811
DOIs
StatePublished - 2024
Event35th IEEE Intelligent Vehicles Symposium, IV 2024 - Jeju Island, Korea, Republic of
Duration: 2 Jun 20245 Jun 2024

Publication series

NameIEEE Intelligent Vehicles Symposium, Proceedings
ISSN (Print)1931-0587
ISSN (Electronic)2642-7214

Conference

Conference35th IEEE Intelligent Vehicles Symposium, IV 2024
Country/TerritoryKorea, Republic of
CityJeju Island
Period2/06/245/06/24

Fingerprint

Dive into the research topics of 'Unlocking Past Information: Temporal Embeddings in Cooperative Bird's Eye View Prediction'. Together they form a unique fingerprint.

Cite this