NeuroIV: Neuromorphic Vision Meets Intelligent Vehicle Towards Safe Driving with a New Database and Baseline Evaluations

Guang Chen, Fa Wang, Weijun Li, Lin Hong, Jorg Conradt, Jieneng Chen, Zhenyan Zhang, Yiwen Lu, Alois Knoll

Research output: Contribution to journalArticlepeer-review

34 Scopus citations

Abstract

Neuromorphic vision sensors such as the Dynamic and Active-pixel Vision Sensor (DAVIS) using silicon retina are inspired by biological vision, they generate streams of asynchronous events to indicate local log-intensity brightness changes. Their properties of high temporal resolution, low-bandwidth, lightweight computation, and low-latency make them a good fit for many applications of motion perception in the intelligent vehicle. However, as a younger and smaller research field compared to classical computer vision, neuromorphic vision is rarely connected with the intelligent vehicle. For this purpose, we present three novel datasets recorded with DAVIS sensors and depth sensor for the distracted driving research and focus on driver drowsiness detection, driver gaze-zone recognition, and driver hand-gesture recognition. To facilitate the comparison with classical computer vision, we record the RGB, depth and infrared data with a depth sensor simultaneously. The total volume of this dataset has 27360 samples. To unlock the potential of neuromorphic vision on the intelligent vehicle, we utilize three popular event-encoding methods to convert asynchronous event slices to event-frames and adapt state-of-the-art convolutional architectures to extensively evaluate their performances on this dataset. Together with qualitative and quantitative results, this work provides a new database and baseline evaluations named NeuroIV in cross-cutting areas of neuromorphic vision and intelligent vehicle.

Original languageEnglish
Pages (from-to)1171-1183
Number of pages13
JournalIEEE Transactions on Intelligent Transportation Systems
Volume23
Issue number2
DOIs
StatePublished - 1 Feb 2022

Keywords

  • Neuromorphic vision
  • advanced driver assistance system
  • database and baseline evaluations
  • deep learning
  • distracted driving
  • event encoding

Fingerprint

Dive into the research topics of 'NeuroIV: Neuromorphic Vision Meets Intelligent Vehicle Towards Safe Driving with a New Database and Baseline Evaluations'. Together they form a unique fingerprint.

Cite this