Event-based Driver Distraction Detection and Action Recognition

Driver distraction is one of the important factors leading to traffic accidents. With the development of mobile infotainment and the overestimation of immature autonomous driving technology, this phenomenon has become more and more serious. However, most existing distraction detection algorithms can not achieve satisfactory performance due to the complex in-cabin light condition and limited computing resource of edge devices. To this end, we introduce a light weight and flexible event-based system to monitor driver state. Compared with frame-based camera, the event camera responds to pixel wise light intensity changes asynchronously and has several promising advantages, including high dynamic range, high temporal resolution, low latency and low data redundant, which makes it suitable for the mobile terminal applications. The system first denoises the events stream and encode it into a sequence of 3D tensors. Then, the voxel features at different time steps are extracted using efficient net and fed into LSTM to establish temporal model, based on which, the driver distraction is detected. In addition, we extend the proposed architecture to recognise driver action and adopt transfer learning strategy to improve the detection performance. Extensive experiments are conducted on both simulated dataset (transform from Drive&Act) and real event dataset (collected by ourselves). The experimental results shows the advantages of the system on accuracy and efficient for driver state monitoring.