Sensor fusion neural networks for gesture recognition on low-power edge devices

The goal of hand gesture recognition based on time-of-flight and radar sensors is to enhance the human-machine interface, while taking care of privacy issues of camera sensors. Additionally, the system needs to be deployable on low-power edge devices for applicability in serial-produced vehicles. Recent advances show the capabilities of deep neural networks for gesture classification but they are still limited to high performance hardware. Embedded neural network accelerators are constrained in memory and supported operations. These limitations form an architectural design problem that is addressed in this work. Novel gesture classification networks are optimized for embedded deployment. The new approaches perform equally compared to high-performance neural networks with 3D convolutions, but need only 8.9% of the memory. These lightweight network architectures allow deployment on constrained embedded accelerator devices, thus enhancing human-machine interfaces.