Analysis of a multimodal-camera and its advantages for autonomous vehicles

For autonomous vehicles and robots a fast, complete, and reliable acquisition of the environment is crucial for almost every task they perform. To fulfil this, optical sensors with different spectral sensibility are one of the most important sensors as they provide very rich information about the scene. Regarding outdoor environments, the contained dynamics are very high which arise on the one hand from object movements and self motion and on the other hand from changing lighting conditions due to varying weather conditions. These high dynamics hinder a reliable scene acquisition using conventional optical sensors as they only offer a limited sampling rate, resolution, and dynamic range. To overcome these limitations without using specialized hardware we propose an assembly of several cameras and beam-splitters which we call a multimodal-camera. The cameras take images from the same scene from slightly different viewpoints and with diverse parameters like exposure, or shutter time which are all adjustable. By combing these images and applying techniques from computer graphics, we are able to create an output by computation that covers the scene's high dynamics and can be used for a reliable scene analysis.