Sensor Fusion using Probabilistic Object Detection for State Estimation

Kalman filtering for global navigation satellite systems (GNSS)-aided inertial navigation has been widely used for state estimation. However, the performance of this technique is highly dependent on the accuracy of the individual sensor measurements - and is limited by the difficulty of modeling of non-linearity for filtering. This has motivated the use of learning-based and hybrid visual-inertial kinematic state estimators using deep learning. However, despite their performance, the deep learning techniques are data inefficient, computationally intensive, and have poor interpretability. We propose an extended Kalman filter (EKF)-based rule-based fusion of visual information with inertial and GNSS measurements. An artificial intelligence (AI)-based probabilistic object detection (POD) algorithm is used to detect (classify and localize) a known landmark, i.e., a landing pad, along with quantifying the semantic and spatial uncertainty of the detection. Monte-Carlo dropout is used to quantify the uncertainties of the object detection. Spatial uncertainty is used as the time-varying statistical noise characteristics for the fusion of the landmark detection. Semantic uncertainty is used to prevent fusion of detections with high uncertainty. Numerical simulations are performed to demonstrate the effectiveness of the proposed method.