Iterative calibration of a vehicle camera using traffic signs detected by a convolutional neural network

Intrinsic camera parameters are estimated during calibration typically using special reference patterns. Mechanical and thermal effects might cause the parameters to change over time, requiring iterative calibration. For vehicle cameras, reference information needed therefore has to be extracted from the scenario, as reference patterns are not available on public streets. In this contribution, a method for iterative camera calibration using scale references extracted from traffic signs is proposed. Traffic signs are detected in images recorded during driving using a convolutional neural network. Multiple detections are reduced by mean shift clustering, before the shape of each sign is fitted robustly with RANSAC. Unique image points along the shape contour together with the metric size of the traffic sign are included iteratively in the bundle adjustment performed for camera calibration. The neural network is trained and validated with over 50,000 images of traffic signs. The iterative calibration is tested with an image sequence of an urban scenario showing traffic signs. The results show that the estimated parameters vary in the first iterations, until they converge to stable values after several iterations. The standard deviations are comparable to the initial calibration with a reference pattern.