Registration strategy of point clouds based on region-specific projections and virtual structures for robot-based inspection systems

Current requirements resulting from high quality standards, individualized mass production, and cost pressure call for new concepts in manufacturing metrology. Robot-based optical inspection systems have shown high potential to cope with these challenges. Since the kinematics of robots usually exhibit significant inaccuracies, an alternative approach becomes necessary to register multiple views of a sensor. With the advancement of optical 3D sensors in recent years, registration methods based on measured data (e.g. point clouds with image data) have shown tremendous progress and potential. Therefore, we present a new registration strategy which uses region-specific projections (image data) and virtual structures (point data) to support the alignment process of multiple point clouds. The fine registration is done in a pairwise manner and by means of an Iterative Closest Point (ICP) algorithm. We first introduce a procedure to obtain a set of virtual structures. Subsequently, influencing factors, such as the parametrization of the structures, noise, detection methods for the projected shapes, and the underlying surface, are examined. Based on these results, we identify virtual structures and employ them to evaluate the registration strategy in a real-world use case in the context of geometric quality assurance in the automotive industry. Acquired sphere-to-sphere distances are compared to reference measurements obtained from a coordinate measuring machine (CMM). The virtual structure comprising three point-based “columns” with an increasing point spacing in a triangular arrangement showed the best overall performance. The mean error (34 μm for one registration; 70 μm and 54 μm for two registrations respectively) in comparison to the robot kinematics was reduced by a factor of about 14 to 23 for the different reference distances. We demonstrated that our registration strategy enables a highly accurate and robust alignment of point clouds, which benefits manufacturing metrology for the inspection of sheet metal parts.