A survey on deep learning-based precise boundary recovery of semantic segmentation for images and point clouds

Precise localization of semantic segmentation is attracting increasing attention, and salient performances are dominated by deep learning-based methods, especially deep convolutional neural networks (DCNNs). However, the outputs from the final layer of DCNNs are not sufficiently localized for accurate object boundaries due to their invariance properties, which makes precise boundary recovery of semantic segmentation an academically challenging question. Both 2D and 3D objects suffer from the same problem. Considering this, this paper conducts a comprehensive survey of precise boundary recovery for semantic segmentation, focusing mainly on 2D images and 3D point clouds. Firstly, we formulate the problem of potential boundary recovery for semantic segmentation based on DCNNs, elaborate on the terminology as well as background concepts in this field. Then, we categorize boundary recovery methods into four strategies according to their techniques and network architectures to discuss how they obtain accurate boundaries of semantic segmentation. Next, publicly available datasets on which they have been assessed are argued. To compare these datasets, we design diagrams based on five indicators to help researchers judge which are the ones that best suit their tasks. Moreover, we further compare and analyze the performance of all the reviewed methods through experimental results. Finally, current challenges and prospective research issues are discussed extensively.