TY - JOUR
T1 - A co-learning method to utilize optical images and photogrammetric point clouds for building extraction
AU - Xie, Yuxing
AU - Tian, Jiaojiao
AU - Zhu, Xiao Xiang
N1 - Publisher Copyright:
© 2022
PY - 2023/2
Y1 - 2023/2
N2 - Although deep learning techniques have brought unprecedented accuracy to automatic building extraction, several main issues still constitute an obstacle to effective and practical applications. The industry is eager for higher accuracy and more flexible data usage. In this paper, we present a co-learning framework applicable to building extraction from optical images and photogrammetric point clouds, which can take the advantage of 2D/3D multimodality data. Instead of direct information fusion, our co-learning framework adaptively exploits knowledge from another modality during the training phase with a soft connection, via a predefined loss function. Compared to conventional data fusion, this method is more flexible, as it is not mandatory to provide multimodality data in the test phase. We propose two types of co-learning: a standard version and an enhanced version, depending on whether unlabeled training data are employed. Experimental results from two data sets show that the methods we present can enhance the performance of both image and point cloud networks in few-shot tasks, as well as image networks when applying fully labeled training data sets.
AB - Although deep learning techniques have brought unprecedented accuracy to automatic building extraction, several main issues still constitute an obstacle to effective and practical applications. The industry is eager for higher accuracy and more flexible data usage. In this paper, we present a co-learning framework applicable to building extraction from optical images and photogrammetric point clouds, which can take the advantage of 2D/3D multimodality data. Instead of direct information fusion, our co-learning framework adaptively exploits knowledge from another modality during the training phase with a soft connection, via a predefined loss function. Compared to conventional data fusion, this method is more flexible, as it is not mandatory to provide multimodality data in the test phase. We propose two types of co-learning: a standard version and an enhanced version, depending on whether unlabeled training data are employed. Experimental results from two data sets show that the methods we present can enhance the performance of both image and point cloud networks in few-shot tasks, as well as image networks when applying fully labeled training data sets.
KW - Building extraction
KW - Co-learning
KW - Multimodality learning
KW - Multispectral images
KW - Point clouds
KW - Remote sensing
UR - http://www.scopus.com/inward/record.url?scp=85145275682&partnerID=8YFLogxK
U2 - 10.1016/j.jag.2022.103165
DO - 10.1016/j.jag.2022.103165
M3 - Review article
AN - SCOPUS:85145275682
SN - 1569-8432
VL - 116
JO - International Journal of Applied Earth Observation and Geoinformation
JF - International Journal of Applied Earth Observation and Geoinformation
M1 - 103165
ER -