Multimodal remote sensing benchmark datasets for land cover classification with a shared and specific feature learning model

Danfeng Hong, Jingliang Hu, Jing Yao, Jocelyn Chanussot, Xiao Xiang Zhu

Research output: Contribution to journalArticlepeer-review

139 Scopus citations

Abstract

As remote sensing (RS) data obtained from different sensors become available largely and openly, multimodal data processing and analysis techniques have been garnering increasing interest in the RS and geoscience community. However, due to the gap between different modalities in terms of imaging sensors, resolutions, and contents, embedding their complementary information into a consistent, compact, accurate, and discriminative representation, to a great extent, remains challenging. To this end, we propose a shared and specific feature learning (S2FL) model. S2FL is capable of decomposing multimodal RS data into modality-shared and modality-specific components, enabling the information blending of multi-modalities more effectively, particularly for heterogeneous data sources. Moreover, to better assess multimodal baselines and the newly-proposed S2FL model, three multimodal RS benchmark datasets, i.e., Houston2013 – hyperspectral and multispectral data, Berlin – hyperspectral and synthetic aperture radar (SAR) data, Augsburg – hyperspectral, SAR, and digital surface model (DSM) data, are released and used for land cover classification. Extensive experiments conducted on the three datasets demonstrate the superiority and advancement of our S2FL model in the task of land cover classification in comparison with previously-proposed state-of-the-art baselines. Furthermore, the baseline codes and datasets used in this paper will be made available freely at https://github.com/danfenghong/ISPRS_S2FL.

Original languageEnglish
Pages (from-to)68-80
Number of pages13
JournalISPRS Journal of Photogrammetry and Remote Sensing
Volume178
DOIs
StatePublished - Aug 2021

Keywords

  • Benchmark datasets
  • Classification
  • DSM
  • Feature learning
  • Hyperspectral
  • Land cover mapping
  • Multimodal
  • Multispectral
  • Remote sensing
  • SAR
  • Shared features
  • Specific features

Fingerprint

Dive into the research topics of 'Multimodal remote sensing benchmark datasets for land cover classification with a shared and specific feature learning model'. Together they form a unique fingerprint.

Cite this