Learning spectral-spatialoral features via a recurrent convolutional neural network for change detection in multispectral imagery

Lichao Mou, Lorenzo Bruzzone, Xiao Xiang Zhu

Research output: Contribution to journalArticlepeer-review

421 Scopus citations


Change detection is one of the central problems in earth observation and was extensively investigated over recent decades. In this paper, we propose a novel recurrent convolutional neural network (ReCNN) architecture, which is trained to learn a joint spectral-spatialoral feature representation in a unified framework for change detection in multispectral images. To this end, we bring together a convolutional neural network and a recurrent neural network into one end-to-end network. The former is able to generate rich spectral-spatial feature representations, while the latter effectively analyzes temporal dependence in bitemporal images. In comparison with previous approaches to change detection, the proposed network architecture possesses three distinctive properties: 1) it is end-to-end trainable, in contrast to most existing methods whose components are separately trained or computed; 2) it naturally harnesses spatial information that has been proven to be beneficial to change detection task; and 3) it is capable of adaptively learning the temporal dependence between multitemporal images, unlike most of the algorithms that use fairly simple operation like image differencing or stacking. As far as we know, this is the first time that a recurrent convolutional network architecture has been proposed for multitemporal remote sensing image analysis. The proposed network is validated on real multispectral data sets. Both visual and quantitative analyses of the experimental results demonstrate competitive performance in the proposed mode.

Original languageEnglish
Article number8541102
Pages (from-to)924-935
Number of pages12
JournalIEEE Transactions on Geoscience and Remote Sensing
Issue number2
StatePublished - Feb 2019


  • Change detection
  • long short-term memory (LSTM)
  • multitemporal image analysis
  • recurrent convolutional neural network (ReCNN)


Dive into the research topics of 'Learning spectral-spatialoral features via a recurrent convolutional neural network for change detection in multispectral imagery'. Together they form a unique fingerprint.

Cite this