Overcoming Language Bias in Remote Sensing Visual Question Answering Via Adversarial Training

Zhenghang Yuan, Lichao Mou, Xiao Xiang Zhu

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

The Visual Question Answering (VQA) system offers a user-friendly interface and enables human-computer interaction. However, VQA models commonly face the challenge of language bias, resulting from the learned superficial correlation between questions and answers. To address this issue, in this study, we present a novel framework to reduce the language bias of the VQA for remote sensing data (RSVQA). Specifically, we add an adversarial branch to the original VQA framework. Based on the adversarial branch, we introduce two regularizers to constrain the training process against language bias. Furthermore, to evaluate the performance in terms of language bias, we propose a new metric that combines standard accuracy with the performance drop when incorporating question and random image information. Experimental results demonstrate the effectiveness of our method. We believe that our method can shed light on future work for reducing language bias on the RSVQA task.

Original languageEnglish
Title of host publicationIGARSS 2023 - 2023 IEEE International Geoscience and Remote Sensing Symposium, Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2235-2238
Number of pages4
ISBN (Electronic)9798350320107
DOIs
StatePublished - 2023
Externally publishedYes
Event2023 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2023 - Pasadena, United States
Duration: 16 Jul 202321 Jul 2023

Publication series

NameInternational Geoscience and Remote Sensing Symposium (IGARSS)
Volume2023-July

Conference

Conference2023 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2023
Country/TerritoryUnited States
CityPasadena
Period16/07/2321/07/23

Keywords

  • deep learning
  • language bias
  • remote sensing
  • visual question answering (VQA)

Fingerprint

Dive into the research topics of 'Overcoming Language Bias in Remote Sensing Visual Question Answering Via Adversarial Training'. Together they form a unique fingerprint.

Cite this