Automatic large-scale 3D building shape refinement using conditional generative adversarial networks

Three-dimensional realistic representations of buildings in urban environments have been increasingly applied as data sources in a growing number of remote sensing fields such as urban planning and city management, navigation, environmental simulation (i.e. flood, earthquake, air pollution), 3D change detection after events like natural disasters or conflicts, etc. With recent technological developments, it becomes possible to acquire high-quality 3D input data. There are two main ways to obtain elevation information: from active remote sensing systems, such as light detection and ranging (LIDAR), and from passive remote sensing systems, such as optical images, which allow the acquisition of stereo images for automatic digital surface models (DSMs) generation. Although airborne laser scanning provides very accurate DSMs, it is a costly method. On the other hand, the DSMs from stereo satellite imagery show a large coverage and lower costs. However, they are not as accurate as LIDAR DSMs. With respect to automatic 3D information extraction, the availability of accurate and detailed DSMs is a crucial issue for automatic 3D building model reconstruction. We present a novel methodology for generating a better-quality stereo DSM with refined buildings shapes using a deep learning framework. To this end, a conditional generative adversarial network (cGAN) is trained to generate accurate LIDAR DSM-like height images from noisy stereo DSMs.