Building instance classification using social media images

Understanding urbanization and planning for the upcoming changes require detailed knowledge about the places where people live and work. Thus, knowing the usage of buildings is inevitable to distinguish between residential and commercial places. Assessing the usage of buildings from an aerial perspective alone is challenging and results in unresolveable ambiguities.As complementary data sources, social media images taken from ground level allow access to the building facades, as well as ongoing social activities around the buildings, which are very valuable information while coming to accessing the building usages. Towards the fusion of social media images and remote sensing data for this purpose, in this work we present a method to assess building usages from social media images taken in their neighborhood. Using a straight forward next neighbor classifier for mapping images to buildings and pre-trained networks for dimensionality reduction we trained a logistic regression classifier to distinguish between five different building usage classes. Applied to a study area of Los Angeles metropolitan area, USA, we obtain an average precision of 0.67. Hence, we show that social media images can be a valuable additional source to remote sensing data.