Object detection-based knowledge graph creation: Enabling insight into construction processes

Compared to other industries, the construction sector shows low productivity worldwide. However, holistic, data-oriented methods for investigating potential bottlenecks within the as-performed construction stage are scarce. Our research presents an approach to acquiring raw data from job sites and its subsequent processing to high-level information. First, images were captured over a period of one year in high frequency using multiple crane cameras. Second, an end-to-end deep learning based approach was developed to derive and link information about construction activities, covering the classification and localization of specific on-site objects. This information was subsequently integrated into a knowledge graph. Finally, additional data sources like the weather were exploited to interpret different on-site scenarios. We demonstrate that construction-related activities like working times can be detected. The presented approach provides a significant step toward exposing correlations on construction sites by using multiple data processing steps and showcases the possibility of identifying process patterns.