The Evolution of Distributed Systems for Graph Neural Networks and Their Origin in Graph Processing and Deep Learning: A Survey

Graph neural networks (GNNs) are an emerging research field. This specialized deep neural network architecture is capable of processing graph structured data and bridges the gap between graph processing and deep learning. As graphs are everywhere, GNNs can be applied to various domains including recommendation systems, computer vision, natural language processing, biology, and chemistry. With the rapid growing size of real-world graphs, the need for efficient and scalable GNN training solutions has come. Consequently, many works proposing GNN systems have emerged throughout the past few years. However, there is an acute lack of overview, categorization, and comparison of such systems. We aim to fill this gap by summarizing and categorizing important methods and techniques for large-scale GNN solutions. Additionally, we establish connections between GNN systems, graph processing systems, and deep learning systems.