Influence-Based Mini-Batching for Graph Neural Networks

Johannes Gasteiger, Chendi Qian, Stephan Günnemann

Research output: Contribution to journalConference articlepeer-review

1 Scopus citations

Abstract

Using graph neural networks for large graphs is challenging since there is no clear way of constructing mini-batches. To solve this, previous methods have relied on sampling or graph clustering. While these approaches often lead to good training convergence, they introduce significant overhead due to expensive random data accesses and perform poorly during inference. In this work we instead focus on model behavior during inference. We theoretically model batch construction via maximizing the influence score of nodes on the outputs. This formulation leads to optimal approximation of the output when we do not have knowledge of the trained model. We call the resulting method influence-based mini-batching (IBMB). IBMB accelerates inference by up to 130x compared to previous methods that reach similar accuracy. Remarkably, with adaptive optimization and the right training schedule IBMB can also substantially accelerate training, thanks to precomputed batches and consecutive memory accesses. This results in up to 18x faster training per epoch and up to 17x faster convergence per runtime compared to previous methods.

Original languageEnglish
JournalProceedings of Machine Learning Research
Volume198
StatePublished - 2022
Event1st Learning on Graphs Conference, LOG 2022 - Virtual, Online
Duration: 9 Dec 202212 Dec 2022

Fingerprint

Dive into the research topics of 'Influence-Based Mini-Batching for Graph Neural Networks'. Together they form a unique fingerprint.

Cite this