Communication Topologies for Decentralized Federated Learning

Conventional federated learning aims at enabling clients to contribute to a global training process while keeping their own data local. However, as the number of devices on the network increases, it can no longer be assumed that there is a central entity with sufficient bandwidth or computing resources to handle the volume of requests. Hence, in this paper, we consider implementing federated learning with different topologies in a network without a central entity. We compare hierarchical and decentralized topologies with varying degrees of interconnectivity. In our experiments, we use 50 clients with small CNNs and MNIST, FashinMNIST or Cifar10 datasets. Our results show that models in a decentralized network can achieve similar performances as models in a centralized network if the topology is carefully chosen. We relate the accuracy of the models to the estimated communication overhead by considering the number of communication connections required for a given topology. These results indicate that cluster topologies can leverage similarities of data distributions and mitigate the communication effort without sacrificing performance. In addition, we present a simple method to estimate the information transfer performance of a topology without empirical testing.