Homomorphic Encryption in Federated Medical Image Classification

The application of Federated Learning (FL) for training neural networks has expanded significantly in domains requiring sensitive data handling, such as the medical field, yielding substantial success. However, while solving some issues, FL also introduces new security concerns due to the need to trust multiple participating clients. In this work, we first demonstrate that FL can be effectively applied for the classification of a novel type of quantitative phase image data in cytology, and we evaluate the potential of Homomorphic Encryption (HE) to enhance security in this federated training. We specifically focus on encrypting the weight updates rather than the image data itself, revealing that this approach drastically reduces computational and communication overhead. We further show that this method has minimal effect on classification performance, maintaining or even increasing the accuracy of a centralized setup. Although encrypting weight updates increases training time by up to 39%, it does not impact inference speed, as inference occurs locally. We validate these findings across various neural network architectures and model sizes, confirming that HE on weight updates offers a practical and secure solution for federated training on sensitive medical images without compromising performance.