TY - GEN
T1 - Differentially Private Federated Learning
T2 - 2023 IEEE International Conference on Big Data, BigData 2023
AU - Pustozerova, Anastasia
AU - Baumbach, Jan
AU - Mayer, Rudolf
N1 - Publisher Copyright:
© 2023 IEEE.
PY - 2023
Y1 - 2023
N2 - Federated Learning (FL) is a method that allows multiple entities to jointly train a machine learning model using data located in various places. Unlike the conventional approach of gathering private data from distributed locations to a central place, federated learning involves solely exchanging and aggregating the machine learning models. Each party shares only a machine learning model trained locally on their private data, ensuring that the sensitive data remains within the respective silos throughout the process. However, these shared models in FL may still leak sensitive information about the training data in the form of e.g. membership disclosure. To mitigate these residual privacy risks in federated learning, one has to use additional defence techniques such as Differential Privacy (DP), which introduces noise into the training data or the model. Differential Privacy provides a mathematical definition of privacy and can be applied in machine learning via different perturbation mechanisms. This work focuses on the analysis of Differential Privacy in federated learning through (i) output perturbation of the trained machine learning models and (ii) a differentially-private form of stochastic gradient descent (DP-SGD). We consider these two approaches in various settings and analyse their performance in terms of model utility and achieved privacy. To evaluate a model's privacy risk, we empirically measure the success rate of a membership inference attack. We observe that DP-SGD allows for a better trade-off between privacy and utility in most of the considered settings. In some settings, however, output perturbation can provide a better or similar privacy-utility trade-off and at the same time better communication and computational efficiency.
AB - Federated Learning (FL) is a method that allows multiple entities to jointly train a machine learning model using data located in various places. Unlike the conventional approach of gathering private data from distributed locations to a central place, federated learning involves solely exchanging and aggregating the machine learning models. Each party shares only a machine learning model trained locally on their private data, ensuring that the sensitive data remains within the respective silos throughout the process. However, these shared models in FL may still leak sensitive information about the training data in the form of e.g. membership disclosure. To mitigate these residual privacy risks in federated learning, one has to use additional defence techniques such as Differential Privacy (DP), which introduces noise into the training data or the model. Differential Privacy provides a mathematical definition of privacy and can be applied in machine learning via different perturbation mechanisms. This work focuses on the analysis of Differential Privacy in federated learning through (i) output perturbation of the trained machine learning models and (ii) a differentially-private form of stochastic gradient descent (DP-SGD). We consider these two approaches in various settings and analyse their performance in terms of model utility and achieved privacy. To evaluate a model's privacy risk, we empirically measure the success rate of a membership inference attack. We observe that DP-SGD allows for a better trade-off between privacy and utility in most of the considered settings. In some settings, however, output perturbation can provide a better or similar privacy-utility trade-off and at the same time better communication and computational efficiency.
KW - DP-SGD
KW - Differential Privacy
KW - Federated Learning
KW - Output Perturbation
UR - http://www.scopus.com/inward/record.url?scp=85184976283&partnerID=8YFLogxK
U2 - 10.1109/BigData59044.2023.10386466
DO - 10.1109/BigData59044.2023.10386466
M3 - Conference contribution
AN - SCOPUS:85184976283
T3 - Proceedings - 2023 IEEE International Conference on Big Data, BigData 2023
SP - 5549
EP - 5558
BT - Proceedings - 2023 IEEE International Conference on Big Data, BigData 2023
A2 - He, Jingrui
A2 - Palpanas, Themis
A2 - Hu, Xiaohua
A2 - Cuzzocrea, Alfredo
A2 - Dou, Dejing
A2 - Slezak, Dominik
A2 - Wang, Wei
A2 - Gruca, Aleksandra
A2 - Lin, Jerry Chun-Wei
A2 - Agrawal, Rakesh
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 15 December 2023 through 18 December 2023
ER -