Federated Learning Based on Dimension Selection
DOI:
https://doi.org/10.62051/ijcsit.v5n1.06Keywords:
Federated Learning, Local Differential Privacy, Dimension SelectionAbstract
Federated learning, as a distributed machine learning framework, enables users to collaboratively train models by sharing model parameters without exposing their raw data. However, model parameters may contain privacy-sensitive information, and directly sharing them still poses a risk of user privacy leakage. Local Differential Privacy (LDP) effectively defends against adversaries with arbitrary background knowledge, providing more comprehensive privacy protection. However, the high-dimensional nature of parameters in federated learning presents challenges for the application of LDP. To address this issue, this paper proposes a federated learning algorithm, FDL, that satisfies local differential privacy. The algorithm employs a dimension selection strategy to identify parameter dimensions critical for global aggregation and applies the Laplace mechanism to perturb these dimensions. Compared to traditional methods, the FDL algorithm significantly reduces the number of parameters to be processed and the amount of noise introduced. Theoretical analysis proves that the FDL algorithm satisfies-local differential privacy, and experimental results demonstrate its high usability while ensuring strong privacy protection.
Downloads
References
[1] WANG X, LI J, LI J, et al. Multilevel similarity model for high-resolution remote sensing image registration [J]. Information Sciences, 2019, 505: 294-305.
[2] CAI Y, LUAN T, GAO H, et al. YOLOv4-5D: An effective and efficient object detector for autonomous driving [J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-13.
[3] MCMAHAN B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data [C]// Artificial Intelligence and Statistics. PMLR, 2017: 1273-1282.
[4] MELIS L, SONG C, DE CRISTOFARO E, et al. Exploiting unintended feature leakage in collaborative learning [C]// 2019 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA, 2019: 691-706.
[5] NASR M, SHOKRI R, HOUMANSADR A. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning [C]// 2019 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA, 2019: 739-753.
[6] XU G, LI H W, ZHANG Y, et al. Privacy-preserving federated deep learning with irregular users [J]. IEEE Transactions on Dependable and Secure Computing, 2020, 19 (2): 1364-1381.
[7] TRUEX S, BARACALDO N, ANWAR A, et al. A hybrid approach to privacy preserving federated learning [C]// Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security,New York, NY, USA, 2019: 1-11.
[8] YIN, L, FENG J, XUN H, et al. A privacy-preserving federated learning for multiparty data sharing in social IoTs [J]. IEEE Transactions on Network Science and Engineering, 2021,8(3), 2706–2718.
[9] DWORK C. Differential privacy [C] //Automata, Languages and Programming: 33rd International Colloquium, ICALP 2006, Venice, Italy, 2006. Berlin, Heidelberg: Springer, 2006, Part II 33: 1-12.
[10] YANG M M, GUO T L, ZHU T Q, et al. Local differential privacy and its applications: A comprehensive survey [J]. Computer Standards & Interfaces, 2024, 89: 103827.
[11] ZHAO Y, ZHAO J, YANG M, et al. Local differential privacy based federated learning for internet of things [J]. IEEE Internet of Things Journal, 2021, 8(11):8836–8853.
[12] JIA W, SUN M, LIAN J, et al. Feature dimensionality reduction: a review [J]. Complex & Intelligent Systems, 2022, 8(3): 2663-2693.
[13] MAHAWAGA ARACHCHIGE P C, BERTOK P, KHALIL I, et al. Local differential privacy for deep learning [J]. IEEE Internet Things, 2020, 7(7): 5827–5842.
[14] WANG, N, Xiao X K, YANG Y, et al. Collecting and analyzing multidimensional data with local differential privacy[C]//2019 IEEE 35th International Conference on Data Engineering (ICDE), Macao, China, 2019: 638-649.
[15] WANG Y S, TONG Y X, SHI D Y. Federated Latent Dirichlet Allocation: A Local Differential Privacy Based Framework [J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(4):6283-6290.
[16] Shin, H., Kim, S., Shin, J., & Xiao, X. Privacy enhanced matrix factorization for recommendation with local differential privacy [J]. IEEE Transactions on Knowledge and Data Engineering, 2018, 30:1770–1782.
[17] Wang J Z, Kong L W, Huang Z, et al. Research review of federated learning algorithms [J]. Big Data Research, 2020, 6(6): 64-82.
[18] Wang B, Yao Y, Shan S, et al. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks [C]//2019 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA, 2019: 707-723.
[19] SHOKRI R, SHMATIKOV V. Privacy-preserving deep learning [C]// Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, New York, NY, USA, 2015: 1310-1321.
[20] ABADI M, CHU A, GOODFELLOW I, et al. Deep learning with differential privacy [C]// Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Ne-w York, NY, USA, 2016: 308-318.
[21] GEYER R C, KLEIN T, NABI M. Differentially private federated learning: A client level perspective [J]. arXiv preprint arXiv:1712.07557, 2017.
[22] SHIN H, KIM S, SHIN J, et al. Privacy enhanced matrix factorization for recommendation with local differential privacy [J]. IEEE Transactions on Knowledge and Data Engineering, 2018, 30(9): 1770-1782.
[23] MIAO Y, XIE R, LI X, et al. Compressed federated learning based on adaptive local differential privacy[C]//Proceedings of the 38th Annual Computer Security Applications Conference, New York, NY, USA, 2022: 159-170.
[24] CUI L, MA J, ZHOU Y, et al. Boosting accuracy of differentially private federated learning in industrial IoT with sparse responses [J]. IEEE Transactions on Industrial Informatics, 2022, 19(1): 910-920.
[25] WANG B, CHEN Y, JIANG H, et al. PPeFL: Privacypreserving edge federated learning with local differential privacy [J]. IEEE Internet of Things Journal, 2023.
[26] Liu R, CAO Y, YOSHIKAWA M, et al. Fedsel: Federated sgd under local differential privacy with top-k dimension selection[C]//Database Systems for Advanced Applications: 25th International Conference, DASFAA 2020, Jeju, South Korea, 2020, Proceedings, Part I 25. Springer International Publishing, 2020: 485-501.
[27] RASHID T. Make Your Own Neural Network [M]. CreateSpace Independent Publishing Platform, 2016.
[28] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition [J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[29] JIANG X, ZHOU X, GROSSKLAGS J. Sign-fl: Local differentially private federated learning with sign-based dimension selection [J]. ACM Transactions on Intelligent Systems and Technology (TIST), 2022, 13(5): 1-22.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 International Journal of Computer Science and Information Technology
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
How to Cite
