FedSL: A Communication-Efficient Federated Learning With Split Layer Aggregation

Faculty Computer Science Year: 2024
Type of Publication: ZU Hosted Pages: 15587 - 15601
Authors:
Journal: IEEE Internet of Things Journal IEEE Volume: 11
Keywords : FedSL: , Communication-Efficient Federated Learning With Split    
Abstract:
Federated learning (FL) can train a model collaboratively through multiple remote clients without sharing raw data. The challenge of federated learning (FL) is how to decrease network transmissions. This article aims to reduce network traffic by transmitting fewer neural network parameters. We first investigate similarities of different corresponding layers of convolutional neural network (CNN) models in FL, and find that there is a lot of redundant information in its model feature extractors. For this, we propose a communication-efficient federated aggregation algorithm named FedSL (Federated Split Layers) to reduce the communication overhead. Based on the number of global model layers, the FedSL divides client models into groups in the depth dimension. A Max-Min client selection strategy is employed to select participants for each layer. Each client only transfers partial parameters of those layers that are selected, which reduces the number of parameters. FedSL aggregates the global model in each group and concatenates the parameters of all groups according to the order of layers. The experimental results demonstrate that FedSL improves communication efficiency compared to the algorithms (e.g., FedAvg, FedProx, and MOON), decreasing 42% communication cost with VGG-style CNN and 70% with ResNet-9, while maintaining a similar model accuracy with baseline algorithms.
   
     
 
       

Author Related Publications

  • Wael Said AbdelMageed Mohamed, "A big data approach to sentiment analysis using greedy feature selection with cat swarm optimization-based long short-term memory neural networks", Springer Nature, 2018 More
  • Wael Said AbdelMageed Mohamed, "Improving the reconstruction of dental occlusion using a reconstructed‑based identical matrix point technique", Springer Nature Switzerland AG, 2021 More
  • Wael Said AbdelMageed Mohamed, "Connection-Adjustable Network Slicing Process for Heterogeneous Service Handling in Real-Time Applications", American Scientific Publishers, 2022 More
  • Wael Said AbdelMageed Mohamed, "Space Division Multiple Access for Cellular V2X Communications", Tech Science Press, 2022 More
  • Wael Said AbdelMageed Mohamed, "A Multi-Factor Authentication-Based Framework for Identity Management in Cloud Applications", Tech Science Press, 2021 More

Department Related Publications

  • Ahmed Salah Mohamed Mostafa, "Cluster-Distribute-Align-Merge: A General Algorithm to Speed Up Multiple Sequence Alignment on Multi-Core Computers", Journal of Computational and Theoretical Nanoscience, 2014 More
  • Zaher Awad Aboelenieen Elhendy, "NEW APPROACH TO IMAGE EDGE DETECTION BASED ON QUANTUM ENTROPY", JOURNAL OF RUSSIAN LASER RESEARCH, 2016 More
  • Sarah AbdelRazek Ahmed AbdulHameid, "Cloud Storage Forensics: Survey", International Journal of Engineering Trends and Technology (IJETT), 2017 More
  • Doaa El-Shahat Barakat Mohammed, "A modified hybrid whale optimization algorithm for the scheduling problem in multimedia data objects", Wiley online library, 2019 More
  • Abdallah Gamal abdallah mahmoud, "A novel model for evaluation Hospital medical care systems based on plithogenic sets", Elsevier B.V., 2019 More
Tweet