Hermes: An Efficient Federated Learning Framework for Heterogeneous Mobile Clients

Overview

Step

1. Each device incorporates a structured sparsity regularization to learn a subnetwork when optimizing the local model.
2. Only such subnetworks are transmitted from devices to the central server.
3. In the central server, only the parameters are intersected across the subnetworks of devices are averaged while keeping the remaining non-intersected parameters untouched.
4. The updated subnetworks will be distributed to each device.
5. Repeat steps 1~4 until reaching the predefined number of communication rounds.
6. Each devices will finally obtain a personalized and structured-sparse model.