Name
Title | ||
|---|---|---|
Resource-Efficient and Convergence-Preserving Online Participant Selection in Federated Learning |
Abstract | ||
|---|---|---|
Federated learning achieves the privacy-preserving training of models on mobile devices by iteratively aggregating model updates instead of raw training data to the server. Since excessive training iterations and model transferences incur heavy usage of computation and communication resources, selecting appropriate devices and excluding unnecessary model updates can help save the resource usage. We formulate an online time-varying non-linear integer program to minimize the cumulative resource usage over time while achieving the desired long-term convergence of the model being trained. We design an online learning algorithm to make fractional control decisions based on both previous system dynamics and previous training results, and also design an online randomized rounding algorithm to convert the fractional decisions into integers without violating any constraints. We rigorously prove that our online approach only incurs sub-linear dynamic regret for the optimality loss and sub-linear dynamic fit for the long-term convergence violation. We conduct extensive trace-driven evaluations and confirm the empirical superiority of our approach over alternative algorithms in terms of up to 27% reduction on the resource usage while sacrificing only 4% reduction on accuracy. |
Year | DOI | Venue |
|---|---|---|
2020 | 10.1109/ICDCS47774.2020.00049 | 2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS) |
Keywords | DocType | ISSN |
resource-efficient,convergence-preserving online participant selection,federated learning,mobile devices,raw training data,excessive training iterations,model transferences,heavy usage,communication resources,appropriate devices,excluding unnecessary model updates,online time-varying nonlinear integer program,cumulative resource usage,online learning algorithm,fractional control decisions,previous system dynamics,previous training results,online randomized rounding algorithm,fractional decisions,online approach,sub-linear dynamic regret,optimality loss,long-term convergence violation,alternative algorithms | Conference | 1063-6927 |
ISBN | Citations | PageRank |
978-1-7281-7003-9 | 3 | 0.38 |
References | Authors | |
0 | 6 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Yibo Jin | 1 | 5 | 3.78 |
| Lei Jiao | 2 | 732 | 54.48 |
| Zhuzhong Qian | 3 | 380 | 51.27 |
| Sheng Zhang | 4 | 113 | 10.66 |
| Sanglu Lu | 5 | 1380 | 144.07 |
| Xiaoliang Wang | 6 | 19 | 5.46 |