FedVar: Federated Learning Algorithm with Weight Variation in Clients
English
Overview
This paper presents FedVar, a federated learning algorithm designed to improve global model aggregation under non-IID (non-independent and identically distributed) data conditions. FedVar refines the server-side aggregation process by computing the variance of client weights and excluding outlier clients with extreme deviations.
Citation:
Shin, W., & Shin, J. (2022, July). FedVar: Federated Learning Algorithm with Weight Variation in Clients.
In 2022 37th International Technical Conference on Circuits/Systems, Computers and Communications (ITC-CSCC) (pp. 1–4). IEEE.
Methodology
The proposed algorithm calculates the average and standard deviation of client model weights, then selects only those clients whose updates fall within one standard deviation from the mean. This mechanism ensures that clients with skewed data distributions are excluded from global aggregation.
Mathematical Definition
\[S(w) = \frac{1}{K}\sum_{k=1}^{K}w_k, \quad SD(w) = \sqrt{\frac{1}{K}\sum_{k=1}^{K}(w_k - S(w))^2}\] \[S(w) - SD(w) \leq w_k \leq S(w) + SD(w)\] \[SDA(w) = \frac{1}{n}\sum_{i=1}^{n}w_{sd,i}\]Clients outside this range are excluded from the update step, producing a global model that is both more stable and accurate under heterogeneous data conditions.
Experimental Setup
- Framework: Federated-Learning-PyTorch (Open Source)
- Models: TinyNet, GhostNet, MobileNetV3
- Datasets: CIFAR-10, CIFAR-100, MNIST
- Clients: 100 total
- Local Epochs: 5
- Rounds: 200
- Evaluation: Accuracy and convergence across Non-IID, Semi-IID, and Fully-IID settings
Results
| Setting | FedSGD | FedAvg | FedProx | FedVar (Proposed) |
|---|---|---|---|---|
| Non-IID (s=1) | 89.9% | 90.1% | 91.0% | 91.2% |
| Semi-IID (s=0.5) | 87.1% | 87.8% | 88.6% | 89.0% |
| Fully-IID (s=0) | 84.9% | 85.3% | 86.0% | 85.8% |
FedVar achieves the best accuracy in Non-IID environments while maintaining comparable performance in IID settings, demonstrating robustness to client heterogeneity.
Conclusion
FedVar effectively addresses data heterogeneity by integrating variance-based client selection into the aggregation process. The algorithm improves both stability and convergence of federated learning under Non-IID data.
한국어
연구 요약
본 논문은 클라이언트 간 데이터 분포 불균형(Non-IID) 상황에서 연합학습의 수렴 불안정성을 개선하기 위한 알고리즘 FedVar를 제안한다. 클라이언트별 모델 가중치의 표준편차(variance)를 계산하고, 평균 ± 표준편차 범위 내의 클라이언트만 전역 학습에 참여하도록 함으로써, 데이터 편향이 큰 클라이언트를 자동으로 배제한다. 이 방식은 Non-IID 환경에서 FedAvg나 FedProx보다 높은 정확도(약 91.2%)를 기록하며, 데이터 다양성이 큰 분산 환경에서도 안정적인 학습 성능을 보인다.