FedEmerge: An Entropy-Guided Federated Learning Method for Sensor Networks and Edge Intelligence

Abstract

Introduction: Federated Learning (FL) is a distributed machine learning paradigm where a global model is collaboratively trained across multiple decentralized clients without exchanging raw data. This is especially important in sensor networks and edge intelligence, where data privacy, bandwidth constraints, and data locality are paramount. Traditional FL methods like FedAvg struggle with highly heterogeneous (non-IID) client data, which is common in these settings. Background: Traditional FL aggregation methods, such as FedAvg, weigh client updates primarily by dataset size, potentially overlooking the informativeness or diversity of each client's contribution. These limitations are especially pronounced in sensor networks and IoT environments, where clients may hold sparse, unbalanced, or single-modality data. Methods: We propose FedEmerge, an entropy-guided aggregation approach that adjusts each client's impact on the global model based on the information entropy of its local data distribution. This formulation introduces a principled way to quantify and reward data diversity, enabling an emergent collective learning dynamic in which globally informative updates drive convergence. Unlike existing methods that weigh updates by sample count or heuristics, FedEmerge prioritizes clients with more representative, high-entropy data. The FedEmerge algorithm is presented with full mathematical detail, and we prove its convergence under the Polyak-Łojasiewicz (PL) condition. Results: Theoretical analysis shows that FedEmerge achieves linear convergence to the optimal model under standard assumptions (smoothness and PL condition), similar to centralized gradient descent. Empirically, FedEmerge improves global model accuracy and convergence speed on highly skewed non-IID benchmarks, and it reduces performance disparities among clients compared to FedAvg. Evaluations on CIFAR-10 (non-IID), Federated EMNIST, and Shakespeare datasets confirm its effectiveness in practical edge-learning settings. Conclusions: This entropy-guided federated strategy demonstrates that weighting client updates by data diversity enhances learning outcomes in heterogeneous networks. The approach preserves privacy like standard FL and adds minimal computation overhead, making it a practical solution for real-world federated systems.

Keywords: Polyak–Łojasiewicz condition; convergence analysis; emergent collective learning; entropy; federated learning; model aggregation; non-IID data.

Figure 1

Illustration of the entropy-guided aggregation process in FedEmerge. Each client computes the entropy $H_i$ of its local data distribution (depicted by pie charts showing class proportions). Clients with more uniform data (higher entropy) receive higher weight $p_i$ in the server’s aggregation of model updates, as indicated by thicker arrows. This strategy emphasizes diverse contributions, enabling an emergent global model that synthesizes broad information from the federation.

Figure 2

Test Accuracy vs. Communication Rounds on CIFAR-10 (non-IID). FedAvg converges slowly and plateaus around 60%, FedProx improves slightly to 65%, while FedEmerge achieves 75% with faster convergence.

Figure 3

Accuracy vs. Communication Rounds for EMNIST. FedEmerge consistently converges faster and to a higher final accuracy compared to FedAvg and FedProx.

Figure 4

Per-client Accuracy Distribution (Standard Deviation). FedEmerge achieves the lowest variance in accuracy across clients, indicating improved fairness. FedAvg and FedProx show higher and more persistent disparities in per-client accuracy over communication rounds.

Figure 5

Ablation study comparing FedEmerge with RandomEmerge (random weights summing to 1) and HybridEmerge (weights proportional to $n_i·H_i$). FedAvg is included as a baseline. FedEmerge and HybridEmerge significantly outperform the others, demonstrating the effectiveness of entropy-based aggregation. RandomEmerge shows no improvement over FedAvg, indicating that the benefit is due to principled entropy weighting, not randomness.

Figure 6

Ablation study comparing entropy weighting strategies: linear (FedEmerge), log-scaled, and softmax. Accuracy is plotted over communication rounds on CIFAR-10 (non-IID).