FedDAG: Clustered Federated Learning via Global Data and Gradient Integration for Heterogeneous Environments

Each client \(i\) is initialized with random parameters \( \theta_{i}^{0} \) and performs local training on local dataset \( D_{i} \) for \( t_{g} = 2 \) rounds to obtain a local gradient update \( \Delta^{i} \).

The local gradient update is \(k\)-sparsified that we only retain a small random subset of entries to get \( \tilde{\Delta}^{i} \). The sparsified update is sent to server, and cosine similarity matrix is computed to be taken as gradient-based similarity. The similarity \( \mathcal{G}_{i,j} \) between client \( i,j \) is computed as

\[ \mathcal{G}_{i,j} = \cos^{-1}\left( \frac{\langle \tilde{\Delta}^{i}, \tilde{\Delta}^{j} \rangle}{\| \tilde{\Delta}^{i} \| \| \tilde{\Delta}^{j} \|} \right) \times \frac{180}{\pi} \]