LA-LoRA: Rethinking LoRA for Privacy-Preserving Federated Learning in Large Models
Table of Contents
This article reviews federated learning under differential privacy federated learning settings.
This work aims at sample-level privacy with differential privacy and federated learning. Federated learning itself only guarantees that clients’ data do not leave clients, but LoRA matrices computed from local data may still carry information about local data. Although the server do not know clients’ data, it may infer from LoRA gradients, LoRA parameter differences, etc.
Thus, the following LoRA adopts the paradigm that adding DP noise at every local training step.
1. Challenge
1.1. Gradient Coupling
1.2. Compounded Noise Amplification
1.3. Sharpness of Global Aggregated Model in the Parameter Space
2. Solution: Local Alternating LoRA
At optimization level, local alternating update strategy is proposed to break the dependency between the 2 LoRA matrices to decouple gradient. At pre-aggregation level, optional Gaussian Low-Pass Filter is proposed to suppress residual variance by filtering out high-frequency components of DP perturbations on each client before aggregation.
2.1. Local Alternating Update Strategy
LoRA consists of matrices \( A,B \). LA-LoRA adopts an alternating scheme that 2 matrices are updated in turn.
For client \( i \) at local step \( k \) of communication round \( t \), the rule is:
- Update \( B_{i} \) if \( k \) is odd while keeping \( A_{i} \) fixed
\[ B_{i,k+1}^{t} = B_{i,k}^{t} - \eta_{B} \nabla_{B}\mathcal{L}_{i}(W_{0} + s B_{i,k}^{t} A_{i,k}^{t}) \]
- Update \( A_{i} \) if \( k \) is even while keeping \( B_{i} \) fixed
\[ A_{i,k+1}^{t} = A_{i,k}^{t} - \eta_{B} \nabla_{A}\mathcal{L}_{i}(W_{0} + s B_{i,k}^{t} A_{i,k}^{t}) \]
Here, \( W_{0} \) is the server-side frozen backbone; \( \mathcal{L}_{i} \) is the local training loss on client \( i \)’s private dateset; \( s \) is the LoRA scaling.
2.2. Smoothing with Low-Pass Filter
DP noise often manifests itself as high-frequency perturbations which de-stabilize local updates and amplify sharpness in global aggregation.
LA-LoRA uses a Gaussian conv kernel \( G_{s} = \frac{1}{16}[1,4,6,4,1] \) to smooth both \( A,B \): for \( A\in\mathbb{R}^{r\times n} \), smoothing is applied row-wise along the input feature dimension; for \( B\in\mathbb{R}^{m\times r} \), smoothing is applied column-wise.
This filtering is applied to LoRA gradients before aggregation.
3. Analysis
4. Experiments
5. TODO Future Exploration idea
Output/Update perturbations. Instead of adding DP noise at every local training step, how about just adding DP noise before gradient loading on \( A,B \)?