FedEx-LoRA: Exact Aggregation for Federated and Efficient Finetuning of Foundation Models

Direct LoRA in federated settings causes inaccurate updates. FedEx-LoRA handles this by adding a residual error term to the pretrained frozen weight matrix, achieving exact updates.