Yashas Malur Saidutta

Not All Ranks Are Equal: Budget-Aware LoRA Merging Across Tasks

Merging low-rank adapters (LoRAs) promises to eliminate the overhead of swapping task-specific weights at inference time. However, existing merging methods assume every layer needs the same rank budget. Further, some methods assume that rank budget needs to be split equally among the tasks too. We show this uniform-bud...