Libra: Taming Attention Workload Skew in Long-Context LLM Training with Bounded Sequence Pool
Libra is presented, which operationalizes the law of large numbers (LLN) as a scaling principle for load balancing: the attention-balancing pool need not grow with the DP degree.