SQuad: Sub-Quadratic Attention Distillation for Efficient Video Generation
This work proposes SQuad, a Sub-Quadratic Attention Distillation framework that achieves a complexity of $\mathcal{O}(n\sqrt{n})$ in the resulting distilled Attention, naturally balancing the efficiency v/s expressivity trade-off.