Preprint

SparSTAR: Sparse Attention for SpaceTime AutoRegressive Video Synthesis

Jongbeom Lee Hyunwoo Yu Jincheol Yang Jaemin Choi Suk-Ju Kang

Aug 2026 · 0 citations · 36 references

Computer Science

TL;DR

SarSTAR is introduced, a training-free block-sparse attention method tailored to this setting, showing that recomputing block selection at each target scale is more reliable than reusing a transferred mask as reuse spans increasingly distant scales.

Abstract

InfinityStar extends visual autoregressive generation to video through a sequence of image and clip pyramids. Its changing scale and cross-clip context, however, leave late-scale attention costly and make sparse patterns reused from diffusion or image VAR models unreliable. We introduce SparSTAR, a training-free block-sparse attention method tailored to this setting. At each expensive scale and attention head, SparSTAR scores contiguous key blocks from the current query and key activations, retains required conditioning context, and executes the selected blocks through a forward-only sparse path. We analyze cross-scale consistency within a clip, pattern persistence across clip boundaries, and quality degradation as reuse spans increasingly distant scales. Across these analyses, important key blocks shift, showing that recomputing block selection at each target scale is more reliable than reusing a transferred mask. On 720p text-to-video and image-to-video generation, SparSTAR preserves every token and refinement scale while providing about a 1.6x end-to-end speedup and maintaining VBench and paired-output reconstruction fidelity close to dense InfinityStar.

arXiv Semantic Scholar

View source

Similar papers

Jul 2026

Ms. Forcing: Efficient Streaming Video Generation with Multi-Scale Patchification and Attention

This work proposes Ms.Forcing, an efficient streaming video generation paradigm that adapts spatial granularity to each state's noise level and introduces Homogeneous-Noise-Level DMD, which assembles each fake video from clean predictions sharing the same source noise level, thereby reducing the mismatch between DMD training sequences and inference-time rollouts.

Zekun Li, Xiaoyan Cong, Hongyu Li et al. · arXiv.org · 0 citations

Preprint Aug 2026

LoSA: Near-Lossless Sparse Attention for Training-Free Video Diffusion Acceleration

LoSA is a training-free sparse-attention method that fixes a retained-mass threshold of 99% rather than a sparsity ratio: it measures exact block attention masses at one early dense step, keeps, for each head and query block, the smallest key/value block set meeting the threshold, and reuses the frozen block indices for all remaining steps.

Enhuai Liu, Yunke Wang, Yutong Wang et al. · 0 citations

Jul 2026

SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation

This work introduces SANA-Video 2.0, a hybrid video diffusion transformer instantiated at 5B and 14B scales under a unified architecture that recovers softmax-level expressiveness at substantially reduced cost, unlocking scalable long, high resolution video generation.

Junsong Chen, Jincheng Yu, Yitong Li et al. · arXiv.org · 1 citation · ⚡1

Preprint Aug 2026

MotionCraft: Latent World Modeling with Sparse Attention for Visual Upscaling

MotionCraft is presented, a controllable VSR framework that formulates restoration as motion-aware latent state prediction inspired by world models and integrates adaptive sparse attention with an explicit user-accessible control interface to deliver temporally consistent, high-quality reconstructions under streaming constraints.

Rong Fu, Chunlei Meng, Yangcheng Zeng et al. · 0 citations

Jul 2026

Chimera: Designing and Chinchilla-Scaling Hybrid Visual Diffusion Transformers

Chimera, a hybrid visual diffusion backbone with a principled scaling recipe that combines Kimi Delta Attention for long-context state tracking with O(N) complexity, interleaved Multi-head Latent Attention (MLA) for direct global interaction, and modality-aware short convolutions for local spatiotemporal context is introduced.

Chongjian Ge, Hanwen Jiang, Tianyu Wang et al. · arXiv.org · 2 citations

Jul 2026

HeadCast: Casting Attention Heads for Efficient Autoregressive Video Generation

HeadCast is proposed, a training-free, plug-and-play acceleration framework built on the observation that a pre-trained AR model's attention heads exhibit stable, heterogeneous behaviors that accelerates inference by up to 1.62x at 720P and 1.95x at 1080P, while keeping VBench quality on par with full attention and largely flicker-free.

Jinliang Shen, Li Su, Zheming Li et al. · arXiv.org · 1 citation