StateFlow: Sequence Pipeline Parallelism for Long-Context Modeling with Linear Recurrence
StateFlow is presented, a sequence pipeline parallelism system for models with linear recurrence that partitions each sequence into chunks and schedules their execution while propagating boundary states and gradients across chunks, thereby reducing activation lifetimes and improving training throughput.