Integrating Agglomerative Perception with One-step Action Generation for Robotic Manipulation.
TL;DR
Flow2Act is presented, a unified framework that integrates agglomerative perception with a deterministic one-step generative policy and devise a curriculum region-aware mechanism via a Spatial-Grounded State Space Duality architecture, demonstrating significant gains in policy performance, robustness to environmental perturbations, and cross-task real-world applicability.
Abstract
Developing generalizable robotic policies that balance inference efficiency, manipulation accuracy, and robustness remains a formidable challenge. Existing Vision-Language-Action models demand prohibitive data scales, while keyframe-based approaches struggle to reconcile the expressivity of generative models with the latency of iterative sampling. To address this trilemma, we present Flow2Act, a unified framework that integrates agglomerative perception with a deterministic one-step generative policy. Unlike prior methods relying on separate semantic encoders or iterative diffusion processes, our approach introduces three key innovations. First, we employ an agglomerative multi-teacher visual backbone that distills complementary strengths from diverse foundation models, capturing semantics, spatial structure, and segmentation to yield robust representations without task-specific pretraining. Second, we propose a conditional MeanFlow policy that parameterizes the interval-averaged velocity field. This formulation enables genuine single-step action generation, eliminating the discretization errors and computational overhead inherent in ODE-based flow matching. Third, we devise a curriculum region-aware mechanism via a Spatial-Grounded State Space Duality architecture, which progressively shifts attention from global flow stability to fine-grained contact precision. We evaluate Flow2Act on challenging simulation benchmarks and real-world robotic tasks, demonstrating significant gains in policy performance, robustness to environmental perturbations, and cross-task real-world applicability. Videos, code, and more details are available at project page.