This work proposes OrthoSkillVLA, a parameter-efficient framework for continual skill learning in pretrained VLA models without demonstration replay, and introduces a lightweight feature-aware MoE decoder that better preserves prior skills while acquiring new ones.
Abstract
Pretrained Vision-Language-Action models provide a strong foundation for robot learning, but sequentially adapting them to diverse skills can perturb the representations and velocity mappings used by previous skills, leading to catastrophic forgetting. Architecture-based approaches improve retention by isolating skills but lead to increased inference footprint. Recent subspace-constrained methods restrict parameter updates in an orthogonal subspace to minimize interference but impose a unified constraint on the entire model. We analyze the distinct roles of internal VLA components and identify two VLA-specific challenges. First, the VLM maintains broad semantic representations, making it vulnerable to capacity exhaustion, whereas the ActionHead refines semantics into localized velocity patterns that are highly sensitive to perturbations. Second, the final velocity decoder serves as a readout layer. Freezing it forms an output-stage expressivity bottleneck, while updating it risks overwriting previous velocity mappings. To this end, we propose OrthoSkillVLA, a parameter-efficient framework for continual skill learning in pretrained VLA models without demonstration replay. Given the representation heterogeneity, we impose separate subspace constraints on the VLM and ActionHead, preserving reusable semantic capacity while protecting localized velocity patterns. For the output layer, we introduce a lightweight feature-aware MoE decoder, where each skill is allocated a compact expert and a training-free router selects the expert according to feature-space affinity. Extensive simulated and real-world evaluations, together with ablations, demonstrate that OrthoSkillVLA better preserves prior skills while acquiring new ones.
As Vision-Language-Action (VLA) models scale toward real-world deployment, contact-rich manipulation exposes a critical blind spot: these policies encode broad visual-semantic priors yet remain unaware of local contact events, producing identical actions whether contact is established, lost, or destabilized. Existing remedies either modify VLA internals, risking catastrophic forgetting, or demand online reinforcement under near-failure contact conditions. Both grant tactile unbounded influence over action generation, conflicting with the priors that make VLAs generalizable. We introduce ViTaR, which reframes tactile feedback from an action-generating perceptual input to an execution modulator that selects and scales bounded residual corrections atop a frozen VLA, preserving pretrained capabilities by construction. ViTaR decomposes adaptation into two stages: Effect-Guided Modeling determines whether and which correction is locally justified via outcome-grounded preference evidence, and Residual Action Modulation converts this evidence into a residual choice with continuously scaled gain from real-time visuotactile observations. On the UniVTAC benchmark spanning seven contact-rich tasks, ViTaR achieves 61.3% average success, a 30.6 percentage-point improvement over its frozen VLA base that also surpasses purpose-built tactile baselines. Physical-robot experiments confirm that bounded tactile modulation transfers to real sensor noise and dynamics.
Yi Wang, Renjun Wu, Jin-Yan Liu et al.· 0 citations
Embodied visuomotor models, including Diffusion Policy (DP) and Vision-Language-Action (VLA) models, have demonstrated promising performance on robotic manipulation benchmarks. However, their potential remains fundamentally constrained by the scarcity of large-scale embodied trajectory datasets, leading to insufficient compositional generalization in out-of-distribution (OOD) scenarios with limited capability to capture reusable skill structures. To address this limitation, we propose Skill-Based Memory (SkillMemo) framework that implicitly decomposes long-horizon demonstrations into latent atomic skills and integrates skill-level features into a dynamic episodic memory bank for solving compositional tasks. Specifically, we first introduce an expert-guided trajectory segmentation module built upon a Mixture-of-Experts (MoE) architecture, which implicitly partitions trajectories into distinct skill primitives represented by learned gating coefficients. We further design a skill-level episodic memory architecture that stores compact skill representations as retrievable key-value pairs. During inference, the memory bank retrieves the most relevant skill primitives which are subsequently fused with the model's current gating distribution, providing a robust contextual prior to refine action predictions. Extensive experiments on the simulation benchmark and real-world manipulation tasks demonstrate that SkillMemo consistently enhances both DP and VLA backbones, achieving state-of-the-art performance and outperforming $\pi_{0.5}$, while exhibiting strong compositional generalization to unseen task configurations.
Changyuan Wang, Chu-Bin Zhang, Zhen-Yu Wu et al.· 0 citations
To prevent the text-to-motion skill from exploiting shortcut pathways instead of learning language semantics, motion intuition distillation is introduced to ground text-to-motion generation in language semantics and a task-guidance module that dynamically adjusts actions based on high-level language instructions is introduced.
Yueyi Zhang, Ming Gong, Linpu He et al.· 0 citations
Vision--language--action (VLA) models acquire broad generalization through large-scale pretraining, yet adapting them to a new task and robot embodiment still requires post-training on newly collected data. Unlike pretraining, post-training targets task- and embodiment-specific adaptation, making it particularly sensitive to data quality. In practice, collected robot datasets often contain heterogeneous errors, including execution mistakes, sensor drift, and timestamp misalignment, which can impair post-training and policy performance. Manual inspection is costly, while existing data-cleaning methods are typically tailored to particular corruption types. To address these challenges, we introduce \textsc{RoboDrop}, a data-curation framework that audits supervision using local gradient compatibility measured along the training trajectory as a proxy for its effect on post-training performance. During a one-epoch warm-up run, RoboDrop scores each candidate sample online by comparing its gradient with those of task-semantic and visually matched validation samples. The resulting sample scores are aggregated at the episode level, and a simple automatic post-processing rule converts them into filtering decisions. We evaluate RoboDrop on controlled observation--action corruptions, naturally suboptimal demonstrations in simulation, and real-robot datasets containing non-expert collection errors. Across these settings, RoboDrop more accurately distinguishes unreliable demonstrations than prior methods, while post-training on the curated data consistently yields stronger downstream policies, with average real-robot rollout success rising from $35.0\%$ to $67.5\%$. These results establish training-trajectory-aware, context-conditioned supervision auditing as an effective approach to robust VLA post-training.
Runze Xu, Yuanfan Xu, Cui-Jie Xu et al.· 0 citations
A latent memory pool is constructed that stores failure cases along with their structure scene representations and expert trajectory labels, and a dedicated Retrieve Model that decouples static road structure and dynamic agent interactions to enable structurally grounded retrieval is designed.
Zebin Xing, Yupeng Zheng, Qiangyu Chen et al.· 0 citations
Learning to execute complex, multi-stage tasks requires skill representations that are both compositionally stable and adaptive in execution. Existing hierarchical approaches often face a fundamental trade-off: continuous skills suffer from representational drift due to unconstrained embedding boundaries, while discrete skills exhibit limited expressivity because their deterministic selection cannot capture the multi-modal nuances required for adaptive execution. This tension makes it difficult to achieve reliable composition and adaptive control within a single framework. To address this, we propose DualSkill, a hierarchical framework that learns stable hard skill primitives and builds adaptive soft skills from them. Specifically, DualSkill acquires discrete hard skills via vector quantization with motion-aware distillation, yielding robust and reusable motion primitives that provide structural anchors for skill composition. Conditioned on these primitives, soft skills are modeled as probabilistic continuous mixtures that adapt skill execution while preserving temporal consistency. DualSkill then predicts future skill intentions autoregressively and decodes them into precise low-level actions. We support DualSkill with both theoretical guarantees on its skill representation and extensive experiments across diverse simulation benchmarks and a real-world robotic platform, showing that it outperforms strong baselines and improves generalization. Note to Practitioners—This paper was motivated by the need for robots to execute complex, multi-step tasks in dynamic environments such as homes, warehouses, and factories. In practice, control systems often struggle to balance modular, reusable skills with smooth transitions, leading to unstable or inefficient behavior when task conditions change. Existing approaches typically force a trade-off: either continuous skills that suffer from representational drift or discrete libraries that result in inflexible behavior. This paper presents DualSkill, a hierarchical framework that bridges this gap by decomposing behaviors into stable hard skills for structural reliability and adaptive soft skills for smooth execution. We validate that DualSkill significantly reduces failure rates in complex manipulation tasks on both simulated benchmarks and physical robots. However, the system still relies on structured training data, which may limit its initial deployment in highly unstructured environments. In the future, DualSkill could be applied to mobile robots and human-robot collaboration, further leveraging its flexible and robust framework for real-world tasks.
Ziru Wang, Long Qian, Hao-Wen Sun et al.· IEEE Transactions on Automat...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
