Skip to content

LRC-JEPA: Disentangling Dynamics and Residual Context for Efficient World Models

Sep 2026 · arXiv.org · Vol abs/2609.34375 · 1 citation · 41 references
Computer Science

TL;DR

This work introduces LRC-JEPA, a lightweight end-to-end world model that routes information into a compact predictive latent and learned-query residual-context embeddings and shows that the resulting representation is sufficient, minimal, nuisance-invariant, and disentangled.

Abstract

Compact JEPA world models enable efficient latent-space planning, but low-dimensional representation trained under reward-free self-supervision must encode both action-conditioned dynamics and predictable visual context. This competition can entangle controllable state with high-rank nuisance appearance and degrade planning as scenes become more complex. We introduce LRC-JEPA, a lightweight end-to-end world model that routes information into a compact predictive latent $\mathbf{z}$ and learned-query residual-context embeddings $\mathbf{u}$. Only $\mathbf{z}$ is propagated by the dynamics model and used for planning, while $\mathbf{u}$ captures temporally persistent information for cross-attention reconstruction; a differentiable residual connection encourages the latent to retain complementary dynamic content. Under explicit assumptions, we show that the resulting representation is sufficient, minimal, nuisance-invariant, and disentangled. Across four simulated control environments, LRC-JEPA improves average planning success over a parameter-matched JEPA baseline by 9 percentage points and matches or exceeds substantially larger pretrained models. On the real-world Bridge-v2 set, its 5.5M-parameter active encoder outperforms DINO-WM (22.1M) and V-JEPA2 (303.9M) encoders while also enabling faster planning. Physical-state probes, reconstruction interventions, and ablations confirm the effectiveness of LRC-JEPA's representation disentanglement.

View source

Similar papers

Preprint Aug 2026

Diagnosing JEPA World Models with Action-Conditioned Predictive Consistency

Action-Conditioned Predictive Consistency (ACPC), a diagnostic that measures how far a clean history and a visually perturbed view of it diverge after being rolled forward under the same action sequence, is introduced and it is proved that this divergence bounds the perturbation-induced change in multi-step prediction...

Guo An, Zijing Wu, Hongzhuang Dong et al. · 3 citations
Preprint Aug 2026

Flow-JEPA: Robust Latent Dynamics for JEPA World Models via Flow Matching

Joint-Embedding Predictive Architectures (JEPAs) provide a powerful framework for latent world modeling and planning in a reconstruction-free manner. Although numerous JEPA-based approaches have been proposed to mitigate representation collapse, our experiments on localized, out-of-distribution visual noise reveal that...

Yan-Chen Huo, Zi-Ying Song, Yadan Luo · 0 citations
#artificial intelligence Preprint Aug 2026

No Gaussian Required: Contrastive Inverse Dynamics for JEPA World Models

Contrastive inverse dynamics thus provides a distribution-free anti-collapse signal that requires no target network, stop-gradient, pretrained encoder, or reconstruction objective, and it is argued that the anti-collapse pressure can instead come from the transition data itself.

Jack Boylan, Chris Hokamp · 6 citations
#machine learning Preprint Aug 2026

Orthogonal JEPA: Factorized Predictive States for Latent World Models

Method, a latent world-modeling framework based on orthogonal predictive factorization, is introduced, a latent world-modeling framework based on orthogonal predictive factorization that can be used by a readout, decoder, planner, or autoregressive rollout of an underlying system.

Tao-Yong Cui, Pheng-Ann Heng, Wan-Li Ouyang · 0 citations
Preprint Aug 2026

JEPA-WAM: Learning Vision-Language-Action Policies with Joint-Embedding World Modeling

JEPA-WAM, a latent WAM built in a pretrained V-JEPA space, which couples latent transition prediction with continuous action generation through a shared predictor, predicts a spatially structured joint current-future target that captures task-shared visual temporal structure between current and future observations, whi...

Yi-Han Lin, Jia-Wei He, Shi-Feng Bao et al. · 12 citations · ⚡1
#machine learning Preprint Aug 2026

JEPA-x: Cross-Predictive Physics Grounding for Forecastable Latent Dynamics

This work introduces the cross-predictive JEPA (JEPA-x), which grounds latent dynamics in privileged physical trajectories, and shows that direct physical-state regression improves decodability without improving forecastability or control, indicating that the benefit comes from shaping latent dynamics rather than merel...

Kehan Wen, Ziming Li, Si-Yuan Luo et al. · 2 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.