Feng Lian

ER-JEPA: Experience Replay Improves Joint-Embedding Predictive Learning in Language Models

Large language models (LLMs) excel at token-level generation but may learn undesirable abstract semantics and lack comprehensive perception. LLM-JEPA mitigates this by aligning different views of the same underlying knowledge via a joint-embedding predictive architecture (JEPA). However, strong alignment does not neces...