On-policy distillation (OPD) trains multi-turn language agents with dense teacher supervision on student-generated responses. However, standard think-then-act rollouts require lengthy reasoning before each short action, delaying environment transitions and experience collection. Generating actions directly reduces this...
Zubin Zheng, Jiahao Wu, Shaofeng Zhang et al.· 0 citations
This work proposes triadic linear attention, which writes the triadic outer product of a key, a second key, and a value, into a third-order tensor state, and reads from it by contracting both key axes with two queries.
Oliver Sieberling, Bharat Runwal, David Jin et al.· 0 citations
State Space Models (SSMs) compress sequence history into a bounded recurrent state, making the resulting memory law a central architectural choice for long-context performance. Most modern SSMs rely on ODE-based dynamics that lead to exponential forgetting, limiting their ability to retain information over broad tempor...
I. Kobyzev, Abbas Ghaddar, Ali Nasiri-Sarvi et al.· 0 citations
Sparse autoencoders (SAEs) expose interpretable features in large language model activations, yet existing structured SAEs impose single-parent trees or forests, while post-hoc graphs permit multiple parents but neither guide feature learning nor ensure reliable relation recovery. We introduce the Adaptive Graph Sparse...
Xiaozuo Shen, Yifei Cai, Tian Tan et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
ForUM is presented, a training-free test-time fusion of frozen MLLMs guided by two fixed geometric rules: agreement-based selection keeps the region supported by the most distinct models, and medoid localization returns an actual member box instead of a coordinate average, so one loose prediction cannot shift the answe...
Taiyo Sato, Takamasa Sanda, Keisuke Maeda et al.· 0 citations
Many localized large language model (LLM) unlearning methods select a small parameter subset from a localization signal and keep it fixed during optimization. The parameters most associated with a target, however, need not be the best ones to update, and candidate interventions can change value as optimization proceeds...
Tian-Hao Qian, Zi-Ming Hong, Chong-Yang Gao et al.· 0 citations
Language-model (LM) harnesses enable LMs to operate effectively over long contexts using additional compute. However, existing long-context evaluations are insufficient for distinguishing modern harnesses, reflected by saturated accuracy across harnesses and largely similar evaluation costs. In this paper, we introduce...
Large Language Models (LLMs) are increasingly used to enrich user queries in information retrieval (IR) so that a standard retriever such as BM25 can bridge vocabulary gaps with the target corpus. Any single LLM, however, is limited by its training data and architectural biases, and its enrichment behavior depends on h...
Tzu-I Ho, Yung-Yu Shih, Shang-Yu Su et al.· 0 citations
Language models tend to agree with whatever a user asserts, and post-training increasingly targets this sycophancy so that models evaluate claims on their merits rather than deferring to the user. Yet the same models are far more compliant when a wrong answer is attributed to a verified source, which is how retrieval r...
Transformer language models (LMs) are feed-forward: deep-layer representations are never fed back to shallower layers, and the only pathway for information to flow downward across generation steps is the decoded token. This narrow channel forces models to recompute intermediate results and to discard alternative contin...
This study delves into the notion of AI agentic architectures for autonomous data engineering pipelines and investigates the potential benefits of intelligent agents in enhancing automation, resilience, and decision-making processes in contemporary data ecosystems.
Harsh Verma· International journal of res...· 0 citations
Policy drift takes shape through a nonlinear differential equation - framed within the policy state space - with support from Lyapunov stability concepts alongside bifurcation methods alongside bifurcation methods, and the Intent Drift Rate appears: a concrete number per dialogue turn built as the time-based change in...
Harsh Verma· International Journal of Sci...· 1 citation
A new method, called CW-Net, translates the reasoning process of an autonomous vehicle’s AI system into understandable concepts that explain its behavior.
MIT News · Artificial Intelligence· news.mit.eduAug 31, 2026
With millions of users across the world, Julia has been used to conduct cutting-edge research and to design new drugs, jet engines, heat pumps, and more.
