A mechanistic intervention that reweighs attention away from the patch tokens in the identified lexical heads at test-time is performed and a test-time adaptive attention head routing is proposed to further reduce the impact on performance at matched robustness, pushing beyond the Pareto frontier of current robustness-...
Bo-Han Liu, Wenqian Ye, Guang-Zhi Xiong et al.· 0 citations
Chart reasoning agents are increasingly used to extract actionable insights in critical domains, achieving state-of-the-art performance on multiple benchmarks. Yet, high benchmark accuracy alone is insufficient for deployment, where stakeholders must be able to audit and verify how a model reaches its answer. Existing...
Sanchit Sinha, Oana Frunza, Kashif Rasul et al.· 0 citations
In-context learning (ICL) enables language models to perform new tasks from demonstrations without weight updates. However, every ICL inference requires processing the full set of examples, resulting in inefficient deployments, and how ICL works mechanistically is not fully understood. Prior work compresses ICL into fi...
Guang-Zhi Xiong, Zheng-Hao He, Bo-Han Liu et al.· 0 citations
This work introduces MemLife, a multimodal memory system that constructs entity-grounded, first-person text episodes and retrieves them via a time-indexed agentic reader and proposes MemOpt, a reinforcement learning framework that optimizes the memory writer to produce faithful, informative, and retrievable memories.
Guang-Zhi Xiong, Xin-Yuan Zhang, Xiao Yang et al.· arXiv.org· 0 citations
Masked diffusion language models (MDLMs) can generate text efficiently by predicting multiple masked tokens in parallel, but predictions from the same forward pass are not necessarily reliable when committed together. We study when parallel commitment is reliable. Our diagnostics show that confidence alone does not det...
This work proposes to view reasoning paths as phase-structured trajectories within fixed questions as PAIR, short for Phase-Aligned Intra-question Reasoning, and improves within-question trajectory ranking and Best-of-N trajectory selection across models and benchmarks.
It is suggested that CoT prompting activates specific latent features to trigger reasoning, and that targeted intervention on these features offers an alternative pathway to elicit efficient reasoning behavior without explicit CoT prompting.
Zhenghao He, Guangzhi Xiong, Bohan Liu et al.· 6 citations· ⚡1
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.