In-context learning (ICL) enables language models to perform new tasks from demonstrations without weight updates. However, every ICL inference requires processing the full set of examples, resulting in inefficient deployments, and how ICL works mechanistically is not fully understood. Prior work compresses ICL into fi...
Guang-Zhi Xiong, Zheng-Hao He, Bo-Han Liu et al.· 0 citations
Masked diffusion language models (MDLMs) can generate text efficiently by predicting multiple masked tokens in parallel, but predictions from the same forward pass are not necessarily reliable when committed together. We study when parallel commitment is reliable. Our diagnostics show that confidence alone does not det...
Zheng-Hao He, Bo-Han Liu, Guang-Zhi Xiong et al.· 0 citations
This work proposes to view reasoning paths as phase-structured trajectories within fixed questions as PAIR, short for Phase-Aligned Intra-question Reasoning, and improves within-question trajectory ranking and Best-of-N trajectory selection across models and benchmarks.
Zheng-Hao He, Guang-Zhi Xiong, Sanchit Sinha et al.· 0 citations
It is suggested that CoT prompting activates specific latent features to trigger reasoning, and that targeted intervention on these features offers an alternative pathway to elicit efficient reasoning behavior without explicit CoT prompting.
Zhenghao He, Guangzhi Xiong, Bohan Liu et al.· 6 citations· ⚡1
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.