The framework and EvalAwareBench provide the tools to measure, attribute, and mitigate evaluation awareness, building the foundation for future solutions to ground evaluation awareness in social psychology.
Changling Li, Terry Jingchen Zhang, Jie M. Zhang et al.· arXiv.org· 1 citation
PRISM, a closed-form upper bound on the cross-entropy risk gap between a target model and a proxy variant, is derived and it is proved that it splits exactly into three measurable axes: scale, shape, and head.
Adaptive optimizers such as Adam are standard for training Transformers, but storing gradient first and second moments incurs substantial memory overhead. We introduce PowerStep, a memory-efficient optimizer that achieves coordinate-wise adaptivity without storing second-moment statistics. Motivated by $\ell_p$-norm st...
Yao Lu, Dengdong Fan, Shixun Zhang et al.· 0 citations
GRAVITY (Generation-time Relational Anchoring Via Injected Topological MemorY), a host-independent auxiliary memory layer, is introduced and shows that generation-time structure is a portable complement to existing memory retrieval, while its interaction with host evidence depends on the benchmark and host.
Chain-of-thought (CoT) distillation transfers reasoning behaviors from a strong teacher to a smaller student, but prior work reports a capacity gap: distillation may fail when the teacher-student capability mismatch is large. We revisit the capacity gap from a practical perspective by re-examining commonly used experim...
T. Kajitsuka, Ukyo Honda, Sho Takase· arXiv.org· 2 citations
Screening is introduced, whose explicit threshold transforms bounded query--key similarities into relevance values, enabling exact rejection, empty selection, and direct inspection on a common scale, in a controlled comparison of 12 attention mechanisms on a matched Transformer backbone.
Accurate confidence estimation is critical for reliable use of large language models (LLMs). Prior work on LLM calibration largely focuses on response-level confidence, which estimates the correctness of a single generated output. However, this formulation is misaligned with many practical settings where the central qu...
This work proposes a family of symmetric multi-player alignment games and proves that any symmetric Nash equilibrium policy of the $(k+1)-player alignment game achieves the optimal $(k,\frac{k}{k+1})$-robust alignment.
This work proposes an approach that recasts the task as automated knowledge base construction: prompting an LLM to synthesize a 3NF-compliant relational schema from Wikipedia infobox timelines, populating the schema to obtain a queryable database, and generating and executing SQL queries against it, with QA accuracy se...
In-context learning is governed by both temporal and semantic relationships, shaping how Large Language Models (LLMs) retrieve contextual information. Analogous to human episodic memory, where the retrieval of specific events is enabled by separating events that happened at different times, this work probes the ability...
Anooshka Bajaj, Deven Mahesh Mistry, Sahaj Singh Maini et al.· 0 citations
A novel threat is unveiled in which attackers steer the RAG system's response by injecting malicious passages into its knowledge base, enabling the attacker to steer the response without altering the user input or modifying the RAG weights.
Jiaqi Xue, Meng Zheng, Yebowen Hu et al.· arXiv.org· 109 citations· ⚡8
The Hitchhiker's Guide to Agentic AI is a comprehensive practitioner's reference for building autonomous AI systems, covering the full stack from first principles to production deployment. The central thesis: building great agentic systems requires understanding every layer of the pipeline, not just one. The book opens...
Flint-interactive gives AI agents a language for creating interactive visualizations. Click a chart, filter data, highlight related content, and explore beyond the view the author originally chose. The post Flint-interactive: A language for creating interactive data visualizations appeared first on Microsoft Research.
A new computational approach reveals how cells communicate to assemble into tissues. This could accelerate stem cell engineering, improve disease models, and inform therapies.
Microsoft Research Blog· microsoft.comOct 15, 2026
Zero-knowledge proofs can verify computing results without rerunning workloads or exposing sensitive data. Magnetar helps bring them into the mainstream with support for standard programming languages, major speed gains, and quantum-resistant security. The post Proof-based general-purpose computing with Magnetar appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduSep 24, 2026