Massive activation features (MAs) in Transformers are extreme-value residual-stream features that persist across layers despite the model's ability to suppress them. Why do they survive? Our investigation using an operator-level mechanistic analysis of attention and feed-forward (FFN) blocks reveals that these blocks s...
Swagatam Mukhopadhyay, Vishal Vivek Saley, Vraj Parikh et al.· 0 citations
We describe the N\"urnberg NLP system for ChildSafeAds 2026. The shared task asks what a monitoring system for commercial content in child-facing YouTube videos can achieve at a given level of data access. We answer with per-subtask ensembles of nine voters, organised into three branches that differ in backbone, adapta...
Philipp Steigerwald, Eric Rudolph, Jens Albrecht· 0 citations
Answer-Convergence Stopping (ACS) is introduced, a training-free stopping rule that measures rather than asks, and reveals that by properly utilizing the output signals of frozen models, it can achieve favorable behaviors like adaptive stopping without the need for additional training.
Muath Alyobi, M. Eltahir, Almoayyad Abuljdail et al.· 0 citations
ReaLVR is proposed, which brings visual-evidence supervision to the model's own free-running latent trajectories, and is the first to scale visual reasoning in latent space, showing that the framework continues to deliver robust improvements at frontier model scales up to 235B.
Xi Xiao, Tian-Chen Zhao, Youngeun Kim et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Transformers process tokens without any inherent notion of order, making positional encoding a fundamental requirement rather than an architectural refinement. Rotary Position Embedding (RoPE) has become the default positional encoding in modern language models, yet it is heavily biased toward nearby tokens. Existing a...
Jarod Lévy, Mathurin Videau, Jad Yehya et al.· 0 citations
As large language models (LLMs) become increasingly widespread, preventing unsafe responses to harmful prompts is essential for their safe deployment. Activation steering offers an approach to improving LLM safety by modifying internal activations during inference without updating model parameters. However, a single pr...
Chen-Xi Wang, Rui-Yang Huang, Li Huang et al.· 0 citations
Tail-Corrected Top-$k$ On-Policy Distillation (TT-OPD) is proposed, which preserves the advantages of TK-OPD, including rich distributional supervision and low computational cost, while providing an unbiased estimator of the gradient of the reverse KL divergence.
Lin-Jian Meng, Si-Yuan Gan, Yu-Hang Li et al.· 0 citations
On-policy distillation (OPD) uses teacher correction on student-generated responses. Full-vocabulary correction can provide important corrections even for tokens that the student assigns low probability, but backpropagating through all token logits becomes memory-intensive for long sequences. Existing memory-saving app...
Yongliang Miao, Shuang Liu, Yan-Guang Liu et al.· 0 citations
The results show domain-specific changes associated with using recursive synthetic-data training in biomedical QA, but do not establish clinical hallucination rates or universal model collapse.
With the advent of word representations, word similarity tasks are becoming increasing popular as an evaluation metric for the quality of the representations. In this paper, we present manually annotated monolingual word similarity datasets of six Indian languages - Urdu, Telugu, Marathi, Punjabi, Tamil and Gujarati. T...
Syed S. Akhtar, Arihant Gupta, Avijit Vajpayee et al.· 0 citations
This work proposes RewardExplainer, a framework that obtains feedback from the target reward model through counterfactual rewriting and uses this feedback to further optimize the explainer, enabling the explainer to more faithfully capture the target RM's scoring preferences and sensitive behaviors than single-pass gen...
Jing-Yi He, Ni-Er Wu, Shuang Liu et al.· 0 citations
High-level text preprocessing is introduced: a systematic, rule-based intervention applied before the standard preprocessing pipeline to isolate each document's actual claim from its discursive structure and introduces the semantic diffusion index (SDI), a per-document metric for assessing the semantic reorientation be...
A new computational approach reveals how cells communicate to assemble into tissues. This could accelerate stem cell engineering, improve disease models, and inform therapies.
Microsoft Research Blog· microsoft.comOct 15, 2026
Zero-knowledge proofs can verify computing results without rerunning workloads or exposing sensitive data. Magnetar helps bring them into the mainstream with support for standard programming languages, major speed gains, and quantum-resistant security. The post Proof-based general-purpose computing with Magnetar appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduSep 24, 2026
A new method, called CW-Net, translates the reasoning process of an autonomous vehicle’s AI system into understandable concepts that explain its behavior.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
