Skip to content

Category

small language model

3,243 papers

#machine learning Preprint Sep 2026

Chimaera: A Mixture-of-Graph-Experts Architecture for Cross-Task and Cross-Dataset Graph Learning

Chimaera integrates mixture-of-experts with graph foundation models (GFM) and integrates different GFM architectures, such as graph prompts and linear GNN models, to extend existing linear GNNs to support link-level and graph-level tasks in addition to node-level tasks.

John P. Frank, David Richerby, A. Scherp · 0 citations
#machine learning Preprint Sep 2026

Dense Structural Compression of Transformers via Gauge-Correct Channel Removal

Inference energy per token drives the cost and carbon footprint of deployed transformers. It is dominated by dense matrix products that incur fused multiply-accumulate (FMA) operations and memory traffic. To reduce these computations while retaining dense tensors for high GPU throughput, we develop a methodology from f...

Jed A. Duersch, Na\"im Es-Sebbani, Nathana\"el Haas et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Do New Attention Mechanisms Actually Fix Attention Sinks at Million-Token Context?

SinkProbe is built, a suite that measures sink mass, massive activation, position resolved recall and the recency gap, and applies it to four small models that differ only in how they mix tokens and depth, and three results follow.

Sara Rizwan, Samaanah Abdus Salam, Mohammed Mudassir Uddin · 0 citations
#artificial intelligence Preprint Sep 2026

IPM-FM: A Foundation Model with Consensus Feature Selection for Industrial Process Monitoring

The industrial process monitoring foundation model (IPM-FM) first learns general-purpose representations from unlabeled industrial process data through self-supervised pretraining, then adapts to specific monitoring tasks using a small amount of task-labeled data, and finally produces calibrated predictions through an...

Liang Cao, Wei-De Liu, Yan Qin et al. · 0 citations
#artificial intelligence Preprint Sep 2026

A*-Thought-V2: Efficient Latent Reasoning via Geometric Dynamics of LLM

A*-Thought-V2 is presented, a geometric dynamics of LLM guided framework that models CoT as a hidden-state trajectory and replaces hard deletion with an explicit-implicit interleaved latent architecture that reflects broader soft targets that encourage richer step-level feature learning.

Xiao-An Xu, Si-Yuan Liu, Shuo Wang et al. · 1 citation
#artificial intelligence Preprint Sep 2026

LoGAN: Multilingual Font Localization with Generative Agents

LoGAN, a VLM-based agentic framework for few-shot multilingual font localization, which takes in a small number of individual glyphs from a font or letters from a logo and uses them to generate complete character sets in other languages, achieves broad language coverage for font localization with various styles.

Zhuoning Yuan, Ta-Ying Cheng, Benjamin Klein · 0 citations
#artificial intelligence Preprint Sep 2026

Steering Interference Reflects the Model's Defaults, Not the Behavior Directions

Activation steering promises modular control of language model behavior: a behavior such as politeness corresponds to a direction in a model's activations, and adding that direction while it generates should switch the behavior on and leave everything else alone. It does not. We ask what decides which other behaviors m...

Srikanth Malla, Chi-Ho Choi, J. Choi · 0 citations
#artificial intelligence Preprint Sep 2026

The Geometry of Refusal: Why Post-Hoc Safety Is Fragile and Pretraining-Time Safety Persists

Models trained from scratch with safety co-training spread continuously across pretraining reach 87 to 98% AdvBench refusal that the same attack erodes by only 2 to 14 pp at every scale from 410M to 6.9B, at a small cost on short-answer capability probes.

Srikanth Malla, Chi-Ho Choi, J. Choi · 0 citations
#artificial intelligence Preprint Sep 2026

Counterfactual Tests for Measuring Chain-of-Thought Faithfulness in Visual Language Models

The analysis shows that CoTs do not reliably track visual evidence that influences model predictions, and it is found that Predict-then-Explain explanations align more strongly with perturbation-induced probability shifts than pre-answer CoTs, while binary vCT scores are often nearly saturated.

Bayar Menzat, Max Süss, Rui-Zhi Wang et al. · 0 citations

From tech blogs

See all →
MIT News · Artificial Intelligence Oct 15, 2026

AI model decodes the language cells use to communicate

A new computational approach reveals how cells communicate to assemble into tissues. This could accelerate stem cell engineering, improve disease models, and inform therapies.

Microsoft Research Blog Oct 15, 2026

Proof-based general-purpose computing with Magnetar

Zero-knowledge proofs can verify computing results without rerunning workloads or exposing sensitive data. Magnetar helps bring them into the mainstream with support for standard programming languages, major speed gains, and quantum-resistant security. The post Proof-based general-purpose computing with Magnetar appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.