Skip to content

Category

small language model

3,244 papers

#machine learning Preprint Sep 2026

The Dynamics of Continuous Mixture Collapse in Language Models

This work studies why pretrained language models often fail to preserve mixtures of many components and shows that exact preservation generally requires context-dependent correction, whose required dimensionality can grow with the number of components.

Ali Backour · 0 citations
#artificial intelligence Preprint Sep 2026

Language Models Can Control Their Own Attention

Declarative Attention is introduced, a protocol that elicits the model to declare where it needs to attend within its chain-of-thought, partitioning generation into three modes:(full context),(a specific region), and(recent output only).

Namgyu Ho, Huzama Ahmad, Woosung Koh et al. · 1 citation
#artificial intelligence Preprint Sep 2026

DeepAffinity: Long-Term Aspect Preference Prediction in eCommerce using Small Language Models

DeepAffinity is proposed, which leverages Small Language Models with structured prompts and specialized prediction heads fine-tuned for this task, and shows DeepAffinity outperforms standard generative fine-tuning methods, while general-purpose open-source LLMs perform poorly without task-specific tuning.

Yotam Eshel, Guy Hadad, Guy Feigenblat et al. · 0 citations
#artificial intelligence Preprint Sep 2026

SEAL: Reinforcing Global Safety in Mixture-of-Experts through Shared Expert ALignment

Mixture-of-Experts (MoE) is a scaling architecture for large language models that activates only a small subset of expert modules per token, enabling massive parameter growth with nearly constant computation. Recent Hybrid MoE architecture adds \textit{shared experts} to capture consistently useful representations, fur...

Qingyu Meng, Yiwei Zha, Jia-Huan Pei et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Door-in-the-Face Requests and Refusal Behaviour in Large Language Models

Does the door-in-the-face technique work on language models? In humans, a large request that is refused makes a smaller follow-up request more likely to be granted. We test this on nine production models from three providers: each model refuses a large request, then receives a smaller version of the same request, and w...

T. Jordan · 0 citations
#small language model Review Sep 2026

Diagnostic accuracy of parent reports for speech-language disorders in young children: a meta-analysis.

A systematic review and meta-analysis of studies evaluating the diagnostic accuracy of parent-report instruments for identifying speech and/or language disorders in children aged 0-6 years shows that several validated parent-report instruments demonstrate good accuracy for identifying children at risk, supporting their...

Selen Gündüz, Selçuk Güven · 0 citations
#small language model Open access Sep 2026

Quality Estimation for Machine Translation in Low-Resource Settings

Evaluating the quality of Machine Translation (MT) for low-resource languages is a challenging cross-lingual problem in Natural Language Processing. Quality Estimation (QE) aims to predict MT quality in the absence of reference translations. This thesis investigates QE through multilingual modelling, architectural adap...

Archchana Sindhujan · 0 citations
#small language model Open access Sep 2026

A Controlled Replication of a Self-Model Loop for Language-Model Agents: Controls, Confounds, and Finite-Horizon Path Dependence

An independent, clean-room reimplementation of a self-model loop architecture for language-model agents (the "AC1 loop" described in Lark Laflamme's 2026 AC1-LLM / Laflamme-3T essays). The architecture wraps an LLM with a Bayesian belief over the agent's own interaction stance, re-injected into generation each turn alo...

Grant Wilson · 0 citations
#small language model Open access Sep 2026

Memory Safety Where It Is Needed: Proof-Guided Runtime Checking in a Toolchain Small Enough to Read

Fil-C showed that C can be made memory-safe at run time by giving every pointer acapability and checking it on every access. We take its diagnostic ambitions and put themsomewhere else: in a compiler small enough for a person, or a language model, to hold inview at once, and with a different goal. Crust’s --mem-safe is...

Brent Hartshorn · 0 citations

From tech blogs

See all →
MIT News · Artificial Intelligence Oct 15, 2026

AI model decodes the language cells use to communicate

A new computational approach reveals how cells communicate to assemble into tissues. This could accelerate stem cell engineering, improve disease models, and inform therapies.

Microsoft Research Blog Oct 15, 2026

Proof-based general-purpose computing with Magnetar

Zero-knowledge proofs can verify computing results without rerunning workloads or exposing sensitive data. Magnetar helps bring them into the mainstream with support for standard programming languages, major speed gains, and quantum-resistant security. The post Proof-based general-purpose computing with Magnetar appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.