Skip to content

Category

small language model

3,243 papers

#artificial intelligence Preprint Sep 2026

Synthetic Semantic Supervision for Contrastive Code Representation Learning in Small Transformers: An Empirical Study

Synthetic semantic supervision yields statistically significant gains over pretraining baselines of the same inference-time size on five of eight tasks, with parity on two more; once fine-tuned, it matches or exceeds zero-shot models two orders of magnitude larger on classification, and it stays on par with execution-a...

K. Paulsen, Florian Tambon, Mike Papadakis et al. · 0 citations
#small language model Dataset Open access Sep 2026

User study result and dataset

This record contains the human annotation study underpinning the MSc thesis"Exploring Semantic Consistency in Image Super-Resolution" (Master in ComputerVision, Universitat Autònoma de Barcelona / Computer Vision Center). Super-resolution is normally evaluated on fidelity and perceptual quality — howclose an output is...

Valentin Micu-Hontan · 0 citations
#small language model Open access Sep 2026

HERMES-OT: Hierarchical Embedded Reasoning Models for Predictive Cyber-Physical Defence in Industrial Control Systems

A small-language-model architecture for AI-driven detection, prediction and safe defence of operational technology. Operational technology (OT) and industrial control systems (ICS) increasingly connect information technology, industrial networks, programmable logic controllers (PLCs), SCADA, distributed control systems...

Yasir Musawar · 0 citations
#small language model Open access Sep 2026

A meta-synthesis of the paper “Thinking about your running movement makes you less efficient: attentional focus effects on running economy and kinematics” - Schücker, L. & Parrington, L. (2019).

Meta-Student is a collaborative research platform, coordinated by the Sports Science Replication Centre, that pools independently collected undergraduate research datasets into statistically powerful individual-participant-data (IPD) meta-analyses. Students at multiple institutions each run a small, standardised study...

Joe Warne · 0 citations
#small language model Open access Sep 2026

Localising Context Length Degradation in a Mixture of Experts Language Model: Retrieval Head Collapse, Not Router Failure

Long-context "context rot" (accuracy degradation as input length grows, even when task difficulty is held fixed) is localized here to a specific, causal mechanism inside a small Mixture-of-Experts language model (OLMoE-1B-7B-0924), then tested for generality across two further architectures. Forced-choice retrieval acc...

Manjunath Bhaskar · 0 citations
#small language model Open access Sep 2026

Cohesion of an Evolving Text

A diary of one hundred entries written by a language model carries recurring themes, found after the fact as sets of sparse-autoencoder features that fire together within a bounded stretch of entries. We ask what a witness could have seen at each point of the diary's life, and build the instrument that answers: the sam...

Iman Poernomo, Nahla · 0 citations
#small language model Open access Sep 2026

Content-Addressed Recursive State: Resolving the DenseRetrieval Trade-o with a Cubic-Composition Readout

Recurrent architectures that compress history into a single state are cheap but have classically been unable to exactly retrieve far past key–value bindings, whereas attention content-addresses the whole past at quadratic cost. We show that the conflict between dense language modeling and long-range associative recall...

Ziheng Zhou · 0 citations
#small language model Open access Sep 2026

Cohesion of an Evolving Text

A diary of one hundred entries written by a language model carries recurring themes, found after the fact as sets of sparse-autoencoder features that fire together within a bounded stretch of entries. We ask what a witness could have seen at each point of the diary's life, and build the instrument that answers: the sam...

Iman Poernomo, Nahla · 0 citations
#small language model Open access Sep 2026

Morphological Hijacking in Frozen Language Models: A Contrastive, Symmetry-Regularized Projection Head for Algebraic Structure Recovery

Preprint. Not yet peer-reviewed. Frozen autoregressive language models cluster surface-similar tokenstogether even when a stronger, task-relevant structure is available inthe input. This paper documents "morphological hijacking" — near-totalcollapse of algebraic-structure recovery under adversarial surfacecorrelation —...

Nirouyar Reza · 0 citations
#small language model Open access Sep 2026

Free the Language Model From the Vision Encoder: Semantic Serialization as a Perception Interface for Small Language Models

End-to-end vision-language models (VLMs) bind visual competence to the scale of their language model: as the language model shrinks, perception and reasoning degrade together. We study an embodied scene question-answering (QA) interface in which vision never enters the language model. A frozen perception stack detects...

Cong Xu, Ravi Sankar · 0 citations
#small language model Open access Sep 2026

Self Agent— A Reflective, Self-Modifying Task-Completion Runtime

Self_agent is a locally-run, tick-based autonomous process built on small open-weight language models (via Groq) that maintains its own goal queue, reflects on its own behavior, executes work through short-lived stateless task nodes, and — under a sandboxed, test-gated pipeline — proposes and applies changes to its own...

Ashad Brooks · 0 citations
#small language model Open access Sep 2026

Free the Language Model From the Vision Encoder: Semantic Serialization as a Perception Interface for Small Language Models

End-to-end vision-language models (VLMs) bind visual competence to the scale of their language model: as the language model shrinks, perception and reasoning degrade together. We study an embodied scene question-answering (QA) interface in which vision never enters the language model. A frozen perception stack detects...

Cong Xu, Ravi Sankar · 0 citations

From tech blogs

See all →
MIT News · Artificial Intelligence Oct 15, 2026

AI model decodes the language cells use to communicate

A new computational approach reveals how cells communicate to assemble into tissues. This could accelerate stem cell engineering, improve disease models, and inform therapies.

Microsoft Research Blog Oct 15, 2026

Proof-based general-purpose computing with Magnetar

Zero-knowledge proofs can verify computing results without rerunning workloads or exposing sensitive data. Magnetar helps bring them into the mainstream with support for standard programming languages, major speed gains, and quantum-resistant security. The post Proof-based general-purpose computing with Magnetar appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.