Skip to content

Category

small language model

3,243 papers

#natural language process... Preprint Sep 2026

From Retrieval to Weights: Parametric Individualization of Small Language Models with Individual Text Corpora

We approach a cognitive simulation perspective on episodic and semantic memory in multiple-choice question answering by incorporating text from individual text corpora (ITC) into retrieval-augmented generation and DoRA fine-tuning. We web-crawl the search histories of 515 participants who answered 36 multiple-choice kn...

Christoph Wigbels, Ali Abusaleh, M. T. Jansen et al. · 0 citations
#natural language process... Preprint Sep 2026

Data-Centric Post-Training for Financial Reasoning: Mining, Distillation, and Verifiable Learning

This work presents a data-centric pipeline that constructs complementary corpora by mining open-source reasoning traces, distilling financial instruction data, and generating knowledge-graph-guided question-answer pairs from financial educational material and shows that retention-aware adaptation can improve financial...

Zhirayr Hayrapetyan, Andrei Kalmykov, Denis V. Kokosinskii et al. · 0 citations
#natural language process... Preprint Sep 2026

BuzzASR: A Swarm of 100+ Monolingual Speech Recognition Models

BuzzASR is introduced, a collection of language-specialized fine-tuned Whisper models adapted for automatic speech recognition (ASR) in 102 languages, which achieve state-of-the-art CER among open-source systems on 27 of 102 languages on the combined FLEURS and Common Voice test set.

Shivam Singh, Aditya Yadavalli, Catherine Arnett et al. · 0 citations
#natural language process... Preprint Sep 2026

Osprey: Target-agnostic Pre-training Makes Stronger Drafters in Speculative Decoding

Speculative decoding is critical for accelerating LLM inference. However, the speedup is fragile: drafters are typically trained against a narrow distribution for a single target model, and their acceptance rate collapses under workload shifts. This is a striking inversion of modern LLM development, where target models...

Fengxiang Bie, Yu-Qing Jian, Yi-Fan Yu et al. · 0 citations
#natural language process... Preprint Sep 2026

StochBench: A Domain-Specific Benchmark for Stochastic Processes in Lean

StochBench is introduced, a Lean 4 benchmark of 450 graduate stochastic-processes problems at varying abstraction levels, each paired with its natural-language source that better represents domain-specific applied mathematics while remaining challenging for advanced provers.

Idan Davidovich, Debargha Ganguly, Vikash Singh et al. · 0 citations
#machine learning Preprint Sep 2026

Privacy-Preserving Split Learning for Federated LLM Fine-Tuning

This work addresses leakage through a learned obfuscate-and-recover scheme that protects participants' private datasets while still allowing an independently deployable model to be trained on the server side, making split-based federated LLM fine-tuning practically viable.

Heng Jin, Chao-Yu Zhang, He-Xuan Yu et al. · 1 citation
#artificial intelligence Preprint Sep 2026

Show-Harness: Just a VLM Agent Can Play Robots

Show-Harness, an Embodied Harness that enables VLMs to"play" robots through a compact semantic interface linking intent to action, and develops GUMI (GUI Manipulation Interface), which extends the same semantic action space to GUI-based demonstration collection, allowing humans and agents to"play"robots across embodime...

Yan-Zhe Chen, Ze-Chen Bai, Zhi-Jun Cao et al. · 26 citations · ⚡2
#artificial intelligence Preprint Sep 2026

Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs

This work proposes Forgetting Only What Matters via Unlearning Layers (FOM-UL), a layer-level unlearning framework that selects transformer layers using a forget-to-retain significance score and provides an empirical path toward quantization-resilient unlearning.

Ravi Ranjan, O. Kotevska, Agoritsa Polyzou · 2 citations
#artificial intelligence Preprint Sep 2026

RiLM: Parameter-Efficient Language Modeling via Geodesic Decoding

Penn Treebank and a 10k-vocabulary stress test confirm that geodesic decoding transfers across corpora and larger |V|, while hyperbolic curvature helps selectively, and characterize boundary collapse in naive hyperbolic recurrence and show how Mobius stabilization restores trainability.

Fang Li · 0 citations
#artificial intelligence Preprint Sep 2026

When Auditors Fabricate: Batch-Size Degradation and Confident Hallucination in LLM Detection of Planted Document Contamination

It is concluded that LLM document auditing degrades not gracefully but deceptively, and the harness such systems require: bounded batch sizes, direct content injection, and mechanical verification of every reported finding against source text is outlined.

K. Parekh, Sanjana Pendyala Ravinder, Sana Mhapsekar et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Looped GPT-BERT: Trading Parameters for Computation in Small Language Modeling

When training data are limited, increasing parameter count is not the only way to improve language-model performance. A small parameter set, when repeatedly applied, can also deliver comparable performance. We study Looped GPT-BERT in the BabyLM 2026 Strict-small setting, combining GPT-BERT's masked next-token and caus...

Tingshuo Fan, Hong-Tao Mu, Tian-Yun Zhou et al. · 0 citations
#artificial intelligence Preprint Sep 2026

From Fixed Keys to Readable Schemas: Small Language Models for Vehicle Agent Function Calls

In-vehicle assistants must translate natural-language requests into accurate vehicle function calls under strict memory and latency constraints, making small language models (SLMs) attractive for on-device deployment. For such models, a key design choice is how the available function surface is presented. Two approache...

Hamed Jafarzadeh Asl, Yuanhao Yu, Vahid Partovi Nia · 0 citations

From tech blogs

See all →
MIT News · Artificial Intelligence Oct 15, 2026

AI model decodes the language cells use to communicate

A new computational approach reveals how cells communicate to assemble into tissues. This could accelerate stem cell engineering, improve disease models, and inform therapies.

Microsoft Research Blog Oct 15, 2026

Proof-based general-purpose computing with Magnetar

Zero-knowledge proofs can verify computing results without rerunning workloads or exposing sensitive data. Magnetar helps bring them into the mainstream with support for standard programming languages, major speed gains, and quantum-resistant security. The post Proof-based general-purpose computing with Magnetar appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.