Skip to content

Category

small language model

2,968 papers

#artificial intelligence Preprint Sep 2026

Beyond the Name: Demographic Leakage in De-Identified R\'esum\'es and Evaluation Artifacts in LLM Bias Audits

De-identified r\'esum\'e screening assumes that redacting explicit fields prevents ethnocultural inference; however, recent audits attribute residual leakage to declared languages. We investigate whether eliminating language fields resolves this leakage across nine open-weight models and 620 counterfactual r\'esum\'es....

Qiang-Ju Chen, Xiao Yang · 0 citations
#natural language process... Preprint Sep 2026

Register Tokens for Bounded-State Reasoning in Diffusion Language Models

Masked diffusion language models (dLLMs) generate text by iteratively denoising masked tokens with bidirectional attention. Extending reasoning across generation chunks normally requires keeping earlier generated text in context. We ask whether a dLLM can instead continue reasoning after that text is cleared, using onl...

Albert Ge, C. Singh, Yu-Fan Zhuang et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Safe Error Correction for Language Models: Frozen-Base Adjustment with Capability Preservation

We study a practical question: can a small correction module fix errors in a frozen language model's outputs without degrading its base capabilities? We propose CRN v2, a lightweight logit-level correction module (~34M trainable parameters, 0.73% of the 4.65B text module) that sits atop a fully frozen Gemma 4 E2B model...

G. Kishore · 0 citations
#artificial intelligence Preprint Sep 2026

Coupled Calibration and Learning: Mitigating Teacher Bias in LLM Distillation without Target-Domain Reward Feedback

Coupled Calibration and Learning (CCL) is proposed, an LLM distillation algorithm that couples teacher calibration with student updates through token-level branching, using reward feedback only on source questions.

Hai-Chen Hu, Yu-Heng Zhang, David Simchi-Levi · 0 citations

Efficient Reasoning Distillation: Small Video-Language Models via Synthetic CoT and Difficulty-Aware Fine-Tuning

We present an efficient method to distill reasoning capabilities into compact video-language models (VLMs) for video question answering (VideoQA). Our approach fine-tunes a 2B-parameter model using only $\sim$900 uncertainty-selected examples, each augmented with synthetic chain-of-thought (CoT) rationales generated by...

Mantek Singh, Jeshwanth Challagundla, Siddharth Raina et al. · 0 citations
#small language model Review Sep 2026

Detecting and Localizing Segment-Level Poisoning in Multi-Source LLM-Agent Inputs

ActProbe, an internal-state-based framework for detecting and localizing poisoned segments in multi-source LLM inputs, is proposed and remains effective against defense-aware adaptive attacks and can protect black-box APIs through surrogate-based poisoned-segment removal.

Xue Tan, Chang-Hui Wang, Sanrui Yang et al. · 0 citations
#small language model Preprint Sep 2026

Two-Stage Mixture-of-LoRA for Multi-Task Medical Vision-Language Learning

The proposed framework uses a shared-specific Mixture-of-LoRA architecture comprising one shared LoRA and six task-specific expert LoRAs, together with a two-stage training procedure, and achieves good results in the FLARE 2026 Task 3 test sets.

Zhang-Hao Chen, Yuan-Yuan Li, Zhen-Yu Lu et al. · 0 citations
#small language model Open access Sep 2026

Fast and accurate taxonomic domain assignment of short metagenomic reads using BBERT

Testing on a large cohort of soil metagenomes, it is found that BBERT identifies bacterial sequence syntax without relying on reference databases, enabling accurate assignment of taxonomic domain, coding potential, and reading frame directly from reads as short as 100 bp.

Dmitry Alekhin, Moshe Alon, Tomer Sidi et al. · 0 citations
#large language models Open access Sep 2026

Carrier Current Sensing for Computation-Level Bias Detection in Large Language Models: A Methodological Manual

This is unedited work with Mistral Vibe. Quality/logic checked through multiple different instance reviews. Proceed with caution. This manual presents a novel methodology for detecting algorithmic bias in large language models (LLMs) by analyzing GPU power-line current fluctuations during inference. Unlike existing bia...

Matty Hilliard · 0 citations
#small language model Open access Sep 2026

Sub-Two-Bit Ternary Weight Storage with Table-Free Arithmetic Decoding for CPU Inference

Ternary (1.58-bit) language models are stored in practice at two bits per weight, 26% above the information content of a ternary symbol. The code that removes that redundancy is not new: five ternary values as the base-3 digits of one byte at 1.600 bits per weight is llama.cpp's TQ1_0 and Spectra 1.1's TQ1; the depth-o...

Justus Theile · 0 citations
#small language model Open access Sep 2026

Pre-Alpha PP-OCRv6 (medium) HTR model for medieval/pre-modern Ethiopic (Geʽez, some Amharic) manuscripts

Pre-Alpha Ethiopic (Geʽez / Amharic) recognition model — PP-OCRv6 medium fine-tune Status: preliminary / pre-Alpha. This deposit is an early public checkpoint, not a finished production model. Better versions are planned as soon as additional medieval and pre-modern Ethiopic ground truth can be annotated and curated. H...

Teodor Bors · 0 citations
#small language model Open access Sep 2026

Agentic AI for Livestock Housing Management: Applications, Benchmarking, and Readiness Assessment

Agentic artificial intelligence is emerging as an extension of Precision Livestock Farming by linking perception, reasoning, planning, and bounded action within human-supervised livestock-housing workflows. This review synthesizes 90 publications on agentic AI, multi-agent systems, retrieval-augmented generation, large...

Alexey Ruchay, Hao Guo, Andrea Pezzuolo · 0 citations

From tech blogs

See all →
Microsoft Research Blog Oct 12, 2026

Simulating student learning for better AI tutoring

StudentSim is a research system for evaluating and improving AI tutoring tools before they are tested with learners. It models how individuals may engage with tutors, giving feedback to improve tutor performance across subjects like chess, writing, and mathematics. The post Simulating student learning for better AI tutoring appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.