Skip to content

Category

small language model

2,971 papers

#artificial intelligence Preprint Sep 2026

The Misery of Mechanistic Interpretability: A Formal Perspective

Mechanistic interpretability has become the dominant lens for understanding frontier language models, as their inner workings are complex and inherently black boxes. To gain insights into these models, interpretable replacement networks (IRNs) are trained at all layers, exposing interpretable features through sparsely...

Tobias Ladner, Matthias Althoff · 0 citations
#artificial intelligence Review Sep 2026

A Conservative OCR-Enabled Workflow for R214 Sodium Screening of South African Packaged Foods

Using food package images to monitor sodium and salt content against South Africa's R214 sodium limits is challenging when screening decisions require product identity, nutrition facts panel evidence, reporting basis, and category-specific thresholds. This study presents a conservative image-based workflow that combine...

Mayimunah Nagayi, A. Khan, Tamryn Frank et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Mirror, Mirror on the Wall: Prompt Echoing in Small Instruct Language Models

Prompt echoing is a recognized failure mode of instruct language models, in which a model instead of generating a response, mirrors the provided prompt, even though it did not receive a specific instruction to do so. Is this phenomenon a sign of the model leaking the content of its training dataset, or is it rather cau...

Inez Okulska, B. Naskrȩcki, Jan Piotrowski et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Online Language Adaptive Sampling for Better Distributed Cross-lingual Gains

This work proposes a simple yet effective adaptive sampling strategy that assigns trainable sampling probabilities to each language and employs an inner-outer optimization loop with a small overhead, leading to consistent performance improvements and, more importantly, distributing the gains across languages.

Quang Phuoc Nguyen, Félix Gaschi, David Anugraha et al. · 0 citations
#artificial intelligence Preprint Sep 2026

TriCalRAG: A Three-Strategy, Retrieval-Augmented Benchmark for On-Premise LLM-Based Root Cause Analysis in AIOps

This work presents TriCalRAG, a benchmark evaluating open-weight LLMs served locally via vLLM on a single high-memory workstation GPU against a classical LSTM-based log anomaly detector (DeepLog), across four real, publicly available log datasets (BGL, HDFS, Thun-derbird, OpenStack).

Rohit Patel, S. K. Mohanty, Jeenal Chaudhary · 0 citations
#artificial intelligence Preprint Sep 2026

ECAS: An Edge-Controlled Agentic System for Validation-Gated Scientific Application Execution

Results show the feasibility of delegating adaptive reasoning to the cloud while retaining execution control at the edge, and closed-loop repair improves application success from 0/6 to 6/6 over one-shot generation, validation gating prevents all three observed target-scale failures, and skill conditioning improves suc...

Bai-Xi Sun, Ming-Ze Xia, Hui-Huo Zheng · 0 citations
#artificial intelligence Preprint Sep 2026

Bangla Sentence Function Classification: Corpus Development, Model Benchmarking, and Interpretability

A corpus of 10,000 Bangla sentences, manually annotated into four functional categories, namely declarative, interrogative, imperative, and exclamatory is introduced, and Experimental results show that TF-IDF consistently outperforms Word2Vec, likely due to its ability to emphasize discriminative lexical cues associate...

Swapnil Kundu Argha, Abdullah Al Shafi, Rowzatul Zannat et al. · 0 citations
#artificial intelligence Preprint Sep 2026

ReWeight: Leveraging Human Data for VLA Post-Training via Demonstration Retrieval and Sample Weighting

ReWeight is introduced, a framework that incorporates human data into VLA post-training through demonstration-level retrieval and sample-level weighting and learns a cross-embodiment visuomotor representation that combines visual observations with future actions to measure behavioral similarity between human and robot...

Chen-Wei Wang, Dian-Ye Huang, Match W. L. Ko et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Not all Negation Cues are Equal: Affixal Negations Yield Better Negation Understanding

Negation remains a longstanding challenge for both language models (LMs) and large language models (LLMs). Prior work mainly focuses on a small set of high-frequency single-word negation cues, such as not and never, with limited exploration of broader negation types and modern LLMs. To address this gap, we construct Ne...

Tian-Jue Tan, Eduardo Blanco · 0 citations
#artificial intelligence Preprint Sep 2026

Domain-Specific Jargon in Large Language Models: A Comparative Analysis between General-Purpose and Specialist Models

The results provide a case study in which a medically fine-tuned checkpoint does not improve jargon comprehension over its general-purpose counterpart, highlighting that domain adaptation should not be assumed to yield better performance on specialized terminology.

Darin Keng, Zhewei Sun · 0 citations
#artificial intelligence Preprint Sep 2026

How Many Thoughts Can a Vector Hold? The Capacity of Reasoning by Superposition

Large language models solve hard problems through intermediate computations across multi-step reasoning. Traditional chain-of-thought encodes these computations as tokens. Recent continuous and recurrent methods instead move partial computations into fixed-dimensional latent states, where a single thought can superpose...

Hong-Yuan Gu, Chang Liu, Jing-Wen Fu · 0 citations
#artificial intelligence Preprint Sep 2026

OrchSLM: Probing the Dynamics of Small Language Model Orchestration

OrchSLM is introduced, a routing framework that unifies existing non-interactive orchestration methods and exposes their underlying design choices as controllable parameters, and reveals how orchestration behavior emerges from diverse knobs, including the task structure, model-pool composition, and multi-agent consensu...

Cheng-Xi Zhang, Yu-Rong Yao · 0 citations

From tech blogs

See all →
Microsoft Research Blog Oct 12, 2026

Simulating student learning for better AI tutoring

StudentSim is a research system for evaluating and improving AI tutoring tools before they are tested with learners. It models how individuals may engage with tutors, giving feedback to improve tutor performance across subjects like chess, writing, and mathematics. The post Simulating student learning for better AI tutoring appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.