Skip to content

Category

small language model

3,243 papers

#artificial intelligence Preprint Sep 2026

Edu-QuRating: Multi-Dimensional Educational Data Curation with Distilled Pairwise Judgements

Educational data filters have become a practical way to improve language-model pre-training, but most filters treat educational value as a single scalar property. This may be too broad for some applications, especially if the data set already features a high density of educational material. Useful learning material nee...

Oliver G. B. Garrod, Robin A. A. Ince, Meng Liu et al. · 0 citations
#artificial intelligence Preprint Sep 2026

In RAG We Trust? Measuring Robustness of Retrieval-Augmented Generation Under Document Poisoning

Retrieval-augmented generation (RAG) grounds a language model in retrieved documents, which reduces hallucination but creates a new attack surface: if retrieved text is tampered with, the model may repeat the falsehood. We study how much a small quantized model, Llama 3.1 8B, degrades when a fraction of its retrieved c...

Iliano Fasolino · 0 citations
#artificial intelligence Review Sep 2026

RAP: Research Attention Prediction Reveals Target-Conditioned Evidence Acquisition Biases

Research Attention Prediction (RAP), a rolling benchmark covering 278 AI/ML fields and 1,390 episodes, is introduced, a rolling benchmark covering 278 AI/ML fields and 1,390 episodes for large language models to track shifts in research attention.

Ying-Qian Wu, Jingcong Liang, Si-Yuan Wang et al. · 0 citations
#small language model Open access Sep 2026

Retrieval-Augmented Generation for Question Answering over a Research Paper: A Hybrid Dense-Keyword Retrieval Study on the Transformer Architecture Paper

Retrieval-Augmented Generation (RAG) combines information retrieval with language generation to improve the grounding of generated answers in a source document, addressing a central weakness of purely parametric large language models: the tendency to produce fluent but unsupported statements. This study designs, implem...

Dileep Choudhary · 0 citations
#small language model Open access Sep 2026

A Framework for Pluralistic Value Assessment (MindOrder)

We present a computational framework for multi‑dimensional value assessment, designed to address a structural problem shared by historical evaluation and AI value alignment: how to adjudicate among incommensurable values without collapsing them into a single score. Mainstream approaches to value aggregation seek a corr...

Wei Wan · 0 citations
#small language model Open access Sep 2026

GQLSA: Grouped-Query Latent Sparse Attention: A Hardware-Native Architecture for Efficient and Effective Sequence Modeling

We introduce Grouped-Query Latent Sparse Attention (GQLSA), a novel attention mechanism that integrates latent compression, grouped-query sharing, and block-sparse attention into a unified hardware-native architecture. GQLSA achieves linear O(T) complexity in sequence length, a 3.8x speedup over multi-head attention at...

Fardin Sabid · 0 citations
#small language model Open access Sep 2026

Not All 4-bit Quantizers Are Equal: Deployment-Time Mitigation of PII Leakage in Fine-Tuned Small Language Models (research artifact)

Research artifact of the paper Not All 4-bit Quantizers Are Equal: Deployment-Time Mitigation of PII Leakage in Fine-Tuned Small Language Models, published at SBSeg 2026 Main Track (full paper). Artifact evaluation (CTA/SBSeg 2026): this artifact was awarded all four seals: Available Artifacts (SeloD), Functional Artif...

Cristhian Kapelinski, Diego Kreutz · 0 citations
#small language model Open access Sep 2026

Structured PREreview of "Training Large Language Models for Small-Molecule Design with Synthetic Task Scaling"

This Zenodo record is a permanently preserved version of a Structured PREreview. You can view the complete PREreview at https://prereview.org/reviews/22662419. Does the introduction explain the objective of the research presented in the preprint? Yes The introduction explains the objective by setting up a gap and then...

Sneh Pankajbhai Vora · 0 citations
#small language model Open access Sep 2026

Judging the Judges: A Controlled Audit of Bias and Instability in LLM-as-Judge Evaluation

Large language models are now routinely asked to grade other large language models, a delegation of judgment that has become the default way organizations track progress on open-ended generation tasks where no single reference answer exists. The convenience of this arrangement is undeniable, but it does not remove the...

Shashwat Pandey · 0 citations
#small language model Open access Sep 2026

LLM and CQL computable phenotypes for myocarditis adverse event feature and outcome determinations using patient FHIR data: a feasibility study

Struct and unstructured FHIR data provided the computable phenotypes using an LLM with sufficient clinical information for myocarditis feature identification and outcome determination and enhanced CQL had excellent feature identification performance across most domains.

Fraser W. Gaspar, Seth Roberts, Francis Campion et al. · 0 citations
#small language model Dataset Open access Sep 2026

Fine-tuned and AWQ-quantized checkpoints for "Not All 4-bit Quantizers Are Equal"

Model checkpoints used by the artifact's second claim: the fine-tuned Qwen 0.5B checkpoint (wave 1, seed 42) and its AWQ 4-bit quantization. With these, a reviewer replays the claim instead of retraining. Data component of the research artifact of the paper Not All 4-bit Quantizers Are Equal: Deployment-Time Mitigation...

Cristhian Kapelinski, Diego Kreutz · 0 citations
#small language model Open access Sep 2026

PREreview of "Training Large Language Models for Small-Molecule Design with Synthetic Task Scaling"

This Zenodo record is a permanently preserved version of a PREreview. You can view the complete PREreview at https://prereview.org/reviews/22662611. Review of "Training Large Language Models for Small-Molecule Design with Synthetic Task Scaling" Summary This preprint asks whether LLMs can be post-trained for expensive,...

Sneh Pankajbhai Vora · 0 citations

From tech blogs

See all →
MIT News · Artificial Intelligence Oct 15, 2026

AI model decodes the language cells use to communicate

A new computational approach reveals how cells communicate to assemble into tissues. This could accelerate stem cell engineering, improve disease models, and inform therapies.

Microsoft Research Blog Oct 15, 2026

Proof-based general-purpose computing with Magnetar

Zero-knowledge proofs can verify computing results without rerunning workloads or exposing sensitive data. Magnetar helps bring them into the mainstream with support for standard programming languages, major speed gains, and quantum-resistant security. The post Proof-based general-purpose computing with Magnetar appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.