Skip to content

Category

natural language processing

7,304 papers

#machine learning Preprint Sep 2026

Exploring Heterogeneous Model Merging Approach for Complex Knowledge Transfer

Specialized models encode task-oriented behavior, but transferring that behavior to a general language model usually requires training, distillation, or representation alignment. We study whether such ability can instead be transferred directly at the parameter level. We apply two existing training-free heterogeneous m...

Jia-He Fan, Si Chen, Ying-Hao Hou et al. · 0 citations
#machine learning Preprint Sep 2026

Concept Subspaces Compute Beyond the Logit Lens: A Weights-Only Test for Locating Representations Upstream of Readout

A concept subspace's effect on model behavior does not establish how it relates to the output readout. We introduce a two-sided geometric diagnostic that measures an extracted subspace's overlap with the dominant right-singular directions of the unembedding matrix, evaluated against output-oriented positive controls. G...

Ao-Jie Yuan, Z. Su, Hai-Yu Zhang et al. · 0 citations
#machine learning Preprint Sep 2026

RSIGame: Autonomous Agentic Game Development with Recursive Self-improvement

Recent advances in large language models have made automatic game generation increasingly feasible, yet reliably improving generated games beyond a playable version remains challenging. Naive iterative refinement can easily overfit a small set of test cases, producing fragile games with unresolved bugs, missing behavio...

Wen-Yi Wu, Ming-Hao Fu, Jie-Yu You et al. · 0 citations
#machine learning Preprint Sep 2026

VOSSA: Voiceprint Optimization for Streaming Speech Architectures

Real-time voice conversion (VC) systems commonly rely on pretrained speaker embeddings from automatic speaker verification (ASV) models. While effective for speaker discrimination, these embeddings are trained to remain stable across phonetic and prosodic variations within-speaker, which may conflict with frame-level a...

Mu-Ruei Tseng, Waris Quamer, Ghady Nasrallah et al. · 0 citations
#machine learning Preprint Open access Oct 2026

Can Terminal Agents Trust Their Own Verification? Diagnosing and Improving Self-Verification

Terminal agents rely on self-verification to assess and correct their solutions as they solve tasks through interaction with command-line environments. Yet how trustworthy such self-verification is remains poorly understood. To investigate this question systematically, we introduce a diagnostic framework that identifie...

Yingfeng Luo, Shaowei Wei, Daixin Wang et al. · 0 citations
#machine learning Preprint Sep 2026

Recovering Off-Policy Supervision for Speculative Decoding

Block drafters for speculative decoding are commonly trained on corpora written by external models, where a single off-policy token invalidates supervision for all subsequent slots in a block. Existing approaches discard these divergent slots, resulting in severe supervision loss. To resolve this problem while preservi...

Jungseob Lee, Chanjun Park, Sugyeong Eo et al. · 1 citation
#machine learning Preprint Sep 2026

Training LLM Judges from Language Feedback via Position-Selective Self-Distillation

We study training LLM judges from natural language feedback, especially for subjective tasks where the verdict depends strongly on which evaluation criteria the judge invokes and how it weighs them. The dominant approach, outcome-supervised RL (e.g., GRPO), credits every token in the rollout with a single scalar determ...

Ilgee Hong, Chang-Long Yu, Zhenghao Xu et al. · 0 citations
#machine learning Preprint Sep 2026

MedKIT: Evaluating Knowledge Integration and Generalization in Large Language Models

Constantly evolving real-world knowledge necessitates models to be updated continuously. Especially in medicine, as clinical evidence changes over time, outdated knowledge can pose safety risks. Existing evaluations of knowledge integration focus on factual recall, offering limited insight into whether newly integrated...

Lukas Thede, Yash Kumar Atri, David Chen et al. · 0 citations
#machine learning Preprint Open access Oct 2026

Shifting Mechanisms: How Positional Encoding Choice Shapes In-Context Retrieval

Language models increasingly use architectures that vary attention span and positional encoding across layers, such as applying RoPE with sliding-window attention and NoPE with global attention (SWA NoPE). However, how these choices shape in-context retrieval remains unclear. To study this question, we take a mechanist...

Eric Enouen, Sainyam Galhotra · 0 citations
#machine learning Preprint Sep 2026

Doc2LoRA Provides Decodable Representations of Scientific Ideas

Representing scientific papers as points in a space lets us search for similar papers and inquire about how fields relate to one another and drive innovation. Beyond search, the vector space of papers invites generation: mixing papers through simple vector operations creates new points, mirroring combinatorial novelty,...

Chand Sahil Mansuri, Joel Zachariah, Sadamori Kojaku · 0 citations
#machine learning Preprint Open access Oct 2026

Which Models Work Well Together? Measuring Heterogeneity for LLM Team Selection

The performance ceiling of an LLM team is constrained not only by individual model capabilities, but also by inter-member error resonance and predictive differences. Although heterogeneous teaming is often observed to be effective in practice, existing approaches lack complementarity metrics that are computable, interp...

Liangyu Teng, Hengsong Liu, Juncen Guo et al. · 0 citations
#machine learning Preprint Open access Oct 2026

Conformal Factuality Control for Multi-Hop Retrieval-Augmented Generation

Retrieval-augmented generation (RAG) can ground large language models in external evidence, but retrieved context does not guarantee that generated claims are factually supported. This problem is especially relevant in multi-hop RAG, where retrieval and reasoning proceed through multiple dependent stages. We study whet...

Muhammad Aimal Rehman, Chi-Kuang Yeh · 0 citations

From tech blogs

See all →
MIT News · Artificial Intelligence Sep 24, 2026

Estimating suicide risk from text

A new language-processing tool could help identify the highest-risk individuals from natural language, enabling swifter interventions.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.