Skip to content

TomasuLLM: Out-of-Order Speculative Execution for LLM Agents

Sep 2026 · arXiv.org · Vol abs/2609.38201 · 0 citations · 37 references
Computer Science

TL;DR

TomasuLLM is presented, a runtime that executes agent tool calls out of trajectory order while preserving task-execution correctness, and improves the reported benchmark means and scales with tool latency.

Abstract

Long-running tools can dominate coding-agent latency: compilers, test suites, and repository commands take seconds to minutes while the agent idles. This observation stall presents the same tension that drove out-of-order processors -- asequential interface hides work that can be predicted and started early, but a speculative result may become visible only after it and every earlier step have been validated. We present TomasuLLM, a runtime that executes agent tool calls out of trajectory order while preserving task-execution correctness. It drafts future actions, runs them in isolated copy-on-write sandboxes, traces their dependencies and effects, and commits results in trajectory order only after validation against committed state. Across three benchmarks spanning sub-second to minutes-long tool calls, TomasuLLM improves the reported benchmark means and scales with tool latency: 1.31x on 100 SWE-bench Verified tasks, 1.35x on 28 Terminal-Bench 2.0 tasks, and 1.27x matched progress on 18 SWE-Marathon sessions. Across 4,010 audited commit-validation records, it produces zero false accepts.

View source

Similar papers

Preprint Sep 2026

JustFit: Just-in-Time State Management for Local LLM Serving

JustFit is presented, an MLX runtime that coordinates their overlapping allocations: KVExec executes and checkpoints four-bit KV with bounded workspace, PhaseSwap loads phase-dependent components, and StateTrans preserves history across execution modes.

Yu-Hua Chen · 0 citations
Preprint Sep 2026

UNISON: A Co-Designed Near-Memory Scheduler of Session KV Residency for LLM Agents

This work presents Unified Native Inter-turn Session Orchestration Nexus (UNISON), an event-driven near-memory scheduler in which Survival-Penalty Eviction for Agent Return-gap and Tiering in Idle-window DMA Events share one live ranking.

Fan He, Yan Li, Xiao-Yang Zeng · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.