Skip to content

The Price of Thinking: Reasoning Effort as a Model-Specific API Contract

Aug 2026 · 0 citations · 1 references
Computer Science

TL;DR

The reasoning-effort term is studied through a registered paired contrast of Sonnet 5 with explicit high effort against the same model with effort omitted, using 30 AIME 2026 items and five calls per item.

Abstract

API buyers purchase a dated contract, not a model name alone: the contract includes the requested and served model, reasoning-effort term or its omission, output rail, service product, prompt, and price schedule. We study the reasoning-effort term through a registered paired contrast of Sonnet 5 with explicit high effort against the same model with effort omitted, using 30 AIME 2026 items and five calls per item. Every paid attempt was assigned one frozen terminal category, and inference resampled items while retaining their repeated calls. Mean delivered cost was \$0.01031 per call higher under the explicit-high contract than under the omitted contract [+\$0.00204, +\$0.01974]. The corresponding accuracy contrast was +0.0133 [-0.0267, +0.0467]; we did not detect an accuracy difference, and the interval permits a gain of up to 4.67 percentage points that this design cannot rule out. Cost per correct answer was \$0.08665 under the high-effort contract and \$0.07662 under the omitted contract, as registered point estimates. A dated contract census, Models-API metadata, and preregistered raw-response probes further documented model-specific omission semantics, including within a provider; claims remained at documentation grade when raw structure was indeterminate. The request registry, parser, terminal taxonomy, statistical plan, and analysis pipeline were frozen before outcomes were examined; the resulting claims are bounded to the model, task, and collection date studied.

View source

Similar papers

#artificial intelligence Preprint Aug 2026

Thinking Costs Tokens: When More Structure is Worth the Price

This paper investigates whether there exists a token-budget threshold, below which the overhead of planning and verification hurts performance and above which it helps, and evaluates two systems on FinQA and TAT-QA financial reasoning tasks.

Thomas Nolasque, J. Grey, Calista Pham et al. · 0 citations
Preprint Jul 2026

Alipay-PIBench: A Realistic Payment Integration Benchmark for Coding Agents

Payment integration is a demanding repository-level software task: agents must select a suitable product, implement coordinated client-server flows, verify payment outcomes, and preserve consistency between transaction and business states. We introduce Alipay-PIBench, a benchmark for evaluating coding agents on realist...

Shiyu Ying, Xuejie Cao, Yingfan Ma et al. · 0 citations
Jul 2026

AllocBench: Measuring Online Tool Allocation Capability in LLM Agents

A paired benchmark that tests whether LLM agents exhibit conscious allocation behavior under a fixed budget in two contexts: an abstract text-based formulation and a code-construction task finds that every frontier model testedacts near-optimally in the abstract framing but fails to transfer this ability to script-writ...

Daniel Wang, Andrew Xu · 1 citation
Jul 2026

Inference Economics of Enterprise Coding Agents: A Case Study of Cloud vs. On-Premise LLMs

A single-developer, non-randomized longitudinal case study over two contiguous 28-day periods on a production monorepo shows hybrid routing gateways trade defect rate for infrastructure savings along a cost-quality frontier rather than dominate the pure-API baseline.

Sheng-Wei Peng, Yi-Hsun Lin, Yi-Pei Lee · 1 citation
Preprint Aug 2026

Keep, Customize, or Exit: Default Design and Token Pricing in LLM Reasoning Services

We study a large language model (LLM) service in which a provider chooses a per-token price and a default reasoning-token allocation, while a user may accept the default, customize the allocation, or exit. Larger allocations can improve accuracy but increase token cost and latency. We model this interaction as a Stacke...

Ahmet Bugra Gundogan, Yigit Turkmen, Melih Bastopcu · 0 citations
Preprint Aug 2026

ACEM: A Cost Estimation Model for Agentic Software Engineering

ACEM (Agentic Cost Estimation Model), which decomposes total agentic development cost into three additive dimensions: LLM, HITL, and infrastructure cost, is presented as a fully specified model structure and calibration methodology, with constants left symbolic pending empirical grounding.

Mohammad El-Ramly · 0 citations

Related blog posts

MIT News · Artificial Intelligence Sep 29, 2026

Who we become when we talk to machines

Professor Sherry Turkle’s new book, “Artificial Intimacy,” offers a withering critique of chatbots and the antisocial dynamics she believes they encourage.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.