Skip to content

Rosetta: Automating First-Principles Performance Modeling Using Multi-Agent LLMs

Sep 2026 · arXiv.org · Vol abs/2609.19376 · 0 citations · 91 references
Computer Science

TL;DR

Rosetta is presented, a multi-agent LLM pipeline that automatically generates first-principles analytical models from research paper PDFs, and four design decisions address failure modes of na\"ive LLM-based generation.

Abstract

Analytical performance models --- derivations of throughput or speedup from hardware parameters --- make claims independently verifiable and expose binding constraints, yet rarely accompany architecture papers because building one by hand takes weeks of expert effort. We present Rosetta, a multi-agent LLM pipeline that automatically generates first-principles analytical models from research paper PDFs. Given a paper as sole input, Rosetta produces a mathematical specification, an executable Python model, and a plain-English interpretation --- all autonomously, with zero human intervention. The formalization process itself is the primary value: it surfaces implicit assumptions and identifies missing parameters. Four design decisions address failure modes of na\"ive LLM-based generation: a scientific constitution that prohibits circular reasoning, verify-repair loops with independent critic agents, dual verification separating functional correctness from scientific validity, and a best-of-$N$ ensemble that exploits LLM stochasticity. We evaluate Rosetta across three complementary tracks: expert evaluation of 12 landmark papers (CS1), automated scoring of 97 unfiltered ISCA 2025 and HPCA 2026 papers (CS2), and author self-evaluation by six active research groups (CS3). Across CS1, specification quality scores 4--5/5 on 10 of 12 papers with zero significant hallucinations; across CS2, 56\% of fit-screened papers reach Tier~A insight quality. The strongest finding comes from CS3: Rosetta's output led to revised claims and new experiments in active submissions, and five of six author-evaluators said they would use it again.

View source

Similar papers

Preprint Aug 2026

AutoMOOSE: Use Case and Logical Views of Agentic Phase-Field Simulation Software

This work deploys AutoMOOSE as a agentic software, complementing the prior work which focused on development of the agentic tool, and describes its software framework and architecture through Use Case and logical views of the 1+5 architectural-views model.

Sukriti Manna, Henry Chan, Subramanian K. R. S. Sankaranarayanan · 0 citations
Preprint Oct 2026

HakiCC: LLM-Driven Multi-Agent Design and Optimization of Concurrency Control Protocols

Large language models (LLMs) have recently been applied in systems research as a tool to reduce human-intensive engineering effort through cost-efficient automation. Decades of research have produced a rich landscape of concurrency control (CC) protocols, each encoding distinct trade-offs in correctness, throughput, an...

Farzad Habibi, Jun-Cheng Fang, Faisal Nawab · 0 citations

Who Holds the Pen? Let Specifications, Not Agents, Sign Off

Experiments on guideline-following and artifact-generation tasks show that specifications can serve not merely as behavioral guidance, but as authority over compliant execution and completion.

Hai-Qing Li, Xin-Yu Ma, Yin-Hao Wu et al. · 0 citations
Preprint Aug 2026

From Atomic to Agentic: Towards Interpretable Evaluation of LLMs'Agentic Mathematical Capabilities

Experiments reveal that models with similar end-to-end accuracy can exhibit markedly different agentic capability profiles, demonstrating that process-level evaluation is crucial for interpreting the true potential of LLMs and guiding the development of next-generation mathematical agents.

Jiayi Kuang, Ying-Hui Li, Yun-Ze Song et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.