Skip to content
Preprint

Praxist: From Experimental Artifacts to Solution Lineages

Aug 2026 · 0 citations
Computer Science

TL;DR

Praxist is introduced, a lineage-centered generational system that converts reproducible artifacts and evaluator outcomes into a typed evidence graph of findings, lane-structured frontiers, and agendas, and Separating local artifact construction from cohort-level evidence synthesis lets later attempts inherit validated mechanisms, unresolved claims, and useful constraints.

Abstract

Autonomous R\&D agents now write, run, and improve executable artifacts under automated evaluation---but largely as laboratory instruments: shown on curated benchmarks, with gains that are hard to trace to a cause and costs well above what sustained engineering practice absorbs. The limitation is structural. Most systems treat each attempt as nearly self-contained, so logs, memories, and search trees record what happened without establishing which design element produced an improvement, whether its evidence survived validation, or how it recombines with others. Long campaigns therefore keep re-learning the same lessons. We introduce Praxist, a lineage-centered generational system that converts reproducible artifacts and evaluator outcomes into a typed evidence graph of findings, lane-structured frontiers, and agendas. Separating local artifact construction from cohort-level evidence synthesis lets later attempts inherit validated mechanisms, unresolved claims, and useful constraints, and leaves results attached to an inspectable lineage. On the standardized 75-task MLE-bench suite, the finalized official-grader results give Praxist 60 medals (80.0\%), 49 of them gold, against 55 medals (73.3\%) and 34 gold for a Claude Code baseline on Claude Opus 4.8---at a recorded model spend of US\$3,054 versus US\$38,370, roughly a twelfth of the cost. Four case studies---quantitative trading, LiDAR-inertial-visual SLAM, tokamak magnetic control, and rocket landing---carry the same process into open-ended engineering problems, improving on each task-native baseline in headline accuracy, survival, or resource cost, with the discovery path on record. Stronger artifacts at an order of magnitude less spend, each backed by an auditable lineage, are, to our knowledge, first brought together here: the operating profile production research requires, not the one a benchmark demonstration establishes.

View source

Similar papers

Preprint Aug 2026

One Recipe, Many Harnesses: What Self-Evolution Encodes Across Languages and Models

Together, these results recast the evolved harness as a legible compensation layer, shaped jointly by the language's engineering demands and the model's behavioral gaps, rather than an opaque benchmark-tuned scaffold.

Siqi Yang, Qianlan Yang, Yu-Xiong Wang et al. · 2 citations
#artificial intelligence Preprint Sep 2026

Bioinfoysis Technical Report

Large language model agents have shown promise in bioinformatics, but most existing systems focus primarily on producing final answers, treating planning, tool use, and code execution as transient interactions. This design is poorly suited to long-horizon bioinformatics tasks, where conclusions must remain connected to the data, computations, and intermediate evidence that support them. We introduce \textbf{Bioinfoysis}, a multi-agent harness that represents each request as a persistent, artifact-grounded analysis run. Bioinfoysis combines global planning with step-wise, evidence-driven replanning: the planner maintains an executable checklist and revises pending steps using structured handoffs returned after each worker execution. These handoffs bind intermediate results to their responsible agent, checklist step, and plan generation, preventing stale evidence from being silently reused after replanning. A controlled runtime validates generated scripts, tables, and figures before they are used in downstream analysis or reporting, while role-specific context, persistent memory, and governed bioinformatics skills support reliable execution over long analysis trajectories. We evaluate Bioinfoysis on BixBench and two question-answering tracks of LAB-Bench 2. On BixBench, Bioinfoysis achieves state-of-the-art accuracy of 82.4\%. Across four underlying language models, Bioinfoysis increases average accuracy from 27.81\% to 64.13\% on SeqQA2 and from 3.13\% to 31.25\% on DbQA2. These results demonstrate that reliable bioinformatics automation depends not only on model capability, but also on the harness that governs planning, execution, memory, and evidence flow. We hope that the emergence of Bioinfoysis will play a driving and leading role in the development of the bioinformatics community. Our demo website can be seen in https://report.bioinfoysis.com/.

Qing-Yang Shao, Xin Zhang, Zhou-Yang Yuan et al. · 0 citations
Preprint Aug 2026

Beyond Execution: Auditing Experimental Fidelity in LLM-Driven Scientific Research

ABE-Ralph is introduced, a reference-anchored auditing framework that represents claims, protocols, required components, baselines, and metrics as structured experimental constraints, guides implementation through an 8-step workflow, and performs quantitative, qualitative, and code-level verification.

Le-Zhi Yu, Xiaogang Xu, Yuhong Zhou et al. · 0 citations
Jul 2026

AgentHPOBench: A Benchmark For Evaluating LLM Agents as Sequential Hyperparameter Optimizers

AgentHPOBench, a sequential benchmark comprising 30 executable machine learning tasks across seven research categories, shows that current agents exhibit measurable experimental optimization ability across domains, but still face clear limitations in sustained iterative refinement, complex log diagnosis, and consistent progress toward reported reference performance.

Tianyu Huai, Tingshuo Fan, Xinchi Chen et al. · 0 citations
Jul 2026

ORCA-bench: How Ready Are Language Model Agents for Oncall?

The ORCA-bench benchmark is introduced, a benchmark that puts general-purpose coding agents in a production-fidelity oncall setting and is a lower bound on the engineering investment required before frontier coding agents can be safely entrusted with production reliability.

Albert Gong, Kyuseong Choi, Abhineet Agarwal et al. · 0 citations
Review Aug 2026

Agentic Artifact Creation: Systems, Evaluation, Principles, and Opportunities

This survey examines agentic artifact creation, which is defined as stateful construction in which an AI system materially constructs or revises a deliverable and intermediate observations redirect later work, and formulate principles for keeping commitments and responsibility explicit, turning feedback into targeted repair, and revalidating affected state after change.

Tianfu Wang, Zhezheng Hao, Xinchi Xia et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.