Aug 2026· Asia-Pacific Workshop on Networking· 0 citations· 3 references
Computer Science
TL;DR
A Network Experimentation Harness for emulator-backed network experiments, helping LLM agents operate across these stateful workflows by pairing a semantic action interface with reusable experimentation skills to handle sequencing, timing, and verification that a careful experimenter would perform by hand.
Abstract
Agent-driven scientific experimentation is emerging across domains such as chemistry, biology, and materials, yet each tool class imposes its own execution discipline. Network experimentation requires more than one-shot topology or configuration synthesis: an experimenter must plan a task, operate a live and evolving network, interpret feedback, refine intermediate state, and validate the resulting behavior. This poster presents a Network Experimentation Harness for emulator-backed network experiments, helping LLM agents operate across these stateful workflows. The Harness pairs a semantic action interface with reusable experimentation skills to handle sequencing, timing, and verification that a careful experimenter would perform by hand. A preliminary study on GNS3-based network protocol experiments shows that this approach reduces wall-clock time by 47% and 37%, and token use by 81% and 76%, on average versus raw GNS3 access and a Python wrapper (GNS3Fy), respectively.
NetConfArena is presented, an executable benchmark for evaluating LLM agents in closed-loop network configuration, and its findings suggest two future directions: using validated trajectories as supervision signals to improve foundation models, and designing harness mechanisms that make agent execution more reliable an...
Chang Liu, Xiao-Hui Xie, Xinyi Chen et al.· 1 citation
Comparisons against stronger model and coding-agent competitors further indicate that both domain-specific agent runtime structure and foundation-model strength matter for autonomous data analysis.
This work uses ASAREE to evaluate key design choices in a mutli-agent machine learning pipeline and highlights ASAREE as a robust framework for evaluating agentic system performance and resource efficiency.
Jay Moran, P. Freda, Attri Ghosh et al.· bioRxiv· 0 citations
AgentCompass is introduced, an open-source, lightweight, and extensible infrastructure for evaluating LLM-based agents that organizes the evaluation process around three independent components, thereby enabling flexible configurations without requiring the reimplementation of complex execution logic.
DataClawEval is introduced, the first comprehensive benchmark designed specifically to evaluate the end-to-end task completion capabilities of autonomous agents in real-world data engineering scenarios, and it comprises 100 rigorous, end-to-end tasks spanning five execution engines.
SyntheticAgentTraceQA is proposed, an execution- first framework for generating scalable supervision data for tool- augmented agents and shows that execution-grounded supervision improves tool execution behavior, reference-trace agreement, and answer-generation performance on the evaluated tasks.
Hafsa Ouajdi, Francesco Giannuzzo, Alaa Boukhary et al.· arXiv.org· 1 citation· ⚡1
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.