TuneBench is introduced, a benchmark for evaluating Agentic AI in power system dynamic model calibration, where controller parameters are adjusted to reproduce target dynamic responses in LLM-based agents.
Abstract
Large Language Model (LLM)-based agents can automate complex engineering workflows by interacting with tools, interpreting feedback, and making iterative decisions. This paper introduces TuneBench, a benchmark for evaluating Agentic AI in power system dynamic model calibration, where controller parameters are adjusted to reproduce target dynamic responses. The benchmark reflects a common Transmission System Operator (TSO) practice, in which equivalent models used for Dynamic Security Assessment (DSA) are calibrated against measurements from grid-code compliance tests when proprietary models are unavailable. Two cases of increasing complexity are considered: the governor and Automatic Voltage Regulator (AVR) of a synchronous generator and the control system of a Battery Energy Storage System (BESS), both tuned against active-power and voltage step tests within a budget of 15 simulations. LLM-based agents are compared with random search and Genetic Algorithm (GA) baselines over 10 independent runs. Only the most advanced models, Opus 5.5 and GPT-6, accurately reproduce all target trajectories, with median R2 above 0.995 in every test and tracking errors up to two orders of magnitude lower than the baselines. The agent logs show that these agents autonomously extract dynamic performance indices, such as settling time and damping, and relate them to specific control parameters, decomposing the tuning problem into physically interpretable sub-problems. The benchmark is publicly available at https://github.com/Power-Agent/PowerAgentBench/tree/main/benchmarks/dynamic/level3.
This paper describes a skill-based agentic framework for power-system studies using Model Context Protocol (MCP)-connected engineering tools. A custom MCP server was developed to expose Siemens PTI PSSE functions for power-flow analysis, dynamic simulation, result extraction, and model-validation workflows. Two impleme...
P. Etingov, Shuchismita Biswas· arXiv.org· 0 citations
PowerZooJax is introduced, a JAX-based benchmark suite for RL in power system operation that provides five constrained Markov decision process tasks spanning generation, transmission, distribution, distributed energy resources, and data center microgrid and shows substantial speedups over CPU-based simulations.
Zhan-Hua Pan, Xiao Liu, Zhi-Long Cao et al.· arXiv.org· 0 citations
Building surrogate models for production system simulators is a recurring engineering task that remains manual, expert-dependent, and inconsistent across teams. Each update cycle requires a specialist to select input parameters, run simulations, choose ML architectures, tune hyperparameters, validate results, and pac...
A. Naufal, B. Bhatt, A. Shariff et al.· ADIPEC· 0 citations
This work proposes PowerBench, a generation framework that derives interconnected heterogeneous operational data through a common dependency chain, and a synthetic dataset generated by this framework, comprising a generation framework that derives interconnected heterogeneous operational data through a common dependenc...
Rosetta is presented, a multi-agent LLM pipeline that automatically generates first-principles analytical models from research paper PDFs, and four design decisions address failure modes of na\"ive LLM-based generation.
A simulator-grounded performance-judgment benchmark evaluates four open-weight checkpoints across a quadruple-tank process, a recycle reactor, and a synthetic 3-by-3 cyclic plant, finding that outputs are associated with lower gain magnitudes, but the association varies with scaling, plant, and controller sampling.
Jia-Xuan Chen, Yang Shu, Hao-Nan Li· Processes· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.