Skip to content
Preprint

TuneBench: Benchmarking AI Agents for Power System Dynamic Model Calibration

Nov 2026 · 0 citations · 22 references
Engineering Computer Science

TL;DR

TuneBench is introduced, a benchmark for evaluating Agentic AI in power system dynamic model calibration, where controller parameters are adjusted to reproduce target dynamic responses in LLM-based agents.

Abstract

Large Language Model (LLM)-based agents can automate complex engineering workflows by interacting with tools, interpreting feedback, and making iterative decisions. This paper introduces TuneBench, a benchmark for evaluating Agentic AI in power system dynamic model calibration, where controller parameters are adjusted to reproduce target dynamic responses. The benchmark reflects a common Transmission System Operator (TSO) practice, in which equivalent models used for Dynamic Security Assessment (DSA) are calibrated against measurements from grid-code compliance tests when proprietary models are unavailable. Two cases of increasing complexity are considered: the governor and Automatic Voltage Regulator (AVR) of a synchronous generator and the control system of a Battery Energy Storage System (BESS), both tuned against active-power and voltage step tests within a budget of 15 simulations. LLM-based agents are compared with random search and Genetic Algorithm (GA) baselines over 10 independent runs. Only the most advanced models, Opus 5.5 and GPT-6, accurately reproduce all target trajectories, with median R2 above 0.995 in every test and tracking errors up to two orders of magnitude lower than the baselines. The agent logs show that these agents autonomously extract dynamic performance indices, such as settling time and damping, and relate them to specific control parameters, decomposing the tuning problem into physically interpretable sub-problems. The benchmark is publicly available at https://github.com/Power-Agent/PowerAgentBench/tree/main/benchmarks/dynamic/level3.

View source

Similar papers

Sep 2026

Skill-Based AI Agents for Power-System Studies

This paper describes a skill-based agentic framework for power-system studies using Model Context Protocol (MCP)-connected engineering tools. A custom MCP server was developed to expose Siemens PTI PSSE functions for power-flow analysis, dynamic simulation, result extraction, and model-validation workflows. Two impleme...

P. Etingov, Shuchismita Biswas · 0 citations

PowerZooJax: A JAX-based Power System Benchmark for Reinforcement Learning

PowerZooJax is introduced, a JAX-based benchmark suite for RL in power system operation that provides five constrained Markov decision process tasks spanning generation, transmission, distribution, distributed energy resources, and data center microgrid and shows substantial speedups over CPU-based simulations.

Zhan-Hua Pan, Xiao Liu, Zhi-Long Cao et al. · 0 citations
Nov 2026

Production Surrogate Modelling Auto-Research Multi-Agentic System

Building surrogate models for production system simulators is a recurring engineering task that remains manual, expert-dependent, and inconsistent across teams. Each update cycle requires a specialist to select input parameters, run simulations, choose ML architectures, tune hyperparameters, validate results, and pac...

A. Naufal, B. Bhatt, A. Shariff et al. · 0 citations

PowerBench: A Benchmark for Agentic Retrieval and Reasoning in Power Systems

This work proposes PowerBench, a generation framework that derives interconnected heterogeneous operational data through a common dependency chain, and a synthetic dataset generated by this framework, comprising a generation framework that derives interconnected heterogeneous operational data through a common dependenc...

Xi-Jing Wang, Yin-Sheng Yao, Jin-Ru Ding et al. · 0 citations
Sep 2026

Rosetta: Automating First-Principles Performance Modeling Using Multi-Agent LLMs

Rosetta is presented, a multi-agent LLM pipeline that automatically generates first-principles analytical models from research paper PDFs, and four design decisions address failure modes of na\"ive LLM-based generation.

K. Sankaralingam · 0 citations
Open access Sep 2026

Benchmarking Performance Judgment in Open-Weight LLM Controller Tuning: Control Knowledge Does Not Ensure Reliable Gain Ranking

A simulator-grounded performance-judgment benchmark evaluates four open-weight checkpoints across a quadruple-tank process, a recycle reactor, and a synthetic 3-by-3 cyclic plant, finding that outputs are associated with lower gain magnitudes, but the association varies with scaling, plant, and controller sampling.

Jia-Xuan Chen, Yang Shu, Hao-Nan Li · 1 citation

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.