Toward Agentic SysAdmin: Rethinking System Administration with AI Agents
NetLLMeval is presented, a framework for automatically evaluating LLM-based systems on network administration tasks by leveraging live network emulation to derive ground truth without human intervention, and is released open-source to support reproducible benchmarking of future models and solver designs.