Omkar Manohar Ghag

Agentic Flakiness: Reframing LLM Agent Reliability Through the Lens of Flaky-Test Research

Large language model (LLM) agents are nondeterministic: the same task, run twice under identical conditions, may succeed once and fail once. The agent-evaluation literature has begun to measure this- most directly through the pass^k metric introduced with τ-bench- but treats it as a novel phenomenon. It is not. Softwar...