POINTERBENCH is introduced, a diagnostic suite of 1,500 screenshot-instruction pairs probing three complementary capabilities: thin sub-element identification and geometric grounding in spreadsheets, character-level grounding in long-text documents, and mixed GUI grounding across 100 professional applications (POINTERB...
Valentino Sacco, Saurabh Pandey, Alexander Florian Jercher et al.· 0 citations
A deployed LLM agent emits tool calls, queries, and code that can be silently wrong -- by the time the error surfaces, the action has run. Frontier chat APIs hide the model's token probabilities; the agent's stated confidence barely beats chance on the mistakes that matter; and resampling does not help, since frontier...
This work presents TRACE (TRajectory Attribution for Automated Context Engineering), an automated feedback loop that mines historical agent trajectories to diagnose and remediate context failures, showing that over 80% of context-layer failures can be automatically diagnosed and remediated by mining historical trajecto...