Context-Aware Feedback Compression in Online Judge Programming with LLMs

Abstract

Large language models (LLMs) can generate non-trivial programs, yet their reasoning often remains ungrounded: without external verification signals, one-shot generation may drift, repeat failure modes, or overfit to examples. We argue that the missing piece is budgeted feedback compression: turning noisy oracle outputs into compact, actionable hints that reliably drive multi-round code revision under tight context limits. We instantiate this idea in online judge (OJ) style algorithmic programming as a modular interactive agent that couples an LLM core with a sandboxed judger, a feedback-to-hint prompt constructor, and trajectory memory (optional error classifier). The key mechanism is feedback compression: converting noisy execution artifacts into compact, actionable hints within a tight prompt budget. In preliminary experiments, execution-grounded iteration improves debugging success from 83.9% (one-shot) to 93.2% on 570 real failed Codeforces submissions, and yields a clear difficulty trend in solving across 50 problems. Beyond OJ tasks, we envision budgeted oracle-to-hint compression as a general foundation for software engineering assistants that interact with continuous integration (CI) pipelines, tests, and profilers, shifting evaluation from final accuracy toward process metrics such as convergence and patch locality.