Large language models (LLMs) can generate non-trivial programs, yet their reasoning often remains ungrounded: without external verification signals, one-shot generation may drift, repeat failure modes, or overfit to examples. We argue that the missing piece is budgeted feedback compression: turning noisy oracle outputs into compact, actionable hints that reliably drive multi-round code revision under tight context limits. We instantiate this idea in online judge (OJ) style algorithmic programming as a modular interactive agent that couples an LLM core with a sandboxed judger, a feedback-to-hint prompt constructor, and trajectory memory (optional error classifier). The key mechanism is feedback compression: converting noisy execution artifacts into compact, actionable hints within a tight prompt budget. In preliminary experiments, execution-grounded iteration improves debugging success from 83.9% (one-shot) to 93.2% on 570 real failed Codeforces submissions, and yields a clear difficulty trend in solving across 50 problems. Beyond OJ tasks, we envision budgeted oracle-to-hint compression as a general foundation for software engineering assistants that interact with continuous integration (CI) pipelines, tests, and profilers, shifting evaluation from final accuracy toward process metrics such as convergence and patch locality.
Jialiang Gu, Keren Zhou, Daming Li et al.· SIGSOFT FSE Companion· 1 citation
Mixture-of-experts models whose expert pools dwarf DRAM force every serving system to contain a cache, yet existing systems typically implement this cache in user space using expert-granular, frequency-ranked, explicitly pinned tiers. We characterize the alternative already provided by the OS: the page cache as the expert tier. We use router traces from three MoE models spanning 128 to 896 experts per layer, including a production trillion-parameter model with a 1.45 TB expert pool, and replay them natively against the full pool on GH200 nodes with capacity enforced by three independent mechanisms. We find four main results. First, iteration time and device traffic are smooth, reproducible functions of cache capacity, making DRAM a practical sizing knob for trillion-parameter serving. Second, the deep-pressure knee is a reclaim artifact requiring both MGLRU and balloon-style mostly-mlocked memory; cgroup limits and physical-memory configurations show no such amplification, indicating that balloon-based studies can overstate pressured-end device traffic by approximately 2$\times$. Third, under an enforced equal-memory wall, untuned kernel LRU serves essentially as much demand as a same-domain oracle frequency table (75.3% versus 74.6% at 256 GB), while the oracle's mechanism advantage is only 1.09$\times$ and disappears off-domain, where LRU remains at 70--71% hit rate. Fourth, router lookahead with 64.7% measured recall provides only a 0.3% benefit as kernel readahead advice and no benefit as synchronous prefetch. End to end, enabling kernel-managed caching improves decode performance by 1.09--1.10$\times$ with token-identical outputs across nine balanced pairs. The resulting design principle is simple: in this regime, let the kernel own eviction, while model-specific knowledge is best spent on admission and advice.
Yuan Si, Yufeng Lin, Daming Li et al.· 0 citations