This analysis covers 2,857 report-test-patch triplets from Defects4J and SWT-Bench using two widely adopted instruction-tuned LLMs from distinct model families and finds that both models exhibit systematic optimism relative to humans and only modest rank agreement, motivating bias-aware evaluation.
Wendkûuni C. Ouédraogo, Yinghua Li, Xueqi Dang et al.· arXiv.org· 0 citations
This work surveys real-world TUI applications, turns them into a headless benchmark spanning ratatui/Rust, bubbletea/Go, textual/Python, and ink/TypeScript, and compares four frontier LLMs with random exploration, finding no model dominates.
Chao Peng, Ruida Hu, Ajitha Rajan et al.· 0 citations
The results recast package hallucination as both a measurement problem and a decoding-time control problem, and they demonstrate that the choice of defense must be matched to the threat model and recommendation utility.
Albérick Euraste Djiré, I. E. Olatunji, Melissa Tessa et al.· Proceedings of the 41st IEEE...· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.