Preprint
Aug 2026
RepoProbe: Benchmarking Architecture-Aware Repository Comprehension with Checklists
This work introduces RepoProbe, a novel benchmark for evaluating repository-level code understanding through open-ended Q&A using GitHub Discussions, which focuses on open-ended architectural inquiries rather than defect reporting and proposes a Checklist-Based Verification Protocol that decomposes answers into atomic, verifiable facts, thereby replacing subjective ratings with objective verification.
Yue Yang, Alyssa Wu, Ji Luo et al.
· 0 citations