How Is Automated Research Evaluated? A Survey of Benchmarks and Evaluation Practices
This synthesis highlights three recurring lessons: output checks, process checks, and human studies provide complementary information; evaluator calibration is specific to the property being assessed; and resource budgets and attempt selection are integral to interpreting performance comparisons.
Liu-Lei Zhang, De-Jing Zhou, Chu-Yue Huang et al.
· 0 citations