This talk presents case studies in which evaluation metrics reward surface-level improvements like polish and fluency while overlooking evidentiary content, correctness, and editorial usefulness, and discusses practical lessons for building reliable evaluation systems in real peer review workflows.
Negar Arabzadeh, Sajad Ebrahimi, Shakiba Amirshahi et al.· Proceedings of the 35th ACM...· 0 citations
The CIFAR Synthetic Evidence Corpus for Detecting AI-Manipulated Images is introduced, a benchmark for evidentiary image authentication in court and justice-system contexts and establishes state-of-the-art baselines with publicly available image-manipulation detectors, showing that current systems exhibit error profile...
Kelly McConvey, Sajad Ebrahimi, Nima Jamali et al.· Proceedings of the 35th ACM...· 0 citations
Findings show that many peer-review evaluation metrics partially conflate review quality with linguistic presentation, and indicate that robustness to meaning-preserving rewriting should be validated before such metrics are used to compare human-written, AI-assisted, and AI-generated reviews.
Shakiba Amirshahi, Sajad Ebrahimi, Hai-Son Le et al.· Proceedings of the 35th ACM...· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.