TL;DR
The results suggest that comparative selection is a low-regret design for AI-assisted research evaluation when model quality cannot be known in advance.
An author-facing LLM system that generates a broad pool of atomic concerns and compresses them into a short report that provides broad coverage with a large candidate pool but compresses poorly is studied.
The rapid growth of scientific publishing has strained peer review, particularly in machine learning, raising concerns about declining review quality and increasing reviewer workload. Large language models (LLMs) have been proposed as automated review assistants, yet their evaluation has focused largely on imitating hu...
On ClimateCheck claim-only and fine-tuned models outperform zero-shot LLM and top-performing AVeriTeC 2025 systems, highlighting that noisy evidence can degrade veracity prediction and replacing retrieved evidence with gold annotations improves veracity accuracy, confirming retrieval remains primary bottleneck.
How much of a conference accept/reject decision would change if the same paper were reviewed by a different set of reviewers? Running a second independent program committee is the gold standard for answering this, but it is prohibitively expensive: done only twice (NeurIPS 2014 and 2021). We build an observational esti...
Results highlight consistent trade-offs across representations and matchers: two-stage re-ranking improves early-rank performance; keyword-aware Sentence-BERT increases top-3 concentration; and KG edge overlap is competitive on full abstracts, while some graph variants substantially concentrate reviewer workloads.
The paper argues for evaluation procedures that disclose the tested configuration, validate questions and successful task completion, report performance alongside cost and execution time, and make the scope of generalization explicit, and discusses Isotanta, a crowdsourced benchmarking platform, as a practical example...
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
