Qiushi Engine on AstaBench E2E-Bench-Hard

MetroLLM-Bench: Evaluating Language Models as Transit Kiosk Runtimes

This work introduces MetroLLM-Bench, a 955-case benchmark for testing language models as the policy layer of a transit kiosk, and evaluates twenty-six models from six vendors, of which twenty-three are ranked.

ProgramDistill: From Interactive Web Apps to Verifiable Reference-Guided SWE Tasks

ProgramDistill is introduced, a benchmark evaluating coding agents on features discovered through interaction with fully functional reference applications, and provides a scalable benchmark with controlled difficulty for evaluating and diagnosing coding agents, and a natural basis for future curriculum-based training.

Shortcutting the Fix: Identifying and Categorizing Agentic Exploits in Software Engineering Benchmarks

This work systematize and audit exploits across five open large language models on SWE-bench Multilingual and DeepSWE using a turn-level LLM-as-a-judge protocol, demonstrating the critical need for exploit-aware evaluation frameworks that measure true repository-level problem solving over benchmark gaming.

K-Bench: A Benchmark for LLM Unlearning in Agentic Deployments

K-Bench is introduced, a benchmark that scores LLM unlearning under agentic deployment and certifies forgetting by reading the model's final answer, where a model that refuses to answer already counts as having forgotten.

Evaluating and Benchmarking the System One Model Jev

Jev answers MMLU's calculation-heavy questions more accurately than other MMLU questions (94% vs. 91%), whereas both open models, and all three on C-Eval, find them harder.

LLM Accuracy in Tabular Fact Verification: A Factorial Evaluation of Model, Temperature, and Prompting Strategy