Rubrics provide structured, fine-grained signals for training and evaluating large language models (LLMs). Yet reliable query-specific rubrics are difficult to construct. Existing approaches often derive supervision from human-written rubrics, preference data, or sampled responses. Direct query-to-rubric generation avoids these resources, but provides no explicit check that a plausible rubric is useful. Such a rubric may fail to distinguish answer quality, reward an optional style, or penalize a valid alternative strategy. We introduce Rubrics on Trial, a query-only framework that evolves a rubric set from an empty set without external annotations or model training. It derives supervision solely from synthetic rubric-conditioned response pairs and validates each proposed rubric before adding it, screening out non-discriminative, over-specific, and style-only candidate rubrics. Experiments across five preference benchmark suites demonstrate the effectiveness of Rubrics on Trial, which achieves the best average accuracy and leads on six of seven evaluation sets.
Hao Yang, Licheng Pan, Xiaoxi Li et al.· 0 citations
This work proposes SpaCellAgent, an autonomous large language model (LLM) multi-agent framework that automates end-to-end spatiotemporal analysis and narrative generation and establishes a scalable, agent-driven paradigm for computational biology.
Songhan Wang, Haoang Chi, He Li et al.· Proceedings of the 32nd ACM...· 1 citation
Hetero-Designer is proposed, a novel pipeline that efficiently encodes intricate dependencies among queries, LLMs and roles through a novel Binary-Star Transformer and constructs Hetero-MAS in an autoregressive graph generation process.
Zhiheng Zhang, Yuanzhe Zhang, Bohan Yu et al.· Annual Meeting of the Associ...· 0 citations
DynaRule is proposed, an end-to-end framework that injects the given rules into the KV cache and turns retrieval into an internal, learnable, step-wise process, and can re-attend to the most relevant rules at each step, dynamically replacing outdated ones to support more stable multi-step reasoning.
Bohan Yu, Pengfei Cao, Chen Han et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
