This work introduces WhatIf-Bench, a benchmark measuring how well forecasters use covariates to forecast a target's response to future events, and reveals generalization to Novel futures as the core challenge, and offers a testbed toward arbitrary scenario analysis.
Arjun Ashok, Mihir Parmar, Palash Goyal et al.· 0 citations
T2J-BENCH is introduced, a benchmark of 63 whole-repository PyTorch-to-JAX migration tasks spanning 18 model families across both training and serving, and TWINHARNESS, a reference-grounded agent harness that turns the agent's executable PyTorch source into legitimate self-verification feedback.
Tao Feng, Jie-Feng Chen, Lin-Xin Song et al.· 0 citations
Evaluation has been central to advances in AI, and it is equally critical to autonomous discovery. Yet scaling autonomous discovery to real-world engineering challenges faces a fundamental bottleneck: the cost of evaluating progress. In real settings, a single true evaluation, such as a full benchmark or serving trace,...
Shubham Agarwal, Akshat Gupta, M. Heddes et al.· 0 citations
Real-world time series are frequently driven by exogenous events and structural shifts, rendering conventional forecasting based solely on historical numerical observations insufficient. While language models can retrieve external news, standard retrieval-augmented approaches struggle with high noise, missing signals,...
Ming-Tian Tan, Palash Goyal, Mihir Parmar et al.· 0 citations
It is shown that explicit idea-level allocation makes semantic coverage directly controllable, and that broader coverage becomes increasingly valuable when competitive research directions are sparse among many plausible alternatives.
Diffusion models can produce striking images and videos, but they still struggle with the compositional details that make a generation faithful to a prompt, such as object counts, attribute binding, spatial relations, and temporally grounded actions. A common way to improve prompt satisfaction is to spend more compute...
Vighnesh Subramaniam, B. Katz, Brian Cheung et al.· 0 citations
The introduction of ScientistTwo, a fully autonomous multi-agent framework designed to realize problem-driven autonomous research, and its results show that ScientistTwo is not merely an assistive tool but an autonomous scientific pioneer capable of pushing the frontiers of human discovery.
Jaehyun Nam, Jinsung Yoon, Yan Pan et al.· Robotics· 0 citations
MTPaperBananaBench, a benchmark for multi-turn diagram generation containing 292 images annotated with 3,518 user requirements, is presented and PaperBanana-Interact, a multi-agent system that refines diagrams via an internal critique-and-refine loop is introduced.
Xue-Qing Wu, Ashwin Balasubramanian, Bing-Xuan Li et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.