This paper examines Data Agents from a harness-centric perspective, introducing a taxonomy of Data Agents and associated data environments, and identifying four open reliability problems: inactive semantic calibration, missing clarification, missing experience transfer, and the missing verification-repair repository.
Hua-Chi Zhou, Yu-Jing Zhang, Jia-He Du et al.· 0 citations
CrossModalQA is introduced, an open-domain benchmark for evaluating multimodal retrieval and reasoning over heterogeneous corpora and it is revealed that complete cross-modal retrieval contributes more to answer accuracy than generator scaling, while multi-image retrieval and reasoning remain the primary bottlenecks li...
Jia-Cheng Cai, Zi-Jin Hong, Zheng Yuan et al.· 0 citations
TIDE-Bench is introduced, a benchmark for conversational text-to-SQL under chain ambiguity and intent drift evaluation, targeting two recurring patterns: chain ambiguity, where an underspecified question triggers layered clarification with conditional dependencies, and intent drift, where the user retracts and replaces...
Yu-Jia Liu, Jia-Yan Lin, Zi-Jin Hong et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.