Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in both visual understanding and code generation. However, existing benchmarks typically evaluate these two modalities in isolation, lacking a dedicated assessment of their unification, i.e., how a model can perceive complex visual struc...
Zi-Jian Chen, Zheng-Yu Chen, Bo-Han Liang et al.· 0 citations
Coding agents are increasingly asked to build software rather than patch it, yet benchmarks for from-scratch repository construction are mostly limited to a single language and depend on manually curated tasks. We introduce Zero2Repo, a benchmark in which an agent receives a product requirements document, an interface...
Pei Yang, Tian-Yu Shi, Yu-Hang Yao et al.· 0 citations
ShareMem is introduced, a memory architecture that shares reusable experience while grounding its application in the receiving user's own preferences, and improves step success, average task success, and dialogue-macro coding scores, respectively, over matched user-local memory across all four models.
Jinming Hu, Haodong Zhao, Qi Jia et al.· 0 citations
This survey systematically survey existing research on HLS tasks, including humor, sarcasm, metaphor, empathy, persuasion, narrative, and other general HLS phenomena, across text, speech, vision, and multimodal scenarios.
Xiujie Song, Ge-Fei Yang, Yi-Ning You et al.· arXiv.org· 0 citations
This work introduces SciMIF, a novel benchmark designed to evaluate the capability of MLLMs in following complex scientific instructions, and proposes a comprehensive taxonomy comprising 10 constraint groups that captures both general functional requirements and discipline-specific characteristics.
Ye Shen, Yu-Ting Zheng, Dun Pei et al.· 0 citations
QoNext is introduced, the first framework that adapts Quality of Experience principles from networking and multimedia to the holistic assessment of human-AI interaction and demonstrates that QoNext effectively decodes the underlying mechanisms of user satisfaction and enables precise prediction of human sentiment acros...
Yi-Jin Guo, Fa-Rong Wen, Ye Shen et al.· arXiv.org· 0 citations
PRISM-Bench is presented, the first audio-centric diagnostic benchmark for T2AV generation and demonstrates that current generation paradigms overfit to perceptual fidelity while struggling with complex grounding and control tasks, particularly in generating music and synchronized On-screen audio.
Yu-Chen Sun, Qian Yang, Jun Wang et al.· 0 citations
This paper introduces SafeAtlas-VL, a dataset of 1.5M training instances that places image-, request-, and response-level judgments on a five-level ordered scale, and trains the SafeAtlas Guard series of models via target-conditioned tuning for multimodal safety detection.
Zong-Rui Wang, Xiang-Yang Zhu, Sixiang Wang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.