A comprehensive benchmark of 20 teachers from 11 well-known ViT families reveals that Attention Transfer is not universally effective, and identifies architectural mismatch between the pre-trained teacher and the standard student as the primary mechanism.
Huai-Yuan Qin, Mu-Li Yang, Gabriel James Goenawan et al.· arXiv.org· 3 citations
Recently, coding agents have emerged as a dominant paradigm for real-world software engineering (SWE) scenarios, which solve complex tasks through multi-turn interactions with development environments. However, frequent interactions with environments would inevitably introduce substantial token overhead, leading to hig...
Hao-Bin Li, Liang Jiang, Zhen-Yu Huang et al.· 0 citations
Effective skill identification is crucial for LLM agents to reuse task-solving experience for complex task execution, which however remains challenging as agents must accurately identify useful skills from a large skill gallery and invoke them on demand. Existing studies have explored various strategies for skill retri...
Shuo Liu, Yu-Tong Yang, Haonan Xiao et al.· 2 citations
Copying MLPs can serve as an effective foundation for Softmax-to-linear transfer: paired with the distilled attention, linear ViTs eventually close the remaining gap and even surpass Softmax ones.
Huai-Yuan Qin, Mu-Li Yang, Gabriel James Goenawan et al.· 1 citation
It is proved that, under a matched fixed-anchor implementation, soft-label cross-entropy and KL divergence differ only by a constant entropy term and therefore induce identical gradients, Hessians, and adversarial optimization trajectories.
Wenyuan Wu, Yuan Sun, Ying-Ke Chen et al.· 2 citations
This paper proposes NeuCME, a novel framework designed to effectively learn and integrate knowledge across tasks with varying modalities, and formulate an evaluation metric to quantify the dynamism of task sequences and then set up a comprehensive benchmark with different degrees of dynamism.
Kai Guo, Chuan-Bin Liu, Peng Hu et al.· 0 citations
This work proposes SkillDreamer, a novel framework that first infers the capabilities necessary for task execution, then imagines how to realize these capabilities by generating pseudo skills, and finally leverages such prospective information to bridge the gap between objective-oriented task queries and execution-orie...
Shuo Liu, Yu-Tong Yang, Haonan Xiao et al.· 3 citations
This work proposes a novel difficulty-aware task formulation pipeline with a dual-track evaluation framework, facilitating comprehensive evaluation of proactive bug-fixing capability, and proposes a novel difficulty-aware task formulation pipeline with a dual-track evaluation framework.
Hao-Bin Li, Ping Deng, Weizhong Qian et al.· 0 citations
RA-CAD (ReAct Agent for CAD), a state-aware agent that interacts with the CAD environment through a Generate--Execute--Critique--Rewrite loop achieves state-of-the-art execution validity and geometric quality compared with existing methods and strong proprietary language models, demonstrating the effectiveness of the p...
Shu-Hao Yan, Changhao He, Peng Hu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
