Large language models are increasingly used to generate test scenarios, executable test suites, assertions, interaction sequences, and fuzzing infrastructure. These artifacts serve different purposes and rely on different sources of information about correct behavior. A test that increases implementation coverage, a te...
Yun-Hao Liang, Cheng-Guang Gan, Rui-Xuan Ying et al.· 0 citations
Web agents are an important application of large language models, yet their evaluation often depends on rule based or language model evaluators that inspect only the final outcome. Human verification of task completion and detailed analysis of failed trajectories remain limited. We audit all 165 WebArena Lite tasks und...
Cheng-Guang Gan, Zi-Meng He, Yoshihiro Tsujii et al.· 0 citations
A task-level bootstrap certificate that is valid in every regime the authors test while matching the naive certificate's coverage, and doubles as a self-training filter that lets a judge enter an unseen domain at in-domain strength with zero target training labels.
Cheng-Guang Gan, Yun-Hao Liang, Qing-Hao Zhang et al.· 0 citations
Web agents are usually evaluated in live environments, where environment state and judge models drift between runs, so the same checkpoint rarely reproduces the same score, making controlled studies of training phenomena impractical. We present WebMRE, an offline benchmark of 541 tasks and 5,293 steps derived from succ...
Cheng-Guang Gan, Yun-Hao Liang, Qing-Hao Zhang et al.· 0 citations
A structured scoping survey organized around the question of what decision a test changes is presented, and distinguishes the Red--Green--Refactor cycle from test-conditioned generation, execution-guided refinement, test-mediated analysis, and evaluation-only testing.
Yun-Hao Liang, Cheng-Guang Gan, Rui-Xuan Ying et al.· 0 citations
Generating tests from a natural-language specification requires both an input that exposes faulty behavior and a correct expected output. These requirements need not improve together: a model can increase test correctness by choosing easier inputs, or discover useful inputs whose expected outputs it cannot predict. We...
Yun-Hao Liang, Cheng-Guang Gan, Rui-Xuan Ying et al.· 0 citations
This work studies test utilization through matched prompting controls, paired semantic interventions, and test suites selected by fault detection to make test-specified rule changes measurable alongside implementation capability and benchmark correctness.
Yun-Hao Liang, Cheng-Guang Gan, Rui-Xuan Ying et al.· 1 citation
Using gradient signatures across 12 tasks, four semantically equivalent interfaces, three model families, and controlled corruptions, it is shown that this interface confounds both measurements.
Cheng-Guang Gan, Han-Jun Wei, Yun-Hao Liang et al.· 0 citations
The Mutual Reinforcement Effect is tested in multimodal document understanding on three corpora, two of receipts and one of scanned business forms, comparing single-task, joint and conditioned training, which puts one granularity's gold output in the other's prompt during training only.
Cheng-Guang Gan, Yun-Hao Liang, Han-Jun Wei et al.· 0 citations
This work asks whether it adds skill to a small language and vision-language model web agent at the 4B to 8B scale, or whether it mostly reshapes behavior the supervised model already has, and explains the failure of GRPO.
Public tests are widely used to guide large language model code generation, but whether models treat them as executable specifications or merely as extra prompt context remains unclear. We study test-driven code generation on HumanEval+, MBPP+, and recent LiveCodeBench tasks using Qwen2.5-Coder-7B and Qwen3.6-27B. We c...
Yunhao Liang, Chengguang Gan, Ruixuan Ying et al.· 0 citations
MAG is introduced, the first benchmark that unifies task execution and guide writing into a single Multimodal Action and Guide task, with two grounding schemes over screenshots: Set-of-Mark element selection and raw pixel coordinates.
