A Jagged Frontier: Evaluating Robustness of Code Agents to Semantics-Preserving Transformations
This work introduces a random variant sampler that applies common semantics-preserving transformations (SPTs) - spanning control-flow rewrites, dead-code injection, and identifier renaming - to produce perturbed variants, demonstrating that even top frontier models are susceptible to semantics-preserving perturbations.
Hasan Mahmud, Shreya Gupta, Isha Chaudhary et al.
· 3 citations