RealSWE: A Compositional Evaluation of Coding Agents under Realistic User Requests
A six-category information taxonomy and four dimensions of linguistic style are defined and applied to real user prompts from SWE-chat and problem statements from SWE-bench Verified and Pro, finding that explicitly stating the desired behavior and motivation, which most real prompts omit, substantially improves the LLM's software engineering performance.
Gyuhyeong Kim, Hyojung Gwon, Jeonghyeon Kim et al.
· 0 citations