Beyond the Traceback: Using LLMs for Adaptive Explanations of Programming Errors

Debugging Functionality-Twisting Translations by LLMs via Differential Testing with Bayesian Prior

Code translation, as a challenging and fundamental task, is increasingly relying on large language models (LLMs). However, LLMs often give seemingly plausible but fallacious translations, misleading and even deceptive to debugging developers. We propose tHinter, an automated approach that frames translation error localization as a differential testing task. By fuzzing and a novel Bayesian-inspired algorithm that integrates developer heuristics into SBFL, tHinter thoroughly exposes translation errors and effectively pinpoints functionality-twisting errors. We evaluate tHinter in a Python to C++ scenario, a representative and challenging task for migrating high-level logic to performance-critical systems. Experiments show that tHinter achieves the highest F1 in localizing translation errors, outperforming all baselines. When debugging LLM-translated code with human efforts, it averagely reduces 71% lines developers need to review. Furthermore, we provide a comprehensive evaluation reference beyond traditional fault localization metrics. We integrate mixed-factorial user studies, expert validation, and SWOT-based strategic analysis to assess the perceived helpfulness and resilience within the rapidly evolving LLM landscape, offering a methodologically rigorous example for researchers in the context of AI-augmented software engineering.

What really happens when a dev vibes with the code? An empirical study on LLM behavioral divergence in response to expressive code comments

Introduction We investigate how expressive inline code comments written in various developer styles, functional to progressively poetic, philosophical, and misleading, affect large language model (LLM) behavior during code optimization. Methods In this pilot study, we used a controlledmerge sort implementation across five stylistic variants and evaluated GPT-5 and Claude Opus 4.1 under standardized console prompts, isolating the effect of embedded comment semiotic variation. Seven expert developers (three senior, four mid-level) scored model outputs against adapted ISO/IEC 25010 criteria and novel LLM suggestibility index (LSI) framework. Results Semiotic character of comments measurably altered code quality, with consensus-score reliability ICC(2, k) = 0.65–0.81 for six of seven dimensions; single-rater Krippendorff's α = 0.232 reflects substantial interpretive variability. Claude exhibited higher interpretive sensitivity (mean behavioral divergence 4.00; SD 1.16), while GPT-5 maintained stronger architectural fidelity (mean divergence 3.58; SD 1.26). Reflective comments (philosophical, conversational) were associated with Claude's highest maintainability scores in our panel (both M = 4.00, ~8% above stock M = 3.71), while the same philosophical comments reduced GPT-5 maintainability (M = 2.86), suggesting asymmetric model responses to expressive context. Conclusions These findings position inline comments as model-sensitive latent semantic prompts, with implications for AI-in-the-loop development and design of comment conventions for AI-assisted maintenance.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code

It is observed that generated code often omits basic input validation or memory-safety checks, which can lead to overflows, resource exhaustion, or other reliability/security issues, and even the largest models frequently make simple mistakes.

Does Great Power Come with Great Explainability? Comparing Explanation Strategies for Automated Program Diagnosis

Debugging is an important activity in software development, yet providing actionable and comprehensible explanations for program failures remains challenging. Automated tools such as ALHAZEN and AVICENNA address this by using distinct strategies: ALHAZEN employs binary decision trees to show failure-inducing conditions, while AVICENNA uses a specification language to model complex input dependencies. To assess their impact on usability and user efficiency, we conducted a controlled within-between-subjects user study with 18 participants tasked with resolving four software bugs using either tool or no support. Quantitative results showed that both tools improved debugging efficiency compared to manual methods, with AVICENNA offering more precise diagnostics but requiring higher cognitive effort. Qualitative feedback revealed a preference for AVICENNA’s expressiveness despite its complexity. Our findings show that effective debugging tools have tradeoffs between accuracy and interpretability to support developers’ decision-making in increasingly complex software environments.

Novices Comprehending Functional vs. Imperative Programs: An Eye-Tracking Study

Programmers Are Poor and Overconfident Judges of LLM-Generated Assertions

The findings suggest that, contrary to common assumptions, AI assistance may not improve the reliability of code comprehension and review, and highlight the importance of helping developers evaluate machine-generated reliability artifacts, in addition to generating them.