Francisco Ortin

SafeLLM4SE: Statistical Evaluation and Reporting for LLM-based Software Engineering Systems

Large language models (LLMs) are increasingly used for software engineering tasks, yet their stochastic behavior challenges the validity, reproducibility, and comparability of their evaluations. Conventional practices such as reporting a single output, an average score, best-of-N, or pass@k performance can obscure vari...