Perspective of software engineering researchers on machine learning practices regarding research, review, and education

Abstract

Abstract Context Machine Learning (ML) significantly impacts Software Engineering (SE), but studies primarily focus on practitioners, neglecting researchers. This ignores practices and challenges in teaching, researching, or reviewing ML applications in SE. Objective This study aims to contribute to knowledge of the synergy between ML and SE from the perspective of SE researchers, by providing information on the practices followed when researching, teaching and reviewing SE studies that apply ML. Method We analyzed SE researchers familiar with ML or who authored SE articles using ML, along with the articles themselves. We examine practices, SE tasks addressed with ML, challenges faced, and perspectives of reviewers and educators using open and axial coding and qualitative analysis. Results We found diverse practices focused on data collection, model training, and evaluation. Some recommended practices (e.g., hyperparameter tuning) appeared in less than 20% of the literature. Common challenges involve data handling, model evaluation (including non-functional properties), and involving human expertise in evaluation. Hands-on activities are common in education, although traditional methods persist. Recent data show a shift from statistical learning towards deep learning and Large Language Models (LLMs), leading to new practices such as prompt engineering. Conclusion Despite the accepted practices in applying ML to SE, significant gaps remain. By improving guidelines, adopting diverse teaching methods, and emphasizing underrepresented practices, the SE community can bridge these gaps and advance the field.