2026· International Conference on Software and Data Technologies· pp. 637-644· 0 citations· 19 references
Computer Science
TL;DR
This paper evaluates various machine learning approaches on Eclipse dataset including using transformer based architectures with frozen weights as embedding with a fine-tuned neural network classification head (MLP) and experimented with Retrieval Augmented Generation (RAG).
Abstract
: Automatic bug triage is a critical software maintenance task that reduces the high human resource cost of manual developer assignment. In this paper we evaluate various machine learning approaches on Eclipse dataset. We compare classical machine learning approaches like TF IDF + XGBoost, LogReg, NaiveBayes and modern transformer based architectures including: Bert, Nomic, Mpnet, Jina, BGE, E5, CodeLlama, Llama-3.1-8B-Instruct, ChatGPT few-shot prompting. We compare various approaches including using transformer based architectures with frozen weights as embedding with a fine-tuned neural network classification head (MLP). For maximal performance we selected the best performing model E5 (47.63% accuracy) and fully fine-tuned it and achieved 72.49% top-1 accuracy which is the best performing deep learning based approach to the best of our knowledge slightly under performing our best approach: TF IDF+XGBoost 76.47%. Furthermore we also experienced with adding the embedding vectors of some of the aforementioned transformer-based models to the best performing TF IDF+XGBoost models but we experienced slightly worse performance than with only TF IDF+XGBoost. We also experimented with Retrieval Augmented Generation (RAG) and achieved 36.81% top-1 accuracy. The RAG results were primarily influenced by the retrieval rather than the generator.
A transformer-based classification framework that fine-tunes CodeBERT on a balanced dataset of 25,000+ bug-report segments drawn from four Apache projects is proposed and CodeBERT is the best-performing transformer in this comparison.
Kanwal Naz, Imran Shafi, M. Z. Mehmood et al.· Computers· 0 citations
The engineering details of ADEPT are presented, a framework that integrates representative adequacy techniques, including neuron-coverage-based metrics, surprise adequacy, input distribution coverage, boundary coverage, and source- and model-level mutation score, under a consistent execution workflow.
Yidi Kao, Shawn Burnham, Tommi Rose Fahy et al.· 0 citations
The design of transformer-based Large Language Models (LLMs) is being radically changed through new architectures that are able to overcome scalability limitations of previous designs, including Mixture-of-Experts (MoE), Multi-Head Latent Attention (MLA), and Multi-Token Prediction (MTP). As an open-weighted model released at the end of 2024, which has both state of the art architectural transparency and production scale efficiency, DeepSeeek-V3 represents the ultimate testing ground for investigating these modern technologies. This paper provides a comprehensive analysis of the architectural structure of DeepSeek-V3 based upon information from the DeepSeek-V3 Technical Report, industry benchmarking data and independent latency testing, to demonstrate how various techniques can be used to optimize training while still providing competitive performance in code generation and mathematical reasoning. In addition, latency testing conducted on a Distilled version of DeepSeek-V3, with approximately 14 billion parameters, running on a T4 GPU, reveals that although significant improvements have been made in optimizing latency there remains substantial barriers to deploying these models. Through this context, this research will serve as a reference document for practitioners and researchers who wish to understand current trends and challenges in increasing accessibility to high performance AI models.
Yassine Zouhdi, B. Hdioud· EPJ Web of Conferences· 0 citations
This work introduces FLARE (Few-shot Learning-based Adaptive Reflective Reflective Engine), a framework that leverages advanced reflective mechanisms and a small set of few-shot reference examples to optimize instructions to optimize instructions.
Dhanasekar Sundararaman, B. Gandhi, Aashna Garg et al.· 0 citations
We propose a novel approach to bug triage in the TianoCore open-source UEFI firmware development ecosystem. This integrated approach, called TianoForge, deploys the state of the art in artificial intelligence, specifically machine learning, to enable automated bug triage. This includes invalid bug report detection, duplicate bug report detection, bug report prioritization, and bug report assignment. We use various Generative Pretrained Transformer (GPT) Large Language Models (LLMs) with and without Retrieval Augmented Generation (RAG) to automate these tasks. Given the crucial role of bug triage in software maintenance and the huge number of untriaged issues in the TianoCore community, in particular, their primary project, EDK II, we expect a significant impact on the efficiency of TianoCore software maintenance processes, primarily bug triage and resolution. Our experimental study shows that TianoForge reduces the average bug triage time from around 11 days to approximately 7 minutes, which is a 99.95% reduction.
Nazanin Siavash, Terrance E. Boult, Armin Moin· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
