Similar papers
Model-Based Agentic Software Engineering
MAGE explains how externalized knowledge, bounded action, independent evaluation, and retained human authority can compose into a governed engineering environment, and proposes tests of when that environment turns commodity intelligence into durable engineering progress.
Evaluating Inference-Time Defenses Against Package Hallucination in LLM-Generated Code
LLMs are increasingly used for code generation, yet they frequently hallucinate non-existent software packages, creating exploitable entry points into the software supply chain. We make four contributions to this problem. First, we show that prior evaluation methodologies systematically inflate hallucination rates by misclassifying standard-library modules as hallucinations in some languages. For Python, the overestimation reaches 9.4 percentage points. Second, we evaluate seven inference-time defenses for mitigating package hallucinations, including five guided decoding strategies (Greedy, Contrastive, DoLa, Nudging, and Active Layer-Contrastive Decoding), an iterative self-refinement approach (Self-Refine), and a Retrieval-Augmented Generation (RAG)-based defense.. Across eight models spanning five families and four programming languages (Python, JavaScript, Ruby, Rust), RAG reduces the package hallucination rate (PHR) in 18 of 32 model--language configurations. Third, we introduce Package Utility (PU) to assess whether defenses preserve valid and task-relevant recommendations. Among strategies evaluated, Greedy decoding provides the strongest average mitigation--utility trade-off. Fourth, we stress-test all strategies under adversarial prompts seeded with fabricated package names and find that PHR surges by up to 45 percentage points relative to standard prompts, with Ruby consistently the most vulnerable language (80.9--95.2\%). Under adversarial conditions, RAG and Self-Refine outperform all decoding-only strategies, indicating that robust defense requires either external grounding or iterative self-verification when prompts are actively hostile. Our results recast package hallucination as both a measurement problem and a decoding-time control problem, and they demonstrate that the choice of defense must be matched to the threat model and recommendation utility.
Auditing and Decomposing Feedback-Driven Evolution in LLM Test Generation under the Oracle Problem
An audit-and-placebo protocol is proposed that separates verifier artifacts, interaction scaffolding, and grounded feedback credit in evaluations of self-evolving test generators in evaluations of self-evolving test generators.
Multi-Disease Prediction Using Machine Learning: A Web-Based Diagnostic Support System for Diabetes, Heart Disease, and Parkinson\'s Disease
A diagnostic support system based on a unified web platform that classifies patients according to the risks of developing three diseases based on regularly collected clinical or audio data using classical supervised learning algorithms is presented.
Mixed Reality Glasses Image Translocation for Binocular Diplopia.
This prototype MRG image translocation software was helpful to 69% of patients with binocular diplopia, but limited by large angle strabismus because of the limited instrument field of view.
Prevalence of non-union complication in femoral shaft fracture patients who underwent IM nailing surgery in Fatemi Hospital Ardabil.
The findings of the present study indicated that potential complications such as delayed union, nonunion, and osteomyelitis in the intramedullary nailing method are approximately comparable to those of the external fixator method.
Related blog posts
Q&A: Rethinking how innovation happens
In his latest book, Professor Eugene Fitzgerald examines the forces that turn breakthroughs into value — and why innovation resists simple formulas.
Can AI build a jet engine? JARVIS Challenge tests role of AI copilots in tough-tech engineering
MIT students designed, built, and tested a jet engine with AI copilots, assessing AI’s usefulness in developing high-performance aerospace systems.
New method aims to keep kids safe from illegal AI-generated content
Researchers developed an auditing technique to test generative AI models for malicious capabilities, without prompting them for illegal outputs.
How novice coders can develop AI programs for military applications
A USAF cadet and a Lincoln Laboratory researcher found AI chatbots can help nontechnical service members produce viable software applications for their unique problems.