Back to #software testing

Unsaid, Unsafe? Implicit Security Obligations in LLM-Based RTL Code Generation

Guang Yang Xing Hu Xiang Chen Xin Xia
Aug 2026 · 0 citations · 31 references
Computer Science

TL;DR

RTL-Obliger is presented, a neuro-symbolic framework that infers implicit security obligations of register-transfer-level RTL in a functionality-preserving two-stage generation and raises mean all-pass rates.

Abstract

Large Language Models (LLMs) generate register-transfer-level (RTL) code with rapidly improving functional correctness. Security of LLM-generated code, however, has been studied mainly for software, where flaws can still be patched after deployment. Insecure RTL offers no such remedy once taped out into silicon. We construct SECRTL-GEN, a multi-language resource-access security benchmark grounded in real SoC IP: 392 tasks over five CWE families and four HDLs (Verilog, SystemVerilog, VHDL, and Python), each with black-box functional and security testbenches. Functional specifications intentionally omit security obligations, matching how obligations are often kept out of functional docs in practice. An empirical study of five frontier LLMs shows a sharp gap: under vanilla prompts they pass functional tests in about 73-79% of cases but security tests in only 14-35%, and stronger functional models are not safer. Adding CWE knowledge raises security, while unaided self-thinking helps less and both security-oriented prompts cut functional pass rates, showing that the bottleneck is missing weakness awareness in the specification, not an inability to write defensive RTL. We present RTL-Obliger, a neuro-symbolic framework that infers these implicit obligations. An LLM extracts a functional-semantic graph from the specification; a symbolic engine then matches it against a CWE pattern ontology to surface mitigation-evidence gaps and signal-level obligations; the LLM finally revises RTL under those obligations in a functionality-preserving two-stage generation. Across five models and four languages, RTL-Obliger raises mean all-pass from 49.6-51.4% (SecV/RESCUE) to 61.6%, with higher security and functional rates than these secure-generation baselines.

View source

Similar papers

#software testing Preprint Aug 2026

Evaluating Inference-Time Defenses Against Package Hallucination in LLM-Generated Code

LLMs are increasingly used for code generation, yet they frequently hallucinate non-existent software packages, creating exploitable entry points into the software supply chain. We make four contributions to this problem. First, we show that prior evaluation methodologies systematically inflate hallucination rates by misclassifying standard-library modules as hallucinations in some languages. For Python, the overestimation reaches 9.4 percentage points. Second, we evaluate seven inference-time defenses for mitigating package hallucinations, including five guided decoding strategies (Greedy, Contrastive, DoLa, Nudging, and Active Layer-Contrastive Decoding), an iterative self-refinement approach (Self-Refine), and a Retrieval-Augmented Generation (RAG)-based defense.. Across eight models spanning five families and four programming languages (Python, JavaScript, Ruby, Rust), RAG reduces the package hallucination rate (PHR) in 18 of 32 model--language configurations. Third, we introduce Package Utility (PU) to assess whether defenses preserve valid and task-relevant recommendations. Among strategies evaluated, Greedy decoding provides the strongest average mitigation--utility trade-off. Fourth, we stress-test all strategies under adversarial prompts seeded with fabricated package names and find that PHR surges by up to 45 percentage points relative to standard prompts, with Ruby consistently the most vulnerable language (80.9--95.2\%). Under adversarial conditions, RAG and Self-Refine outperform all decoding-only strategies, indicating that robust defense requires either external grounding or iterative self-verification when prompts are actively hostile. Our results recast package hallucination as both a measurement problem and a decoding-time control problem, and they demonstrate that the choice of defense must be matched to the threat model and recommendation utility.

Albérick Euraste Djiré, Iyiola E. Olatunji, Melissa Tessa et al. · 1 citation
#software testing Review Aug 2026

Model-Based Agentic Software Engineering

MAGE explains how externalized knowledge, bounded action, independent evaluation, and retained human authority can compose into a governed engineering environment, and proposes tests of when that environment turns commodity intelligence into durable engineering progress.

James C. Davis, Kelechi G. Kalu, Huiyun Peng et al. · 1 citation
#software testing Review Aug 2026

"A Second Set of Eyes": The Process and Challenges of Software Documentation Review

The work identifies five distinct stages of the documentation review process: self review, technical review, editorial review, play testing, and post-publication feedback, and draws on practitioners with distinct expertise to address quality across content, presentation, and user experience.

Avinash Bhat, Ian Arawjo, Disha Shrivastava et al. · 0 citations

Mixed Reality Glasses Image Translocation for Binocular Diplopia.

This prototype MRG image translocation software was helpful to 69% of patients with binocular diplopia, but limited by large angle strabismus because of the limited instrument field of view.

Edsel B Ing, Kevin Sha, Sarosh Dandoti et al. · 0 citations
#software testing Open access Aug 2026

Multi-Disease Prediction Using Machine Learning: A Web-Based Diagnostic Support System for Diabetes, Heart Disease, and Parkinson\'s Disease

A diagnostic support system based on a unified web platform that classifies patients according to the risks of developing three diseases based on regularly collected clinical or audio data using classical supervised learning algorithms is presented.

Vedamurthy D R, Dr. Anup Ritti, A. Bibi et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Aug 17, 2026

Q&A: Rethinking how innovation happens

In his latest book, Professor Eugene Fitzgerald examines the forces that turn breakthroughs into value — and why innovation resists simple formulas.