Skip to content

UNBIND: UNlearning By INference-time Directional Steering for Code LLMs

Sep 2026 · 0 citations · 35 references
Computer Science

TL;DR

Multilingual and related-code evaluations further show effective forgetting with limited impact on useful programming capabilities, supporting UNBIND as a practical approach to selective code unlearning.

Abstract

Code large language models acquire programming capabilities from large code corpora, but can also memorize implementations that later require removal. Code unlearning is needed to control their continued reproduction when copyright or security concerns arise. However, targeted and retained code share computational patterns, creating a tension between forgetting specific implementations and preserving general programming ability. We propose \textbf{UNBIND}, a code unlearning framework that separately considers which hidden states correspond to the target code and how to suppress its reproduction. By constructing separate directions for these objectives, UNBIND achieves selective unlearning at inference time while keeping model weights fixed. Our evaluation covers fourteen baselines across two code models and two corpora. UNBIND achieves the highest joint forgetting and utility score in every setting. It reduces target code reproduction by 97.3\% to 99.1\% as measured by F-BLEU, with at most two fewer HumanEval+ and six fewer MBPP+ problems solved than the original models. In repeated extraction tests under a fixed budget, the number of targets yielding exact spans of at least 50 tokens falls from 188--262 to 0--2 out of 300 per setting. No extracted span reaches 100 tokens, and the mean best recovery ratio ranges from 0.43\% to 6.45\%. Multilingual and related-code evaluations further show effective forgetting with limited impact on useful programming capabilities, supporting UNBIND as a practical approach to selective code unlearning.

View source

Similar papers

UnlearningSoup: Is Repeated Tuning Necessary for Large Language Model Unlearning?

Whether unlearning runs exhibit exploitable structure in weight space is investigated, and it is observed that models from different runs still lie in a shared evaluation-performance basin, suggesting that stronger models may be recovered through an unlearning-tailored soup strategy, reducing the need for repeated tuni...

Pu-Ning Yang, Qi-Zhou Wang, Jun-Chi Yu et al. · 0 citations
Preprint Sep 2026

What Was Once Learned May Need to Be Unlearned: Machine Unlearning for Deprecated API Knowledge in Large Language Models

Large language models (LLMs) for code completion may generate deprecated APIs because their pre-training corpora contain code from historical library versions. Existing approaches use inference-time intervention, model editing, or machine unlearning, but multiple plausible completions make predefined replacements restr...

Jin Liu, Yan-Zhong He, Guan-Cheng Lin et al. · 0 citations
Preprint Aug 2026

Unlearning Is Not Just Erasing: Temporal Decoupling via Generation Inequality

ADU is presented, a fine-grained, training-based framework that shifts unlearning from token erasure to contextual attention-pathway decoupling, and achieves the strongest aggregate performance among evaluated baselines on the TOFU and WMDP benchmarks.

Xun-Lei Chen, Qi-Rui Ye, Yuang Li et al. · 0 citations
#natural language process... Preprint Oct 2026

Not Every Change Is Necessary: Recoverable Drift in Large Language Model Unlearning

Machine unlearning in large language models aims to remove unwanted knowledge while preserving the model's remaining capabilities. Although existing methods use retention objectives or restrict where edits occur, achieving the desired forgetting level can still leave collateral changes that impair non-target behavior....

Xun-Lei Chen, Qing Gong, Jing-Kun Xue et al. · 0 citations

Test-Time Unlearning via Sparse Autoencoder

This work proposes ARIA (autoencoder-gated inference-time unlearning), a test-time unlearning method that leaves model weights intact and gates access to unwanted knowledge only when generation enters a forget-related state and introduces three post-unlearning adversarial attacks targeting weight-space and decoding-spa...

Ping-Zhi Li, Jinhao Duan, Vaishnav Tadiparthi et al. · 1 citation
#artificial intelligence Preprint Aug 2026

Evaluating Tiny Recursive Models Across Training for Code Generation

This work compares a ~28M-parameter autoregressive Tiny Recursive Model (TRM-AR) on natural-language-to-Python code generation against parameter-matched and depth-matched controls, tracking fit and generation across 40 epochs and three seeds.

Anjani Sirivella, Aanisha Newaz, Glaucia Melo · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.