The paper's two contributions are a negative result for native KDA's tested receipt classes and a positive training-free construction for addressable pretrained memory for addressable pretrained memory.
Abstract
Exact deletion from persistent language-model memory depends on whether a record's effect remains addressable after later computation. Native Kimi Delta Attention (KDA) gives a negative result for the tested receipt interface: the corpus-pooled raw recurrent contribution changes by 12-49% with the suffix and remains 8-49% after a decay-ledger correction. Native omission also changes later transition and write terms and other active caches. Frozen-input transport succeeds on its fixed-input control; the changed terms place native omission outside the tested receipt classes. Checkpoint replay supplies the evaluated recomputation path; zero residual on final logits and all 80 audited KDA arrays verifies restoration across the declared checkpoint surface. The complementary result is constructive. We retrofit support-vector memory into frozen Gemma 3 without attention transfer, low-rank recovery, distillation, adapters, or language-model parameter updates. Prefix-mass preservation and one box per prefix solve give base-matched admission at 4B with 1.85% perplexity overhead. At 1B and 4B, verified deletion agrees with its conditional retained-key refit within 1.3e-10 maximum next-token KL; behavioral attacks at 4B reach never-stored or chance baselines. Across 1B, 4B, and 12B, the 4B checkpoint uniquely combines base-matched admission with low overhead. The paper's two contributions are a negative result for native KDA's tested receipt classes and a positive training-free construction for addressable pretrained memory.
Auditable memory requires a precise contract: which output is preserved, relative to which reference solve, and across which updates. We introduce Support Vector Attention (SV-Attention), a one-class support vector data description (SVDD) gate whose coefficients enter the readout. Zero-coefficient keys are reserve; positive-coefficient keys are active. Removing a reserve token without re-solving preserves the current readout. Maintained deletion, which updates the existing solver state, targets a fresh retained-key fit under the same coefficient cap C. Across 1,200 fp64 deletion/refit trials on Gaussian, redundant, MIMIC-IV, and learned keys, 1,199 complete. Median maximum gate-score discrepancy over declared probes ranges from 4.5e-13 to 5.7e-7, and fresh refitting is 24-223 times slower than maintained deletion on the reference CPU. A deterministic example shows that current reserve status does not guarantee equivalence after future admissions. At matched token counts, rare-group recall is 0.861 versus 0.319 for an oracle attention-mass proxy (H2O-style). In a held-out-channel ICU control, SpO2 below 90 percent defines events but SpO2 is excluded from every selector; event-hour retention is 0.464 versus 0.225 for an RBF-density baseline. A separate batched approximation supports end-to-end training; at 3.22M parameters, seven paired seeds yield mean best-validation bits per character of 2.178 versus 2.383 (p=0.001). The contracts are point-in-time and fixed-C; future-safe streaming and general-purpose performance remain open, and larger fixed-step runs suggest slower optimization.
A falsifiable certificate is presented separating representation, semantic demand, scheduler requests, and traffic while naming resource authorities, exactness horizons, and tested reuse transitions while naming resource authorities, exactness horizons, and tested reuse transitions.
MemTxn is a governance layer outside the answer model that verifies whether an update is supported by its source and restores the application-visible state after a fault, and achieves the highest average F1 across all twelve answer-model configurations.
Han-Shuai Cui, Zhiqing Tang, Z. Yao et al.· arXiv.org· 2 citations
Right Reset (RR) is introduced and an observed-token likelihood-ratio readout is competitive in some architectures, indicating that the central contribution is the intervention: context dependence itself can provide a boundary signal when surface structure is weak.
GPM is introduced, an auditable bitemporal state-transition model with source-bound admission, derived lifecycle state, current public barriers, and fail-closed structured release with bounded contract and implementation results, not open-world model accuracy or evidence of world truth.
LLM coding agents issue Bash commands through interfaces that may serialize, wrap, and reparse model output. Matched execution scores alone cannot distinguish command-generation errors from failures introduced after generation. QuoteBench measures this boundary with exact final-state validation on 56 one-shot tasks from 14 incident-derived families, crossing the generation contract with the execution transport around one deliberately unescaped added parser. Escaping at the interpolation point reproduces each replayed reply's raw-path outcome, so any recovery under a disclosed boundary must come from the model changing its generation. Across eight same-window configurations, replaying the same reply through the added parser lowers success by 55.4 to 73.2 percentage points; disclosure recovers 30.4 to 60.7 points for six configurations, and zero or slightly negative for the other two. Raw generation is nearly saturated at the frontier; boundary adaptation is what still separates models. GPT-5.6-sol's matched gap of -3.6 points hides -64.3 points of damage and +60.7 points of compensation. The deployment configuration reorders models: one reversal among 26 comparable pairs is unambiguous and four more sit on single-task margins. Evaluations of command-issuing agents should report the model configuration, generation contract, execution path, operating point, and final-state validator rather than treat a matched score as an intrinsic model property.
Shangao Li, Yao Zhang, Volker Tresp et al.· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduSep 2, 2026
A new method, called CW-Net, translates the reasoning process of an autonomous vehicle’s AI system into understandable concepts that explain its behavior.
MIT News · Artificial Intelligence· news.mit.eduAug 31, 2026
With millions of users across the world, Julia has been used to conduct cutting-edge research and to design new drugs, jet engines, heat pumps, and more.
A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
