This work proposes MemCatalyst, a set of data poisoning tools, aiming to amplify the data auditing performance on VLMs, and forces VLMs to over-learn specific inconsistencies between image features and textual semantics during training, thereby increasing their susceptibility to membership information auditing.
Abstract
Vision-Language models (VLMs) achieve outstanding performance largely due to the amount of training data available on the internet. At the same time, data holders (e.g., artists) urgently need to determine whether their data has been used for model training without authorization, which concerns both intellectual property rights and personal privacy. Data auditing, particularly through membership inference (MI), has attracted attention as a direct tool. This work proposes MemCatalyst, a set of data poisoning tools, aiming to amplify the data auditing performance on VLMs. MemCatalyst employs two strategies: Poisoning Text (PT) and Poisoning Image (PI). MemCatalyst forces VLMs to over-learn specific inconsistencies between image features and textual semantics during training, thereby increasing their susceptibility to membership information auditing. Crucially, the transferability of poisoned samples across different VLM architectures is demonstrated to be effective in the black-box setting. Extensive evaluations using five state-of-the-art data audits on two prominent VLMs demonstrate that MemCatalyst markedly enhances MI AUC scores with a minimal budget of poisoned samples, while maintaining a negligible impact on model performance.
Training-time data poisoning during fine-tuning poses a significant threat to large language models (LLMs) deployed for abstractive text summarization, where small task-specific datasets exert disproportionate influence on model behavior. In this setting, adversaries manipulate fine-tuning data to induce persistent summarization failures, such as biased or harmful summaries, while preserving standard evaluation metrics. We present a unified post-hoc defense framework for detecting and remediating fine-tuning-stage poisoning in summarization models across the machine learning supply chain. Our experiments show that in white-box settings, poisoned document-summary pairs exhibit abnormally high training influence, enabling detection via influence-function analysis with semantic consistency checks. In black-box settings, poisoned models display two to three times greater sensitivity to semantics-preserving perturbations, enabling behavioral auditing without training data access. Beyond existing poisoning formulations, we introduce novel attacks targeting factual distortion and representational bias, showing that poisoning alters summarization behavior without triggering conventional alarms. Across nine architectures and six benchmark datasets under adaptive attacks, our defenses achieve 85-92% detection precision, while gradient-ascent unlearning restores up to 96% of original behavior with minimal utility loss (less than 0.6% ROUGE degradation). These results indicate that fine-tuning-time poisoning leaves persistent structural artifacts, enabling practical detection and post-deployment recovery without full retraining.
This work demonstrates the importance of estimating whether poison injections are included in pretraining data, and establishes third-party webpage content as a possible vector for attacking language model pretraining.
Victoria Graf, Hanna Hajishirzi, Noah A. Smith et al.· 0 citations
This work proposes a novel information overloading method that is equipped with both extensive text and multi-dimensional image attacks, underscoring the need for stronger defenses against complex multimodal jailbreak inputs.
Haoyu Zhang, Yangyang Guo, Mohan S. Kankanhalli· 0 citations
Phishing attacks via desktops, smartphones and internet of things devices are becoming increasingly sophisticated, posing critical security challenges for digital infrastructures. Defending against these attacks requires AI-based detection models that maintain high accuracy, since false positives or negatives can lead to severe breaches, while remaining lightweight enough to run on resource-constrained client devices. Split Learning (SL) meets these requirements by having clients compute only initial model layers locally and transmit intermediate activations (“smashed data”) to a server for the remaining inference, avoiding direct sharing of raw inputs. However, prior work in the image domain has shown that smashed data can leak original content, suggesting that SL may not be safe for user privacy. Therefore, it is essential to investigate whether these privacy risks also extend to language-model–based SL systems, which have fundamentally different neural network architectures, including attention mechanism. This paper introduces the Semantic Information Reconstruction Attack (SIRA), a novel framework designed to infer sensitive semantic elements directly from smashed data by leveraging the generative capabilities of large language models. In experiments on real-world phishing datasets, SIRA outperforms conventional reconstruction attacks in accurately inferring private webpage information. These findings reveal a potential privacy vulnerability in SL-based language models for security applications and motivate the development of targeted defense strategies.
Yushin Kim, Jungin Kim, Yongseok Kwon et al.· International Journal of Inf...· 0 citations
Leveraging capabilities of large language models (LLMs) in text-to-image (T2I) synthesis is an important research direction. In this work we investigate whether the knowledge of a frozen LLM can be effectively utilized in T2I generation when trained exclusively on standard text-image pairs. We integrate a frozen, reasoning-capable LLM with a diffusion-based image generator via shared attention within the Mixture-of-Transformers (MoT) architecture. Our experiments span two critical questions: (1) what degree of the LLM's intrinsic knowledge remains accessible during T2I training, and (2) what novel capabilities emerge in the resulting system. Across established benchmarks, our models achieve strong performance among unified understanding-generation systems: 0.85 on GenEval, 86.75 on DPG-Bench, and 0.66 on WISE with inference-time reasoning, using only text-image data. Remarkably, we uncover emergent behaviors absent from training data, including cross-lingual image generation, color-guided composition, emoji / ASCII scene construction, and generation directed by world knowledge. These results demonstrate that pretrained LLM knowledge can guide image synthesis under standard text-to-image training paradigms, without interleaved multimodal signals or explicit reasoning supervision. Our findings open new avenues for harnessing frozen model capabilities in resource-constrained multimodal learning.
Achin Jain, Jie An, Siddharth Chaudhary et al.· arXiv.org· 0 citations
These findings endorse the viability of LLMs as a feasible alternative for data error detection; nonetheless, challenges such as hallucination and cost aspects remain, especially regarding the resources necessary for the effective implementation of these models in practical applications.
Maximilian Plazotta, Meike Klettke· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduAug 18, 2026
A new method for surgically removing training examples from a model reveals that as datasets grow, the link between what a model learns and what it produces dissolves.
