MemCatalyst: Amplifying Data Auditing on Vision-Language Models via Data Poisoning
This work proposes MemCatalyst, a set of data poisoning tools, aiming to amplify the data auditing performance on VLMs, and forces VLMs to over-learn specific inconsistencies between image features and textual semantics during training, thereby increasing their susceptibility to membership information auditing.
Xukun Luan, Jinyan Liu, Yuhui Gong et al.
· 0 citations