Katarzyna Kapusta

Breaking the weakest link to evade vision language models

To efficiently generate adversarial examples, a gradient-based attack method is proposed that performs optimization exclusively on the vision encoder of the VLM rather than on the entire multimodal architecture, which significantly reduces the computational cost and resource requirements of the attack while maintaining strong effectiveness.