Similarity Is Not Logic: Factored Inference for Dual-Encoder Vision-Language Models
Despite-encoder vision-language models expose a similarity interface that enables zero-shot retrieval but fails compositional constraints, this work proposes factored inference, which separates evidence extraction from constraint execution, and introduces LCSE (Logic-Constrained Score Editing), a training-free method t...