SSMH: Semantic Self-refinement Multi-modal Hashing for Multi-modal Retrieval

Abstract

Multi-modal hashing converts high-dimensional features into compact binary codes while preserving semantic similarity, thereby achieving high performance with lower storage and computational costs. However, existing methods generally overly depend on discrete supervisory information, ignoring the ambiguity of labels and intra-class diversity. To address these challenges, we propose a novel Semantic Self-refinement Multi-modal Hashing (SSMH), which adaptively learns stable semantic representations through a unified end-to-end semantic learning. First, we dynamically integrate inter- and intra- modal correlations to achieve semantically consistent multi-modal integrated representation. Then, we generate prototype-guided binary codes to explicitly align multi-modal representations with label semantics, thereby strengthening supervision and mitigating the effect of ambiguous labels. Extensive experiments show SSMH outperforms state-of-the-art methods in multi-modal retrieval. The source code can be found at https://github.com/vindahi/SSMH.