Learning to Adapt: Instance-Level Visual and Semantic Adaptation for Generalized Zero-Shot Learning
Abstract
Generalized zero-shot learning (GZSL) aims to recognize novel categories by leveraging semantic knowledge transferred from previously observed categories, where learning transferable features for effective visual-semantic alignment plays a critical role. Conventional methods typically utilize all shared attributes to learn semantically related features. However, not all attributes are related to a specific instance, and the inconsistency may result in less effective image features. Additionally, the visual features extracted by the pre-trained backbone may be unsuitable for a particular sample, reducing the discrimination. To address these problems, we propose a novel instance-level visual and semantic adaptation framework that effectively adapts pre-trained image features for the GZSL tasks. From the semantic adaptation aspect, we propose to choose instance-specific attributes for dynamic semantic prompt tuning. From the visual adaptation aspect, we construct instance visual prototypes to produce channel attention, which adaptively strengthens crucial visual features for each sample. Extensive experiments on three GZSL benchmark datasets demonstrate that our approach achieves the new state-of-the-art performance.