InterHier: Learning Interconnected Hierarchical Semantics for Open-Vocabulary Object Detection

Abstract

Open-vocabulary object detection (OVD) aims to localize and classify objects from arbitrary categories. These categories are specified through textual input and are not limited to a predefined set. The existing methods explicitly utilize hierarchical semantic representations of super-/sub-categories to establish semantic relationships between base categories and unseen novel categories. These methods rely on a fixed connector such as “, which is a”, and this fixed connector is placed between each adjacent super-/sub-category to integrate them together. However, these methods are not an optimal solution because these relationships rely on hand-crafted connectors. To address this issue, we propose interconnected hierarchical semantic representations (InterHier). InterHier utilizes a prepended learnable context to globally guide the interpretation of prompts containing hierarchical relationships. InterHier operates in two main stages. In the first stage, InterHier constructs the hierarchy-aware prompt by integrating super-/sub-categories and then prepending a learnable context. In the second stage, InterHier optimizes this learnable context to align the visual region embeddings and textual embeddings. InterHier demonstrates consistently improved performance over methods that rely on fixed connectors. Moreover, InterHier offers high versatility and allows for seamless integration into existing OVD models. On OVD benchmarks, InterHier achieves competitive performance against other state-of-the-art methods.