Omnimodal embeddings naturally involve both shared representations and modality-specific features across heterogeneous inputs. However, existing omnimodal embedding methods often rely on a single shared parameter space over mixed-modality data, limiting structural separation between universal and modality-specific repr...
Youngtaek Oh, Qi-Yu Wu, Hiromi Wakaki et al.· 0 citations
KME-CLIP is proposed, which leverages this structure through the inner product in a reproducing kernel Hilbert space (RKHS) and theoretically proves that the KME-CLIP similarity can bring the contrastive loss arbitrarily close to its optimal value, which is attained by PMI, as the size of the point set grows.
This paper theoretically analyzes the MaskGIT sampler for image modeling, revealing its implicit temperature sampling mechanism, and shows that MaskGIT is asymptotically equivalent to a choose-then-sample (CTS) formulation, instantiated as the "moment sampler," which explicitly separates index selection from token samp...
MusTBench, a music-expert-validated benchmark designed to evaluate temporal grounding in LALMs through five temporally grounded question-answering tasks, is introduced, establishing temporal grounding as a key missing capability in current LALMs and position MusTBench as a challenging benchmark for future research in t...
Daeyong Kwon, Qiyu Wu, S. Kuriya et al.· arXiv.org· 3 citations
A dynamic schema-guided world model optimized for visual dynamic prediction and simulation, DynaVieW, achieves an in-depth understanding of visual dynamics by learning interleaved state-transition sequences, where states cover broad visual scenes from video keyframes and transitions capture comprehensive dynamic consti...
VideoTreeSearch (VTS) is proposed, a framework that casts grounded LVQA as iterative self-correcting search over an adaptive temporal tree, and trains an agent to navigate the tree through four discrete operations: zoom_in, zoom_out, shift, and answer.
Ce Zhang, Ziyang Wang, Yu-Lu Pan et al.· arXiv.org· 0 citations
Compared to prior CLIP-enhancement methods, MLLMCLIP achieves state-of-the-art compositional accuracy while delivering consistent gains on standard zero-shot classification and image-text retrieval, showing that feature-level distillation strengthens both compositional and general vision-language representation capabil...
Jongsuk Kim, Qi-Yu Wu, Zhuoyuan Mao et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.