The question-first paradox reveals a tension between steering what a model sees and preserving access to what it was asked; echoing resolves this through prompt design.
Rakshanda Hassan Abhinandan, J. Galeotti, Deva Ramanan et al.· arXiv.org· 1 citation
This work proposes TEVI, a framework that uses captions as a signal for what to retain from image embeddings and uses sparse autoencoders to disentangle image embeddings and train a masking module to selectively reconstruct the embedding based on a given caption.
S. Mahajan, Sukrut Rao, Jiahao Xie et al.· arXiv.org· 1 citation
LRConv–NeRV, an efficient NeRV variant that replaces selected dense convolutional layers with structured low-rank separable convolutions, trained end-to-end within the decoder architecture, is proposed, establishing LRConv–NeRV as a potential architectural alternative for efficient neural video decoding under low-preci...
Omni-modal large language models (OLLMs) aim to unify multimodal understanding and generation, yet extending them to jointly produce speech and 3D facial animation remains largely underexplored. A key challenge is the mismatch between the discrete semantic reasoning of LLMs and the dense temporal dynamics required for...
Haoyu Zhang, Zhipeng Li, Yiwen Guo et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
AnyBox is presented, an efficient zero-shot framework that exploits the geometric regularity of boxes to jointly recover pose and dimensions from a single RGB-D observation, and approaches instance-level pipelines that have access to ground-truth CAD models.
Yintao Ma, Sajjad Pakdamansavoji, Charles Eret et al.· 0 citations
A Short-Window Sliding Learning framework for real-time violence detection in CCTV footages that divides videos into 1-2 second clips and applies Large Language Model (LLM)-based auto-caption labeling to construct fine-grained datasets, enabling precise recognition of rapid violent events.
Seoik Jung, Taekyung Song, Yan-Cen Lee et al.· arXiv.org· 1 citation
Wildlife field operations demand efficient parallel deployment methods to identify and interact with specific individuals, enabling simultaneous collective behavioral analysis, and health and safety interventions. Previous robotics solutions approach the problem from the herd perspective, or are manually operated and l...
Makram Chahine, William Yang, Alaa Maalouf et al.· 0 citations
A taxonomy of reasoning enhancement techniques is proposed, categorized into training-time strategies (e.g., supervised fine-tuning, reinforcement learning) and test-time mechanisms (e.g., prompt engineering, multi-agent systems), and outlining future directions toward building efficient, robust, and sociotechnically r...
Zi-Zhan Ma, Wen-Xuan Wang, Meidan Ding et al.· 16 citations
This paper proposes ScoreMix, a self-contained synthetic generation method to produce hard synthetic samples for recognition tasks by leveraging the score compositionality of diffusion models, and finds that mixing classes distant in the discriminator's embedding space yields larger gains, compared to selection based o...
Omni-modal dialogue models can understand multimodal inputs and synthesize spoken replies, but a spoken answer still leaves the agent visually absent. We introduce \textbf{Ex-Omni-2D}, a framework that answers a multimodal query with coordinated text, personalized speech, and reference-conditioned video. The dialogue m...
Haoyu Zhang, Zhipeng Li, Xiaoying Tang et al.· 0 citations
It is demonstrated that even with multi-billion parameter models and extensive training, current Vision Language Models fall short in the seemingly simple task of tool detection in neurosurgery, and experiments suggest that current models could still face significant obstacles in surgical use cases.
K. Skobelev, Eric Fithian, Yegor Baranovski et al.· 0 citations
Video editing spans diverse editing paradigms, yet achieving high-quality instruction-guided and subject-guided editing within a single unified framework remains challenging. We introduce EditVid, a training-free framework combining sparse causal memory for local coherence, correspondence-based post-attention token inj...
A. Juvekar, O. Susladkar, K. A. Nguyen et al.· 0 citations
Exploring how generative AI could make machine vision more accessible to businesses. The post GenEye in a Box: Making Machine Vision Something You Can Just Ask For appeared first on GPT-Lab.
Jennifer Neville did not want to go into computer science—but that’s exactly where she landed. Neville discusses the starts and stops that led to her professional sweet spot and her work identifying “surprising failures” making it hard for AI to handle complexity. The post What AI gets wrong and what failure teaches us appeared first on Microsoft Research.
Computer scientist, entrepreneur, and philanthropist will collaborate with the MIT Schwarzman College of Computing to advance AI and scientific discovery.