Author

Derek Hoiem

h-index 5 View on Semantic Scholar

We have 6 of 21 papers

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

Preprint Oct 2026

Render to Reason: Novel-View Semantic Prediction Improves Spatial Understanding in VLMs

Recent works augment Vision-Language Models with geometry features from pretrained 3D models, expecting that the geometric signal will boost spatial reasoning. However, we find that simply fusing geometry features and training on standard spatial QA yields only marginal improvements on high-level multi-hop tasks. We at...

Yuqun Wu, Yao Xiao, Chu-Hang Zou et al. · 0 citations

#machine learning Preprint Sep 2026

SAM-V: Geometry-Aware Segment Anything for Multi-View Instance Segmentation

SAM-V introduces a prompt-fusion mechanism that enriches sparse SAM prompt tokens with view-specific camera tokens and local VGGT features, making the prompt representation both view-aware and spatially grounded, together with a mask decoder that attends to dense 2D and 3D features.

Jiang-Shan Gong, Yuqun Wu, Qiqian Fu et al. · 1 citation

#large language models Preprint Sep 2026

ViTAL-X: Video-Text Alignment with Cross-Modal Temporal Edits

ViTAL-X, a lightweight model that equips frozen image-text backbones with temporal awareness while preserving their foundational spatial knowledge, achieves state-of-the-art performance and demonstrates that targeted, high-quality temporal alignment provides a highly efficient alternative to pure scaling.

V. SethuramanT., Savya Khosla, O. Susladkar et al. · 0 citations

#artificial intelligence Apr 2026

Why Fine-Tuning Encourages Hallucinations and How to Fix It

This work proposes a self-distillation-based SFT method that facilitates effective factual learning while minimizing hallucinations w.r.t.~pre-existing knowledge by regularizing output-distribution drift and investigates the mechanism, contrasting capacity limitations, behavior cloning, and localized interference.

Guy Kaplan, Zorik Gekhman, Zhen Zhu et al. · arXiv.org · 5 citations

Preprint Aug 2026

Fine-Tuning VLAs with Self-Demonstrated Generative Control for Multi-Task Manipulation

This paper proposes a self-supervised method that generates online interaction rollouts from the zero-shot VLA as additional training data for finetuning and demonstrates the success of this approach across test sets probing generalization on a real ALOHA robot and a new simulation benchmark in RoboTwin.

Prachi Garg, Steve Xing, Prahit Yaugand et al. · 0 citations

#machine learning Jul 2026

ReToken: One Token to Improve Vision-Language Models for Visual Retrieval

ReToken is a single learnable embedding trained as an explicit retrieval target that selects a sparse set of query-relevant visual tokens from a pre-filled visual KV cache that yields consistent gains across image and video benchmarks.

Yao Xiao, Reuben Tan, Zhen Zhu et al. · arXiv.org · 0 citations