GUI World Models (GUI-WMs) are increasingly used to predict future states for agent planning and simulation, yet most existing formulations condition only on the current GUI observation and action. We identify state aliasing, where the vis- ible interface omits transition-relevant environment state, so identical observ...
Dongsheng Liu, Chao Jin, Wenkui Yang et al.· 0 citations
ELBO-based reinforcement learning offers a sampler-agnostic approach to fine-tuning flow matching models with reward feedback. Timestep weighting in ELBO-based RL has large impact on performance, and it also provides a unified view (as we show in this work) to understand prediction losses heuristically chosen in prior...
Qin-Wei Ma, Jing-Zhe Shi, Si-Min Fan et al.· 0 citations
InterTab is a structure-aware framework for CoT reasoning over table images that interleaves chain-of-thought with tool calls that crop structure-aligned table regions, and improves the average accuracy of its backbone from 68.28% to 73.17% and achieves the best average performance among all compared methods.
Hanqian Li, Si-Rui Huang, Chen Ling et al.· 0 citations
Video generation has broad applications in content creation and entertainment. Diffusion transformers have advanced the quality of generated videos, but attention over spatiotemporal tokens becomes increasingly expensive as video resolution and duration increase. We present DraftAttention2, a training-free framework th...
Rui Ding, Haopeng Li, Wei-Ze Ma et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Feed-forward models regress dense 3D point trajectories directly from monocular video, yet the residual after global alignment is substantial and lacks a structural explanation. Measured on dynamic query points across models and datasets, the error concentrates along the view direction, while the scale correction each...
Zhuo-Qi Feng, Wei-Xing Chen, Zi-Zhao Chen et al.· 0 citations
Work on memory for multimodal agents optimizes what is written, updated and retrieved. Between retrieval and the answer, however, is a stage that multimodal memory evaluations do not isolate: what of the retrieved memory reaches the model, and in what form. We call it delivery, and a controlled decomposition on MemLens...
Yuhang Jiang, Qingwei Liao, Kaize Yin et al.· 0 citations
Dynamic contrast enhanced breast MRI (DCE-MRI) is a powerful clinical tool for breast cancer detection, providing high resolution anatomical detail together with rich temporal contrast information. However, high dimensional 4D inputs, small lesions, and heterogeneous acquisition protocols across clinical sites hinder r...
Benjamin Hamm, Y. Kirchhoff, Maximilian R. Rokuss et al.· 0 citations
TRACE is proposed, a continuous latent framework that keeps stream states separate and couples their updates and achieves the highest contact precision, recall, and F1 among the compared methods.
Few-step autoregressive video diffusion generates a long video by splitting the video into temporal chunks and generating chunk-by-chunk, each through a short sequence of denoising stages. To memorize chunks that are already generated, previous methods reconstruct a clean or less-noisy key--value (KV) cache by addition...
Yikai Wang, Xiao Han, Mengmeng Xu et al.· 0 citations
Depth is a natural way to increase the computational capacity in Transformers, yet the contribution of deeper layers can diminish as depth grows larger. Recent approaches enhance normalization (\text{e.g.}, LayerNorm Scaling) or residual connections (\text{e.g.}, mHC, AttnRes) to enable better information flow and dept...
Keyu Wang, Yangyi Huang, Jiale Kang et al.· 0 citations
World modeling under partial observation requires reasoning about the complete worlds that remain compatible with limited visual evidence. Occluded objects and unseen regions can leave several world states possible; additional views can exclude alternatives and strengthen the conclusions supported by the observations....
Wei Wang, Wen-Qiao Zhang, Yu-Tong Lin et al.· 0 citations
Skeleton sequences can represent both individual actions and multi-entity interactions, encompassing human bodies, hands, objects, and robots. Existing approaches to recognize skeleton-based actions and interactions usually adopt a late fusion strategy, which expects individuals are independent and identically distribu...
Mengyuan Liu, Yuhang Wen, Yi Zhang et al.· 0 citations
Exploring how generative AI could make machine vision more accessible to businesses. The post GenEye in a Box: Making Machine Vision Something You Can Just Ask For appeared first on GPT-Lab.
Jennifer Neville did not want to go into computer science—but that’s exactly where she landed. Neville discusses the starts and stops that led to her professional sweet spot and her work identifying “surprising failures” making it hard for AI to handle complexity. The post What AI gets wrong and what failure teaches us appeared first on Microsoft Research.
Computer scientist, entrepreneur, and philanthropist will collaborate with the MIT Schwarzman College of Computing to advance AI and scientific discovery.