Graphical User Interface (GUI) grounding is a fundamental perception task for multimodal agents, enabling them to interpret natural language instructions and interact with digital interfaces. Existing methods face a fundamental trade-off between accuracy and efficiency: direct full-image inference often fails to captur...
Li-Yang Fan, Xin-Ping Bi, Yi-Tai Li et al.· 0 citations
ReFigBench, a benchmark and evaluation framework built on 1,000 real overview figures retrieved from arXiv papers with full provenance, is introduced, exposing the tension between fidelity and editability as the central challenge for practical multimodal document agents.
Li-Yang Fan, Chi Wei, Yi-Tai Li et al.· 0 citations
This work presents the first zero-shot cross-lingual framework for handshape recognition, transferring from ASL to Catalan Sign Language (LSC), and leverages the decomposition of handshapes into five phonological features shared across both languages, to decode LSC handshapes from predicted features via a composite pho...
Marcel Granero-Moya, Carolina del Corral Farrarós, G. Haro et al.· 0 citations
Large language models (LLMs) are increasingly used for structured information extraction from documents, yet their behavior under realistic OCR noise remains poorly understood. We present a systematic benchmark of open-source instruction-tuned LLMs for key-value pair (KVP) extraction under both clean-text and noisy OCR...
Zahra Anvari, Vassilis Athitsos· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Interpretability is crucial for utilizing machine learning models as clinical decision support tools for medical diagnostics. However, most state-of-the-art image classifiers based on neural networks are not interpretable. As a result, clinicians often resort to known biomarkers to guide diagnosis, although biomarker-b...
Laurin Lux, Alexander H. Berger, Maria Romeo Tricas et al.· 0 citations
Model compression is critical for deploying networks on resource-constrained edge devices. While pruning-based methods can significantly reduce model size, they often suffer from abrupt performance collapse beyond a sparsity thresh-old, making it difficult to identify the feasible compression limit of the model. To add...
The Internal Waves Service screens the Sentinel-1 Wave-mode archive for internal solitary waves, routing detections to experts whose adjudication time is the resource the effort exists to conserve. Because attention is the cost of error, precision leads. Its classifier was trained and reported at a one-to-one class bal...
João Pinelo, João Gonçalves, Arun Shukla et al.· arXiv.org· 0 citations
Traditional fake news detection methods are falling behind as multimodal misinformation grows more advanced, seamlessly blending deceptive text, manipulated visuals, and factually incorrect claims. Most prior work focuses on text-image fusion or applies external knowledge only as a post-processing step, limiting their...
This paper studies instrument classification of solo sheet music. Whereas previous work has focused on instrument recognition in audio data, we instead approach the instrument classification problem using raw sheet music images. Our approach first converts the sheet music image into a sequence of musical "words" based...
In autonomous driving, perception models often struggle to generalize to new environments due to domain shifts. While unsupervised model adaptation offers a feasible solution without labor-intensive manual labeling, existing methods that rely solely on the ego-vehicle's data often lead to inferior pseudo-labeling perfo...
Ya-Nan Ma, Yi-Hang Tao, Zheng-Ru Fang et al.· 0 citations
This work introduces Contrastive Noise Alignment (CNA), a training-time method that creates dynamic, contrastive couplings by optimizing the noise representations directly, and shows theoretically that this equilibrium asymptotically preserves Gaussian structures, maintaining tractability during inference.
MCLC-NET is proposed, a multimodal continual learning framework for leaf counting that learns tasks sequentially using a memory-based strategy with a memory buffer to retain important samples from previous tasks.
Ruchi Bhatt, Pratibha Kumari, Shreya Bansal et al.· 0 citations
Exploring how generative AI could make machine vision more accessible to businesses. The post GenEye in a Box: Making Machine Vision Something You Can Just Ask For appeared first on GPT-Lab.
Jennifer Neville did not want to go into computer science—but that’s exactly where she landed. Neville discusses the starts and stops that led to her professional sweet spot and her work identifying “surprising failures” making it hard for AI to handle complexity. The post What AI gets wrong and what failure teaches us appeared first on Microsoft Research.
Computer scientist, entrepreneur, and philanthropist will collaborate with the MIT Schwarzman College of Computing to advance AI and scientific discovery.
