Skip to content
Preprint

Does It Render Everywhere? A Study of Cross-Environment Compatibility in MLLM-Generated Webpages

Aug 2026 · 0 citations · 38 references
Computer Science

TL;DR

It is revealed that 68% of generated webpages exhibit at least one compatibility issue, underscoring the pervasive reliability concerns surrounding MLLM-generated front-end artifacts and developing XCompat, a lightweight offline compatibility issue detector that combines visual screenshots and the structural DOM tree for analysis.

Abstract

Multimodal Large Language Models (MLLMs) have been increasingly adopted to automate webpage generation from visual designs (e.g., screenshots). However, existing evaluations are limited to visual fidelity assessment under a fixed browser-device configuration. Such a setting overlooks the cross-environment rendering compatibility for real-world deployments. To address this gap, we present the first systematic empirical study of cross-environment compatibility in AI-generated webpages. Specifically, we construct WebCompat, a dataset of 2,032 annotated instances, comprising webpages generated by 8 representative AI tools, each rendered across 9 browser-and-device combinations. We analyze the prevalence of compatibility issues, their user-perceptible symptoms, and underlying code-level root causes. Our findings reveal that 68% of generated webpages exhibit at least one compatibility issue, underscoring the pervasive reliability concerns surrounding MLLM-generated front-end artifacts. The most prevalent symptoms are failures that disrupt the entire page layout (88.3%): pages shrink directly to fit the target screen with too small fonts, or exhibit scale mismatches that produce cut-off content. Failures localized to individual elements, such as image distortion or missing components, are comparatively less common (13.4%). Furthermore, although most MLLMs incorporate responsive design patterns into the generation, they fail to properly implement these codes. Guided by the findings, we develop XCompat, a lightweight offline compatibility issue detector that combines visual screenshots and the structural DOM tree for analysis. It achieves an F1 score of 0.903 on the WebCompat-test, outperforming the existing compatibility checking tools and LLM baselines. All datasets and tools are released to support future research on rendering reliability in MLLM-based front-end code generation.

View source

Similar papers

ProgramDistill: From Interactive Web Apps to Verifiable Reference-Guided SWE Tasks

ProgramDistill is introduced, a benchmark evaluating coding agents on features discovered through interaction with fully functional reference applications, and provides a scalable benchmark with controlled difficulty for evaluating and diagnosing coding agents, and a natural basis for future curriculum-based training.

Jeonghye Kim, Minseon Kim, Young Jin Kim et al. · 0 citations
Sep 2026

WebCraftBench: Evaluating Web Application Generation from a Software Testing Perspective

WebCraftBench, an interactive benchmark for evaluating web application generation from a software testing perspective, is proposed and experiments show that coverage guidance improves exploration coverage and the model rankings remain stable when the judge model is replaced.

Chen-Xu Liu, Zi-Lu Zou, Pei-Zhong Gao et al. · 1 citation
#computer vision Preprint Oct 2026

RefCraft: Benchmarking Visual-Reference-Conditioned Agentic Generation through Web Development

Beyond basic tool use and coding skills, effectively incorporating visual references into agentic generation is equally essential for applying AI agents to real-world productivity tasks. We study this crucial capability through webpage development, a practical and readily evaluable setting in which existing benchmarks...

Yu-Chi Wang, Rongyao Fang, Wei-Hao Yu et al. · 0 citations
Book Open access Sep 2026

LADar: an LLM-based Tool to Detect Web Accessibility Violations

LADar is presented, an accessibility auditor that leverages Agentic AI to bridge the gap between static analysis and real browser interaction and successfully detects critical violations that traditional automated tools consistently overlook.

Forough Majidi, Marina Ribaudo, Maurizio Leotta · 0 citations
Open access Aug 2026

Evaluating the Efficiency of Open-Source AI Chat Models in Local Deployments

This study evaluates the performance and operational practicality of locally deployed large language models (LLMs) as organizations seek cost-efficient, privacy-preserving alternatives to cloud AI. The objective is to generate workflow-aware evidence that supports informed model selection for automation use cases. Four...

Ida Bagus Kerthyayana Manuaba, Juwono · 0 citations
#artificial intelligence Preprint Oct 2026

TaoD2C-Bench: Benchmarking MLLMs for Industrial UI Code Generation Beyond Visual Fidelity

Evaluating eight MLLMs reveals substantial gaps in generating UI code that satisfies implementation requirements, alongside distinct performance profiles in inference and realization, and shows that MLLMs'visual reconstruction ability does not necessarily imply an ability to generate code that meets these requirements.

Cheng-Wei Shi, Yun-Nong Chen, Ting-Ting Zhou et al. · 1 citation

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.