PlanarBench: Evaluating LLM Spatial Reasoning via Planar Graph Drawing

Abstract

Existing LLM graph benchmarks typically ask models to answer graph-theoreticquestions or compute symbolic solutions rather than construct spatial layouts.Within-task difficulty is also primarily stratified by vertex count. However, existingresearch also suggests that task difficulty is more closely related to the number ofconstraints imposed by the edges than to the number of vertices being arranged.We introduce PlanarBench, a benchmark that asks models to produce crossing-free ASCII drawings of planar graphs given only an edge list. Across 91 modelconfigurations and 199 non-isomorphic connected planar graphs with 2–7 vertices,edge count is more strongly associated with mean task score than vertex count(r = −0.85 versus r = −0.47) and remains strongly associated after controllingfor vertex count (rpartial = −0.80). PlanarBench provides a controlled settingfor separating these two difficulty axes. In addition, neither drawing area nortotal response length demonstrated a meaningful correlation with score, which isevidence against a simple output-size explanation. Performance varies widely: thebest model scores 159.5 out of 199, most models below 30B parameters scoreunder 25, and substantial failures remain among frontier systems.