Can Urban Planners Rely on Large Language Models? An Exploratory Evaluation of Four Frontier Models on Two Contested Greek Masterplans

Abstract

Urban planners are increasingly using Large Language Models (LLMs) to draft project appraisals, raising the question of whether such systems can act as independent evaluators. Here we test this question on two contested urban regeneration projects in Greece: Hellinikon in Athens and TIF-HELEXPO in Thessaloniki. A validation rubric was compiled for each case from court rulings, professional body statements, peer-reviewed scholarship and civic-movement documentation. Four frontier LLMs (ChatGPT 5.4, Claude Opus 4.6, Gemini 3.0 Pro, Grok 4) evaluated each case under four prompts: minimal, comprehensive-impartial, comprehensive-sceptical and comprehensive-advocating. Three of the four models covered 77–79% of rubric positions under comprehensive prompts. Coverage depended more on the neutral factsheet than on model choice. When the framing flipped from sceptical to advocating, sceptical-position coverage fell by 9 percentage points and advocating-position coverage rose by 21 percentage points. Positions articulated by local professional and academic bodies were consistently missed. Multi-model ensembles reached 94% and 84% sceptical-position coverage on Hellinikon and TIF-HELEXPO. This evidence is inconclusive but suggests that LLM outputs still require expert verification of locally documented positions, though multi-prompt and multi-model elicitation with a neutral factsheet can support drafting planning reviews.