CityLLM: A framework for natural-language querying of semantic 3D city models

Query2Property: Semantic retrieval of IFC properties for natural language BIM queries

Abstract. IFC models store detailed building information, but their complex schema and deeply nested property sets make querying difficult for non-expert users and challenging for large language models (LLMs) to handle directly. Current LLM-based approaches are inefficient because prompts often include entire IFC schemas, many properties of which are irrelevant to the user’s query, leading to higher inference costs and potential errors. This paper presents Query2Property, a semantic retrieval system that maps natural language queries to the most relevant IFC properties. By embedding both property descriptions and user queries in a shared vector space, the system retrieves contextually relevant properties for dynamic and concise prompt construction in LLM-driven workflows. Evaluation on 55 representative BIM queries achieves a top-1 accuracy of 87.3% and top-3 accuracy of 100%, demonstrating effective alignment with user intent. Query2Property simplifies LLM-based workflows over BIM data, supporting semantic search and natural language exploration of complex building information.

AuthentiCity: A Multi-Source Provenance-Aware Knowledge Graph and Benchmark for 3D City Models

Urban digital twins increasingly combine authoritative, crowd-sourced, machine-learned, and reconstructed data with differing reliability, coverage, and semantics. Yet few urban datasets provide a unified representation supporting multi-source integration, provenance tracking, spatial reasoning, and machine learning. We present AuthentiCity, a multi-source, provenance-aware 3D city knowledge graph spanning five cities across three continents (Hamburg, Helsinki, Zurich, New York, and Tokyo) and comprising 180 GiB, 180M nodes, 220M edges, 1.2B properties, and 3.6M buildings. The labeled property graphs integrate authoritative CityGML and OpenStreetMap data for all cities, adding roof-material predictions and reconstructed LoD3 geometry for Hamburg, under a provenance model in which derived information never replaces authoritative data. Confidence-weighted edges resolve cross-source correspondences, constructing canonical urban entities while preserving traceable links to contributing evidence. AuthentiCity is primarily a data contribution. We introduce two benchmark families that demonstrate the tasks enabled by the representation. The first evaluates natural-language-to-query translation beyond conventional text-to-SQL and text-to-Cypher benchmarks, including 3D spatial reasoning, provenance-aware filtering, cross-source agreement and disagreement, coverage-aware aggregation, and infeasible-query detection. The second evaluates graph representation learning through multi-source attribute prediction, node classification, and cross-source matching prediction, enabling comparison of provenance-agnostic and provenance-aware embeddings. Even a strong commercial LLM reaches only 54-69 % execution accuracy and a 7B open-weight model 6-19 %, while the open-weight model never abstains on unanswerable questions.

Semantic enrichment and automatic completion of regional geospatial metadata using NLP: a comparative study of spaCy and transformers

Toward Tool-Augmented Semantic Computing: Integrating Large Language Models with Data Grounding and Optimization for Real-World Problem Solving

Large Language Models (LLMs) have significantly advanced semantic computing by enabling systems to interpret and generate natural language with high flexibility. However, they remain limited in solving real-world problems that require precise computation, reliable data access, and structured optimization. Tasks such as route planning, geospatial reasoning, and combinatorial optimization highlight these shortcomings, particularly in terms of factual grounding and solution feasibility. This paper proposes a framework for tool-augmented semantic computing, in which LLMs are combined with external data sources and specialized algorithmic components. The framework consists of a semantic layer for interpreting user intent, a tool layer for data access and computation, and an orchestration layer coordinating their interaction. Using travel planning as a representative use case, we demonstrate how tasks such as point-of-interest selection, coordinate retrieval, and route optimization can be decomposed into modular processing steps. In particular, route sequencing is formulated as a variant of the Traveling Salesman Problem, requiring dedicated optimization beyond the capabilities of LLMs. The results show that integrating semantic reasoning with data grounding and algorithmic optimization enables reliable solutions to complex real-world problems, highlighting the potential of hybrid AI systems for future semantic computing applications.

Multimodal Embeddings for 3D Similarity Search in Semantic Web-of-Things Digital-Twin Platforms

Semantic Web of Things (SWoT) platforms model physical infrastructure as knowledge graphs typed against domain ontologies, enabling expressive structural and logical queries. However, they lack native mechanisms to express similarity beyond strict ontological equivalence, which represents a critical gap for 3D digital twins in domains such as telecom infrastructure and industrial IoT, where queries must combine ontological constraints with multimodal similarity search over heterogeneous, temporally-evolving scene data. We propose a framework that extends SWoT platforms with a multimodal embedding layer: ontology-typed entities comprising 3D point clouds, temporal attributes, and semantic labels are encoded into latent vector representations stored alongside the knowledge graph, enabling hybrid ontology-vector queries that combine graph-based filtering with similarity search. Implemented on Orange Research's Thing'in platform with the Clock-G temporal graph database, a feasibility evaluation on S3DIS demonstrates that graph filtering effectively restricts the search pool under temporal and relational constraints, and that general-purpose pretrained encoders produce representations sufficient for similarity retrieval and as a preliminary encoding step for downstream predictive tasks.

Inter-3D VQA: A Roadside Multimodal Benchmark for 3D Spatiotemporally Grounded Visual Question Answering

Recent advances in visual question answering (VQA) and multimodal large language models (MLLMs) have enabled natural-language reasoning over traffic scenes. However, existing benchmarks are largely built from ego-vehicle views or 2D roadside videos, limiting their ability to evaluate 3D-grounded reasoning over real-world distances, trajectories, infrastructure topology, and safety-critical interactions. We introduce Inter-3D VQA, a large-scale roadside multimodal benchmark for 3D spatiotemporally grounded VQA at intersections. Built from synchronized point clouds and multi-view images, Inter-3D VQA contains 407K QA pairs covering lane-level positions, object relationships, motion patterns, and near-miss-oriented interaction reasoning. We further propose Inter-Geo, an MLLM baseline that integrates object- and scene-level aligned LiDAR representations, and Inter-Metrics, a unified evaluation framework for textual consistency, numerical accuracy, and semantic correctness. Experiments show that Inter-Geo outperforms image-based VLMs, especially on grounded spatial and temporal reasoning tasks. Our benchmark and codes are available at https://github.com/ASU-Suo-Lab/Inter-3D-VQA .