MAPLE: Metadata Conditioned LLM Pretraining for Locale-Aware Question Answering
TL;DR
LocalNewsQA is introduced, an 18,700-item English-news benchmark that pairs the same question across two locales and scores whether a model actually switches its answer when the locale changes, and pretraining with metadata in MAPLE produces measurable switching and improves accuracy on questions whose correct answer depends on locale.
Abstract
Large language models can memorize competing locale-specific facts yet fail to select among them when the locale changes, defaulting instead to a single globally dominant answer. We formalize this as localized knowledge disambiguation and introduce LocalNewsQA, an 18,700-item English-news benchmark that pairs the same question across two locales and scores whether a model actually switches its answer when the locale changes. We also introduce MAPLE, a controlled family of decoder-only models pretrained with document-level geographic metadata (source URL, country, and continent) already present in the training corpus, and compare it to metadata-free controls trained on identical data with the same token budget, architecture, and optimization. In controlled experiments at 1B and 3B, with inference-time metadata fixed, pretraining with metadata in MAPLE produces measurable switching and improves accuracy on questions whose correct answer depends on locale. Ablations and external-benchmark evaluations further suggest that locale-conditioned prediction benefits from geographic provenance learned during pretraining and that these benefits strengthen at larger model sizes.
