Generative debunking of climate misinformation

Abstract

Misinformation is a persistent roadblock to progress on climate change, and the ubiquitous, rapid spread of misinformation means that manual solutions are inadequate to meet the scale of the problem. While many studies have developed methods for automated detection of misinformation, the generation of effective debunkings—particularly in line with the findings of psychological research (e.g., providing relevant facts, explaining fallacies)—remains underexplored. Our research synthesizes generative large language models (LLMs) with established climate misinformation taxonomies and fallacy detection methods, creating a novel approach described as “generative debunking”. We introduce three LLM-based models: a “Vanilla” model using a single prompt, a “Structured” model decomposing the debunking task into subtasks, and a “Structured+RAG” model that extends the structured approach, drawing upon authoritative external sources. Results indicate that while all models performed similarly overall, the Structured+RAG model showed improvements in identifying and explaining misinformation fallacies. The Structured model surpassed the Vanilla model in generating on-point relevant facts. Our contributions include addressing evaluation challenges with expert-annotated data, ensuring diverse misinformation coverage, and bridging psychological best practices for debunking with computational methods. These findings highlight the potential of AI-driven, psychologically-informed debunking to combat misinformation, particularly through structured and curated approaches.