Enhancing Fake News Detection in Low-Resource Linguistic Contexts using Translation-based NER and Lightweight NLI
TL;DR
A robust preprocessing pipeline incorporating Google Translation, spaCy-based NER with hyphenated-word normalization, and a future-event-aware fallback logic is proposed, achieving a 40 percent reduction in inference latency and closing the linguistic generalization gap left by previous works.
Abstract
While existing ensemble-based fake news detection models achieve high accuracy on benchmark datasets, they suffer from critical linguistic and preprocessing bottlenecks. The reliance on case-sensitive regex for entity extraction causes failure on lowercased or vernacular (Hinglish) inputs. Furthermore, hardcoded entity+year fallback mechanisms fail for hyphenated scientific missions (e.g., Aditya-L1) and future-dated events (e.g., 2025 summits) due to case mismatches and missing temporal markers in Wikipedia snippets. This paper proposes a robust preprocessing pipeline incorporating Google Translation, spaCy-based NER with hyphenated-word normalization, and a future-event-aware fallback logic. By replacing the heavy BART-large-MNLI with a lightweight DeBERTa-v3-base cross-encoder, we achieve a 40 percent reduction in inference latency. Experimental results demonstrate that the proposed system retains 99.68 percent accuracy on the ISOT benchmark while achieving 100 percent classification on a diverse Hinglish/low-resource test suite—closing the linguistic generalization gap left by previous works.