Fake News Detection Using Machine Learning and LLM Embeddings: A Comparative Study of TF-IDF and BERT Representations on the Welfake Dataset

Abstract

The rapid growth of social media and online news platforms has significantly increased the spread of misinformation and fake news. Fake news can influence public opinion, create social unrest, and negatively impact decision-making processes. Therefore, developing intelligent systems capable of automatically identifying fake news has become an important research area in Artificial Intelligence and Natural Language Processing (NLP). The proposed work presents a Fake News Detection System that combines traditional machine learning techniques with Large Language Model (LLM) embeddings to improve classification performance. The proposed system utilizes textual news data from benchmark datasets such as WELFake and LIAR. Initially, the news articles undergo pre-processing steps including text cleaning, normalization, and removal of irrelevant characters. Two different feature extraction approaches are employed: Term Frequency-Inverse Document Frequency (TF-IDF) and contextual embeddings generated using the pre-trained BERT (Bidirectional Encoder Representations from Transformers) model. The extracted features are used to train and evaluate multiple Machine learning classifiers, including Logistic Regression, Support Vector Machine (SVM), Multi-Layer Perceptron (MLP), and XGBoost. Performance evaluation is carried out using standard metrics such as Accuracy, Precision, Recall, F1-Score, and Area Under the ROC Curve (AUC-ROC). Comparative analysis is performed to assess the effectiveness of traditional TF-IDF features versus BERT-based semantic embeddings. Experimental results demonstrate that BERT embeddings capture contextual and semantic information more effectively than conventional text representations, leading to improved fake news classification accuracy. The system also provides visualization tools such as confusion matrices, ROC curves, and performance comparison charts for comprehensive evaluation. The proposed framework highlights the potential of integrating transformer-based language models with classical machine learning algorithms to build robust and scalable fake news detection systems. Our work contributes to combating misinformation by providing an efficient and automated approach for identifying deceptive news content in digital media environments.