Deepfake recognition via cascaded neuro-fuzzy learning and parallel convolutional harmonic networks

Abstract

The harmful use of Deepfake technique is a major risk to public safety, causing a crisis in public opinion. Despite numerous attempts to identify deepfake videos, current approaches lack generalization. These approaches often lack robustness, struggle with generalization across different types of deepfakes, and fail when the manipulations are subtle or generated by advanced models. As deepfake generation technology evolves, it consistently outpaces traditional detection techniques, necessitating more sophisticated, adaptive, and learning-based detection systems. Thus, this article proposes an effective Cascaded Neuro Fuzzy Parallel Convolutional Forward Harmonic Network (Cascaded NFPCFHNet) for deepfake recognition. At first, frame extraction is conducted. Then, Deep Neural Network (DNN) performs face detection on the extracted frames and Facial Action Unit (AU) detection is performed by AUNet. Moreover, Feature extraction is performed for each AU, capturing essential features for improving the detection accuracy. Finally, Deepfake Recognition is done using Cascaded NFPCFHNet. The Cascaded NFPCFHNet is a novel architecture formed by combining a Cascaded Neuro Fuzzy Network (CNFN) and a Parallel Convolutional Neural Network (PCNN) with Harmonic analysis. Moreover, Cascaded NFPCFHNet attained accuracy, True Positive Rate (TPR), as well as True Negative Rate (TNR) of 91.892%, 90.967%, and 92.583% respectively.