Hybrid Approach to Deepfake Defence: Multimodal Detection and Perturbation-Based Prevention

Abstract

The rapid advancement of deep learning-based generative models has enabled the creation of highly realistic synthetic media, commonly referred to as deepfakes. While these technologies have legitimate applications, their misuse poses serious threats to digital trust, identity protection, and information authenticity. Existing countermeasures predominantly focus on post-hoc detection of manipulated media, which is often insufficient due to generalization issues, adversarial robustness concerns, and delayed response after content dissemination. This paper investigates a combined deepfake detection and prevention framework, where detection is approached from a multimodal perspective, encompassing image, video, and audio-based cues to improve robustness across different forms of manipulated media, achieved through an ensemble of neural networks trained for spatial as well as frequency domain artifact detection. The overall ensemble system achieved 99.2% classification accuracy and an Area Under Curve (AUC) of 0.990. In contrast, the prevention component is scoped specifically to image-based deepfakes, focusing on exploratory techniques such as controlled perturbations and noise-based interventions to study their impact on manipulation and generation processes. A Discrete Cosine Transform (DCT)-based method for prevention resulted in an identity disruption rate of 98% in various facial recognition systems and the visual distortion was imperceptible with Structural Similarity Index Measure (SSIM) values bigger than 0.999.