AI-Driven Big Data Security Framework for Intelligent Threat Detection and Privacy Preservation

Abstract

With the explosive growth of big data environments, cybersecurity and privacy concerns have become more complex. The complexity of connecting distributed storage, cloud platforms, enterprise applications, and various data types creates a vast attack surface that traditional security has difficulty protecting against from advanced attacks. This paper proposes an innovative Dynamic Entropy-Guided Security Genome Framework (DESGF) for securing and protecting privacy in the large-scale big data ecosystem with intelligent threat detection. The proposed framework will present a novel method for Security Genome based representation by entropy profiling, interaction density analysis, sensitivity assessment and security genome vector generation to take into account intrinsic security properties. Thereafter, a Genome Intelligence Layer maps the genome for abnormal security behavior, computes genome mutation intensity, evaluates genome threat emergence and assigns a genome risk score without relying on predetermined attack signatures. Multiple machine learning classifiers are used to classify the security conditions and produce the threat aware outputs to support adaptive security decision making. This framework also significantly improves the preservation of privacy by representing sensitive information in the data as security-agnostic genome representations, which minimize direct exposure of raw data in analysis. The standard security performance metrics such as accuracy, precision, recall, F1-score, detection rate and false alarm rate are used as the measures in the experimental evaluation. Based on the results obtained, design time works well to enhance the capability of threat identification while at the same time preserving high levels of security regarding privacy. The proposed framework provides an adequate and scalable way to protect modern data-centric environments from cyber threats evolving over time. The suggested Security Genome Framework attained an overall accuracy of 98.12%.