Skip to content
Open access

An advanced framework for malware detection using adversarial learning

Aug 2026 · International Journal of Information Security · Vol 25 · 0 citations · 47 references

TL;DR

A systematic framework to enhance adversarial robustness is proposed, validated on the Malimg dataset and supersedes previous approaches by 13.15% in terms of the evasion rate and 37.34% in terms of retraining success.

Abstract

Malware is a type of software designed to intentionally disrupt or gain unauthorized access to a computer system. As technology progresses, new variants of malware continue to emerge. In the domain of image-based Windows PE malware family classification, modern deep learning models are highly susceptible to adversarial attacks, leading to biased learning between clean and adversarial samples, suboptimal retraining, and reduced capability to capture family-specific discriminative features. Existing attack generation methods rely heavily on cybersecurity expertise, making them complex and difficult to scale. Additionally, retraining often overfits adversarial samples or loses sensitivity to clean data and weakening defenses. To address these issues, this paper proposes a systematic framework to enhance adversarial robustness, validated on the Malimg dataset. Firstly, malware binaries are converted into grayscale images, enabling image-based deep learning analysis. Secondly, a multi-stage oversampling strategy is applied before and after generating adversarial samples to reduce bias and improve data diversity. Thirdly, a novel feature mask is introduced, combining Convolutional Neural Network (CNN) derived gradients, structural masks from PE parsers, and unused null byte positions. This mask selectively alters non-critical regions, preserving malware functionality while reducing adversarial noise. Finally, to address long-range dependencies, adversarial samples are tested against Vision Transformers (ViTs) in a black-box environment, ensuring transferability and improving detection across various models. Detailed analysis of the results reveals the achievement of the 81% evasion rate with the average L0 norm 0.284%. However, retraining the model increases the accuracy from 16 to 97%. These results are mirrored on the MalEVis dataset, where a 94.40% F1-score is maintained and accuracy is restored to 90.79% after byte-level perturbations, validating the framework’s scalability. Compared to existing methods, the proposed model supersedes previous approaches by 13.15% in terms of the evasion rate and 37.34% in terms of retraining success.

Read PDF

Similar papers

Aug 2026

Enhancing robustness of deep learning-based malware detection against adversarial attacks

This paper introduces DefendMal, a novel framework that synergistically combines Denoise Autoencoder with Sequence Squeezing, a Context-aware Adversarial Generator (CAG-AdvGAN), Projected Gradient Descent (PGD) adversarial training, and a Positive–Negative Detector with Variational Autoencoder (PNDetector-VAE) to enhance robustness against evolving adversarial threats.

Dennis Benedict Crasta, Vikash Kumar · 0 citations
Open access Aug 2026

Analyzing Malware Behavior Using Generative Neural Networks

Traditional detection techniques are struggling with ever-evolving malware threats like zero-day attacks, polymorphic malware, and adversarial samples. Current detection systems (signature-based, heuristic-based, conventional machine learning) fail to generalize to unseen/obfuscated malware variants. In an attempt to overcome these constraints, this paper investigates the possibilities of employing Generative Neural Networks (GNNs), in the form of Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs) for the purpose of malware be haviour analysis and detection. We aim to create a novel framework for detecting malware samples that provides some of the best performance in terms of accuracy, precision, and recall while remaining robust to new or unseen malware. This work aims to firstly implement a generative learning-based approach and to measure its adversarial robustness in comparison with the four existing detection techniques. Experimental results show the accuracy, precision, recall of the proposed model is found to be 96.5%, 95.9%, 94.6% with the false positive rate of the model which can be negligible and it is 3.2% which outperforms the traditional machine learning and deep learning models. Our results demonstrate that GNN-based malware detection not only addresses the limitations of conventional approaches in terms of scalability but also provides a more robust and adaptable framework that could be integrated into future real-time threat intelligence and automated defense systems.

Wurood A. Jbara, N. A. Hussein · 0 citations
Review Open access Aug 2026

Generative Adversarial Networks for Anomaly and Malware Detection

A comprehensive survey of how GAN-based methods are utilized for identifying unusual and harmful activities in cyber settings and addresses ongoing challenges and potential future avenues for employing GANs to counteract emerging cybersecurity threats.

A. Thakore, Neha Gupta, Akash Saxena et al. · 0 citations
Review Aug 2026

A Comprehensive Review on Adversarial Attacks and Detection Techniques in Deep Learning Models for Image Analysis

The research methodology involved a systematic literature review using the Scopus database, adhering to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, and focusing on recent advancements in attack and defence techniques.

Reeti Jaswal, Vikas Khullar, Surya Narayan Panda · 0 citations
Book Open access Jul 2026

Co-evolving Adversaries for Online Robust Malware Detection

Adversarial robustness remains a significant challenge in machine-learning malware detection. We propose a co-evolutionary adversarial training framework that integrates gradient-based adversarial attacks with evolutionary controllers to improve robustness. A hybrid Convolutional Neural Network (CNN-MLP) processes spatial and vector-based features, while evolved controllers adaptively guide Projected Gradient Descent (PGD) using gradient statistics, optimizing adversarial effectiveness while respecting feature constraints. This online, co-evolutionary process exposes the classifier to increasingly adaptive attacks. Experimental results demonstrate that the proposed approach enhances adversarial robustness relative to standard training while maintaining clean accuracy, exemplifying a co-evolutionary arms-race framework for realistic, feature-constrained malware detection.

Sabre Didi, Geoff S. Nitschke · 0 citations
Review Open access Aug 2026

A Survey on Deep Learning Approaches for Malware Detection and Classification

Malware is a serious threat in the cybersecurity area because of its dynamic nature, the variety of malware families, stealth, propagation and the capability of evading traditional security products. Therefore, proper malware detection and classification are crucial for detecting malicious software and for securing computer systems from unauthorized access and data stealing, and for disrupting systems. This study covers all the bases when it comes to deep learning approaches for malware detection and classification. It covers the principles, different forms of malware, how to detect deep learning malware, how to represent data, obtaining features, and applications. The traditional detection methods are described with their drawbacks, namely based on signature, behavioral and heuristic methods. The report also delves into the methodologies used by deep learning to classify malware, namely CNNs and Bidirectional Long Short-Term Memory (BiLSTM) networks. BiLSTM models excel at learning sequential features from code-or behavior-related data, whereas CNN-based representation learning approaches excel at learning spatial features from malware representations. Moreover, the various detection techniques (static, dynamic and hybrid) are discussed so that their role in malware analysis can be understood. The survey identifies the current challenges and gaps in research and emphasizes the need for strong, scalable and adaptive deep-learning models to combat new malware threats and enhance cybersecurity protection.

Shivani Jain · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.