AEGIS: Real-Time Latent-Space Backdoor Detection for Dependable and Secure Small Language Model Inference

Abstract

Backdoor attacks pose a serious threat to small language models (SLMs) because compromised models can behave normally on benign inputs while producing attacker-specified outputs when a hidden trigger is activated. Existing defenses commonly require model retraining, operate only before deployment, rely on input-level signals, or incur excessive inference overhead. This paper presents AEGIS (Activation Evaluation and Guardrail for Inference Security), a non-invasive runtime framework for detecting backdoor-induced anomalies in transformer representations. AEGIS dynamically monitors equidistant internal layers, mean-pools and concatenates their hidden states, and compresses the resulting high-dimensional representation into a 64-dimensional latent space. A hybrid compression mechanism uses zero-shot principal component analysis for predominantly linear text representations and a lightweight autoencoder for nonlinear or multimodal representations. Detection is then performed entirely on the GPU using cosine distance from a centroid calibrated on clean data, without modifying or retraining the protected model. We evaluate AEGIS across Mistral-7B, Qwen2.5-7B, a 4-bit QLoRA-backdoored Qwen2.5-1.5B model, and a BadNets-backdoored Vision Transformer, covering simulated latent and PEFT-style attacks, genuine fine-tuned backdoors, FP16 inference, and 4-bit NF4 quantization. Across the main optimized shield-overhead experiments, AEGIS achieves AUROC values between 0.95 and 1.00, true-positive rates between 86% and 100%, and prefill-stage detection latency between 0.47 and 2.40 ms. In the dedicated T4 deployment stress tests, which measure the full guardrail overhead under edge and real QLoRA settings, latency ranges from 5.20 to 7.77 ms with combined model-plus-guardrail memory between 2.59 and 4.02 GB. AEGIS obtains AUROC = 1.00 and 100% true-positive rate against the genuinely fine-tuned QLoRA backdoor, while achieving AUROC = 0.997 against the fine-tuned visual BadNets attack. Under 4-bit quantization, it retains an AUROC = 0.95 with a 4.02 GB memory footprint. These results show that a compact GPU-native implementation of activation-space monitoring can provide effective, retraining-free backdoor detection for real-time transformer inference.