Jul 2026· Digital Threats: Research and Practice· Vol 7, pp. 1-25· 0 citations· 31 references
TL;DR
Evaluation across semantic similarity, distributional consistency, discriminability tests, and structural fidelity shows that LLM-generated logs—particularly using GPT-4o-mini, closely approximate real MHMS activity, enabling safe and scalable experimentation for intrusion-detection research.
Abstract
Multi-Host Multi-Stage (MHMS) cyberattacks are increasingly sophisticated, spanning multiple systems through coordinated stages and evading traditional single-host detection approaches. Such attacks leave correlated traces across diverse log sources, including system, audit, and authentication logs. However, research on MHMS intrusion detection is constrained by the scarcity of realistic multi-host log datasets, as real-world data are difficult to share due to sensitivity and the operational complexity of reproducing attacks in controlled environments. To address this challenge, we present AttackLogGen, a framework for benchmarking Large Language Models (LLMs) in generating realistic MHMS logs. Malicious activities are modeled using the MITRE ATT&CK knowledge base, while benign background behavior is synthesized from generic event patterns across interconnected hosts. Leveraging LLM contextual reasoning, AttackLogGen produces logs tat preserve causal dependencies, temporal ordering, and cross-host interactions characteristic of complex attacks. The framework integrates coordinated modules to generate time-aligned and semantically coherent system behavior. For validation, we built a small-scale MHMS testbed executing end-to-end multi-stage attacks to collect ground-truth logs, complemented by single-system data from the LogHub dataset. Evaluation across semantic similarity, distributional consistency, discriminability tests, and structural fidelity shows that LLM-generated logs—particularly using GPT-4o-mini, closely approximate real MHMS activity, enabling safe and scalable experimentation for intrusion-detection research.
LogSanitizer is proposed, a family of input sanitization defenses operating at two levels: a pre-prompt log-transformation pipeline that disrupts trigger patterns in the structured log representation, and a post-tokenizer perturbation strategy that corrupts trigger-bearing token configurations before they reach the model.
Leszek Wronski, Bogdan Ksiezopolski· International Conference on...· 0 citations
While LLM attacks span prompt optimization, multi-turn context manipulation, retrieval poisoning, and model backdoors, white-box defenses are typically evaluated on isolated attack families. Consequently, whether heterogeneous attacks leave internal representation shifts that generalize to unseen threat mechanisms remains unknown. We present MechAudit-40, a systematic evaluation of 40 attack mechanisms across five open-weight model architectures. Threat-specific success criteria, 100,000 matched clean-attack representation pairs, predefined categories, and grouped holdouts isolate genuine attack-induced displacement from target scale, corpus bias, and data-leakage shortcuts. Across this testbed, attacks induce structured multi-depth trajectories rather than isolated layer spikes. While raw peaks are non-portable across architectures, target-calibrated profiles preserve transferable geometric signatures: under complete mechanism holdout, hidden states alone recover the threat category of unseen attacks with 82.5% accuracy. Guided by this finding, we design MechAudit, a runtime auditor that operates under strict zero-oracle constraints without requiring clean baseline traces or attack metadata. MechAudit detects 81.1% of held-out attack executions at a 0.70% false-positive rate and maintains 78.1% recall when an entire functional category is withheld. In matched comparisons, MechAudit is the only detector that avoids mechanism-level coverage collapse, maintaining over 50% recall across all 40 mechanisms. Internal representations thus support cross-mechanism attack-exposure auditing against calibrated benign references, but decouple from downstream task compromise and parameter integrity.
Zhen Guo, Shang-Hao Shi, Shamim Yazdani et al.· 0 citations
Experimental results show that LLMs, when guided by rubric-based prompts and supplemented with ATT&CK domain knowledge, achieve robust performance across detection, localization, and TTP mapping tasks.
Joon-Young Gwak, Aubrey Strier, Zhaohan Xi et al.· 0 citations
This paper presents a framework for evaluating prompt injection attacks against LLM-based log interpretation using log traces generated during real cyber attacks, and creates adversarial examples through generic injection generation, refinement, and attack-specific optimization.
Max Landauer, F. Skopik, Markus Wurzenberger et al.· arXiv.org· 0 citations
This work proposes Continuous Agents for Injection Threats via Lifelong Yielding Nexus (CAITLYN), an agent-agnostic defense middleware that matches the detection performance of state-of-the-art defenses at lower token overhead than LLM-as-a-judge baselines.
Zi Liang, XiaoYu Xu, Yanyun Wang et al.· 0 citations
We present a new attack that reconstructs the text generated by locally hosted LLMs by observing CPU cache activity during detokenization. Unlike prior attacks that rely on deployment-specific assumptions, such as shared data memory, CPU offloading, or Mixture-of-Experts architectures, our approach targets the detokenizer, a component used in default LLM inference pipelines. To obtain clean signals, we use Flush+Reload on shared tokenizer code to detect when decoding occurs, which lets us perform Prime+Probe at the right moment and isolate token-dependent cache activity. We then apply a clustering-and-language-model pipeline to recover text from noisy cache observations. We evaluate the attack across multiple datasets, hardware platforms, inference frameworks, and model families, and show that it can recover semantically accurate outputs from real-world local LLM deployments, including agentic systems. This vulnerability is particularly significant because the most widely used tokenizer implementations are susceptible to the attack and are embedded in many popular local LLM products and agent frameworks, including systems such as OpenClaw (which we demonstrate), substantially broadening the practical attack surface.
Roy Weiss, B. Konstantinov, Eitam Sheetrit et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
