Adversarial attack detection in resource-constrained environments: a stable and sequential federated learning architecture with TinyLlama-1.1B

Abstract

Large language models (LLMs) create significant training bottlenecks on resource-constrained edge devices because of their massive size. Existing federated learning architectures typically require clients to train simultaneously; however, this leads to system crashes due to the high amount of VRAM required by LLMs. While PEFT methods such as LoRA in the literature reduce hardware load, they can cause performance degradation and security vulnerabilities in complex tasks by limiting the model’s parameter updates. This study presents a novel methodological framework that enables a “full fine-tuning” approach—which does not compromise on LLM capabilities—to run stably despite hardware constraints. The distinctive innovation of the proposed architecture is a Mutex-based asynchronous and sequential execution model that defines GPU resources as a “critical section” among clients. The requirement for concurrent training in traditional methods has been overcome by allocating the GPU to a single client at a time and applying aggressive memory deallocation procedures after training. On the server side, full-parameter updates have been combined with low latency using the “Incremental Weight Averaging” technique, which is compatible with asynchronous streaming. In experiments conducted on the TCAB dataset, while traditional parallel architectures failed due to memory constraints, the proposed method successfully completed the “full fine-tuning” process—in which all model parameters are updated—with 100% stability and improved attack detection accuracy to 99.02%. This study presents a novel architecture that addresses hardware constraints not by reducing model capacity, but by restructuring the methodology to enable full fine-tuning training.