Hybrid Spiking LoRA: Asymmetric Bit-Width Design for Language Model Adaptation With Theoretical Neuromorphic Efficiency Potential—A Case Study on Korean NLU

Abstract

Parameter-efficient fine-tuning adapts pre-trained language models to downstream tasks with reduced training cost, but widely used methods such as Low-Rank Adaptation (LoRA) still rely on conventional multiply-accumulate computation in the adapter path. This paper proposes Hybrid Spiking LoRA, an asymmetric spiking adapter that replaces the LoRA down-projection with a one-bit spiking encoder while retaining a full-precision up-projection. The design introduces sparse event-driven computation into the low-rank adapter while preserving continuous reconstruction capacity for language model adaptation. We evaluate the method on Korean natural language understanding tasks covering topic classification, relation extraction, natural language inference, and extractive question answering. Across timestep, rank, membrane time constant, and bit-width ablations, the encoder-only spiking design consistently outperforms fully spiking alternatives and remains competitive with standard LoRA. Hybrid Spiking LoRA retains 95.9% of standard LoRA performance on KLUE topic classification at rank 16 and 96.8% in a matched five-seed KorQuAD 2.1 comparison, where Hybrid- $T{=}4$ achieves $79.92~\pm ~0.30$ F1. On relation extraction, it underperforms standard LoRA on average but clearly exceeds fully spiking variants, and single-seed bit-width gains over FP32 are interpreted as seed-dependent preliminary observations rather than a general performance claim. Operation-level analysis indicates substantial theoretical adapter-path energy-saving potential from sparse accumulate operations, although current graphics processing unit simulation introduces latency overhead. These results position asymmetric spiking adapters as a feasible neuromorphic-compatible direction for language model adaptation, while motivating future hardware-level validation.