Log anomaly detection requires models that capture long-range event dependencies without the quadratic sequence-length cost of self-attention. We present LogMamba, a reconstruction-based model that combines a bidirectional selective state-space branch with a Multi-Scale Frequency Learner (MSFL). The sequence branch mod...
Xian-Lang Hu, Guang-Sheng Feng, Rui-Ni Wang et al.· Computers· 0 citations
All for 1-Bit (AF1) is proposed, a genuine 1-bit PTQ framework for LLMs that consistently outperforms existing binarization-based PTQ methods in perplexity and zero-shot accuracy, providing a practical path toward deployable genuine 1-bit compression for LLMs.
Zhi-Xiong Zhao, Zu-Kang Xu, Guang-Yu Sun et al.· 1 citation
Mixture-of-Experts (MoE) architectures provide an efficient paradigm for scaling large language models (LLMs), yet fixed top-k routing activates the same number of expert slots for every token, causing substantial redundant computation. Existing expert-skipping methods often rely on router confidence, calibration data,...
Zu-Kang Xu, Zhi-Xiong Zhao, Xing Hu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.