2026· International Conference on Language Resources and Evaluation· pp. 10313-10323· 0 citations· 38 references
Computer Science
TL;DR
PRiSM is proposed, a training-free approach for layer-wise partial fine-tuning that leverages the cosine similarity between pre-trained aggregate token representations across layers to identify inter-layer relationships and achieves competitive performance compared to full fine-tuning.
In recent years, the performance of large language models (LLMs) on reasoning tasks has been remarkable, even surpassing human capabilities on various benchmarks. However, there remains a lack of clear understanding in the academic community regarding how the structure and internal parameters of LLMs progressively solv...
Recent training-free pruning approaches for Multimodal Large Language Models (MLLMs) effectively cut computational overhead by exploiting visual redundancy or text-vision attention. However, they frequently suffer from semantic degradation due to their task-agnostic design or unreliable attention estimates. Based on ou...
Shuo Yang, Chang-Bai Li, Lin-Lin Yang et al.· 0 citations
Experimental results across three models across both domain-specific and general benchmarks demonstrate that NGFT significantly outperforms existing mainstream fine-tuning methods in both efficiency and performance, while effectively mitigating catastrophic forgetting.
Ze-Yu Wu, Junchao Wu, Shu-Dong Liu et al.· 0 citations
Parameter-efficient fine-tuning (PEFT) enables large language models (LLMs) to adapt to specialized tasks, but often at the cost of degrading general capabilities acquired during pretraining. Existing approaches primarily mitigate this trade-off through data replay or regularization, relying on additional data or expli...
Zhi-Qian Pang, Zi-Hong Sun, Qi Xie et al.· 0 citations
GLA-LoRA establishes a unified learning strategy that synergistically integrates multi-granular contrastive learning with knowledge distillation and establishes that explicit global-local knowledge alignment is essential for achieving high-fidelity, parameter-efficient fine-tuning across diverse language tasks.
Transformer-based architectures have established state-of-the-art benchmarks across Natural Language Processing (NLP) tasks; however, the computational overhead of full fine-tuning remains a significant barrier to scalable deployment. This paper presents SentiMatrix, a systematic evaluation of Low-Rank Adaptation (LoRA...
Md. Easin Arafat, Muhammad Usman Akmal, Ali S. Abosinnee et al.· Machine-mediated learning· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
