Performance consistently improves from conventional single-policy PEFT to multi-policy PEFT, with the proposed automatic multi-policy framework achieving the best performance under the same trainable capacity, suggesting that optimization-path organization is more effective than simply increasing adapter capacity for heterogeneous parameter-efficient fine-tuning.
Abstract
Parameter-Efficient Fine-Tuning (PEFT) commonly adapts large language models using a single shared Low-Rank Adapter (LoRA). This shared optimization space often suffers from interference when adapting heterogeneous task sequences, leading to poor transfer and catastrophic forgetting. Existing approaches mainly improve adapter expressiveness by increasing parameter capacity or composing multiple adapters, yet they still rely on a shared optimization path. In this paper, we propose an optimization-path organization framework for parameter-efficient fine-tuning of large language models, implemented as an automatic multi-policy PEFT architecture. Specifically, optimization-compatible adaptation paths are automatically organized through task grouping and task sequencing under a fixed parameter budget. The organized optimization paths are implemented as independent Quantized Low-Rank Adapters (QLoRA), enabling heterogeneous tasks to be optimized in decoupled adaptation spaces while preserving positive transfer among compatible tasks. Experiments on the TRACE benchmark demonstrate that performance consistently improves from conventional single-policy PEFT to multi-policy PEFT, with the proposed automatic multi-policy framework achieving the best performance of 44.78 under the same trainable capacity. This suggests that optimization-path organization is more effective than simply increasing adapter capacity for heterogeneous parameter-efficient fine-tuning.
Parameter-efficient fine-tuning (PEFT) methods have become essential for adapting large language models (LLMs) to downstream tasks without updating all parameters. However, the optimal PEFT strategy varies significantly across model architectures, task types, and resource constraints, making method selection a non-trivial problem. This paper presents PEFT-Assess, a systematic framework for evaluating the adaptability of four mainstream PEFT methods—LoRA, QLoRA, Adapter, and Prefix-Tuning—across five evaluation dimensions: accuracy, F1-score, training speed, memory efficiency, and generalization capability. We benchmark these methods on LLaMA-7B and Qwen-7B across six NLP tasks from the GLUE and SuperGLUE benchmarks. Furthermore, we propose a gradient boosting-based strategy selector that predicts the optimal PEFT method given task characteristics and resource budgets. Experiments show that LoRA achieves the best overall adaptability score of 0.847, while our strategy selector attains 91.2% prediction accuracy for recommending the optimal PEFT method.
Zhaoyang Hu, Shixuan Cao, Yuqi Sun et al.· International Conference on...· 0 citations
This paper presents a unified technical survey of Large Language Models (LLMs), connecting three layers of the modelling pipeline that existing surveys address in isolation: internal architecture, distributed training optimisation, and downstream adaptation. Its organising principle is the dependency between these layers—how a choice at one constrains what remains feasible at the next. The survey examines fundamental mechanisms (tokenisation, scaled dot-product attention, activation functions, and normalisation, including RMSNorm and pre- versus post-normalisation placement), then the engineering of training at scale: data, tensor, and pipeline parallelism, hybrid schemes, mixed-precision training with BF16 and FP8, ZeRO-Offload memory management, activation checkpointing, and compute-optimal scaling laws together with the conditions under which they fail. The adaptation section covers supervised and instruction fine-tuning, a comparison of parameter-efficient methods (LoRA, QLoRA, adapters, prefix and prompt tuning), Reinforcement Learning from Human Feedback with its reward-hacking failure mode, alternatives including DPO, KTO and Constitutional AI, Retrieval-Augmented Generation beyond the basic pipeline, and decoding strategies. Practical configuration guidance is given for 7B, 70B and trillion-parameter regimes. Dedicated treatments of Mixture-of-Experts architectures, long-context modelling, and hardware-aware co-design close the survey, with open challenges classified by origin and severity.
Martin Lukáč, F. Duchoň, Jakub Ivan et al.· Applied Sciences· 1 citation
Large Language Models (LLMs) trained using Chain-of-Thought (CoT) supervision have achieved state-of-the-art performance on complex reasoning tasks. However, the generation of long reasoning chains introduces substantial computational overhead during inference, limiting their deployment in low-latency and resource-constrained environments. This paper proposes AdaptiReason, a novel framework that dynamically compresses intermediate reasoning steps based on task difficulty and model confidence without requiring retraining of the underlying base model. AdaptiReason employs a lightweight difficulty estimator to determine the appropriate reasoning depth for each input, followed by a learned token-pruning policy that eliminates redundant or low-information reasoning steps. Experimental evaluation on the MATH, GSM8K, and ARC-Challenge benchmarks demonstrates that AdaptiReason reduces the average number of generated tokens by 3.7× while preserving 98.2% of the baseline reasoning accuracy. Furthermore, the proposed framework is model-agnostic and can be seamlessly integrated with instruction-tuned LLMs without requiring access to model parameters, relying solely on output logits for adaptive reasoning compression. The results demonstrate that AdaptiReason significantly improves inference efficiency while maintaining high reasoning performance, making it suitable for real-time and resource-constrained LLM applications.
V. A, Mithaguru, Amrita Kundu et al.· 2026 4th International Confe...· 0 citations
By eliminating routers entirely, CD-LoRA employs a consistency-driven alignment mechanism to enforce representation congruence across tasks in a shared low-rank space, which fosters robust, task-agnostic features without explicit partitioning overhead.
Reinforcement learning with verifiable rewards (RLVR) improves large language model reasoning, but its practical scaling is constrained by expensive on-policy rollouts and the cost of obtaining reliable targets at scale. Existing methods address sample selection, incomplete supervision, or noisy labels separately, often entangling supervision logic with distributed training and hindering controlled comparison and reuse. We present DE-Venus, a unified framework for data-efficient RLVR that treats supervision as evolving state across data preparation and policy optimization. It organizes this lifecycle into three modules: Active Data Selection allocates training and annotation budgets; Weak Supervision Construction derives learning signals from unlabeled examples; and Training-Time Supervision Refinement filters or corrects unreliable supervision. DE-Venus supports seven representative methods and a data-selection pipeline by expressing method-specific decisions as offline dataset transitions or online transformations of targets, rewards, batches, and advantages while preserving verl's distributed execution contracts. Across public benchmarks and three business scenarios, separate configurations preserve or improve model quality with only 10% of labels or as little as 13% of relevant data; selected business configurations also reduce observed convergence steps by 63%--75%. DE-Venus thus reduces annotation and training costs without sacrificing scalable RL execution.
Shen-Zhi Yang, Guang-Cheng Zhu, Kai Tang et al.· 0 citations
Evolutionary Soups, a mixture-of-experts framework for fine-grained generation control, with gating networks trained via an evolutionary algorithm, achieves the best hypervolume, linear utility, and Tchebyshev utility among controllable methods on all tasks.
Lingxiao Kong, Steffen Staab, Cong Yang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
