Jun 2026· International Conference on Mobile Data Management· pp. 363-366· 0 citations· 21 references
Abstract
Deploying large language models in resource-constrained environments such as mobile devices requires effective compression. Pruning and quantization are largely used, yet applying pruning to a new model typically demands manual inspection of its architecture to identify layer names, MLP structure, and projection patterns, a process that is tedious, error-prone, and must be repeated for every new model design. We present ShrinkLLM, an end-to-end compression system that eliminates the manual inspection of model structure during pruning by delegating architectural analysis to an AI agent. Given a causal LLM, the agent automatically identifies MLP layer organization, expansion layers, gate projections for Gated Linear Unit architectures, and down-projection layers, structural information that is then passed directly to a compression engine to perform neuron pruning, quantization, or a combination of both. We demonstrate the system on llama-3.1-8b-instruct, Qwen2.5-3b-instruct and Gemma-2-2b-it showcasing how the agent correctly resolves architecturespecific structural details and drives compression across all MLP layers automatically.
Large Language Models (LLMs) trained using Chain-of-Thought (CoT) supervision have achieved state-of-the-art performance on complex reasoning tasks. However, the generation of long reasoning chains introduces substantial computational overhead during inference, limiting their deployment in low-latency and resource-cons...
V. A, Mithaguru, Amrita Kundu et al.· 2026 4th International Confe...· 0 citations
CoRun is presented, a scheduling-based system that achieves deterministic inference without requiring batch invariance, and employs isolated prefill and fixed-shape batched decode to handle the two stages of LLM inference, respectively, leveraging CUDA graphs for efficient execution and simplified implementation.
Shiju Zhao, Jiacheng Yang, Qi-Hang Chen et al.· 2 citations
This study introduces CodeGrep, a 14B retrieval agent trained end-to-end with GRPO to issue multi-turn parallel grep, glob, and read tool calls and return candidate files to a frozen downstream coding agent, and applies the efficiency signal at the advantage layer rather than the reward layer to reduce KL drift and tra...
Wu-Ya Chen, Yihao Yang, Yang Cao et al.· 2 citations
This work proposes a mechanism that bridges these two paradigms: an LLM produces a high-quality seed architecture, then decomposes it into a "slotted architecture", a scaffold with named, interchangeable module slots that automatically defines a bounded, task-specific search space for conventional NAS to explore, witho...
Seokhoon Jeong, Mijung Kim, Taehwan Kim· 0 citations
OptiFlow is presented, among the first LLM-driven frameworks for automated design of high-performance collective communication algorithms, with key insight is a two-layer decomposition: the LLM generates compact data-movement intent expressed in a domain-specific language, while deterministic scheduling algorithms comp...
Fei Long, Ziyue Yang, Kaihui Gao et al.· Asia-Pacific Workshop on Net...· 1 citation
SWE-Pruner Pro is proposed, which prunes tool outputs directly inside the agent, with a small head turns the agent's own internal representations into a keep-or-prune label for each line, with a length-aware embedding keyed to each tool output's line count.
Yuhang Wang, Yuling Shi, Shaoqiu Zhang et al.· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
