Prompting Strategy Evaluation for Offline Code Generation Agents on Resource-Constrained Hardware: A Comparative Study of Sub-2B Parameter Models

Abstract

The deployment of large language model (LLM)-based code generation agents typicallyrelies on cloud infrastructure, limiting their applicability in air-gapped, low-resource, oroffline environments. This paper investigates the feasibility and effectiveness of runningsub-2B parameter LLMs as offline code generation agents on consumer-grade hardwarewith 8GB RAM. We conduct a systematic evaluation of three prompting strategies—DirectPrompting, Chain-of-Thought (CoT), and ReAct—across three small open-source models:Qwen2.5-Coder 0.5B, DeepSeek-Coder 1.3B, and Llama 3.2 1B. We evaluate performanceon two code generation task categories—function synthesis and bug fixing—using automated test-case execution as the primary scoring metric. Our results reveal that DirectPrompting consistently outperforms CoT and ReAct strategies across all models, withLlama 3.2 1B achieving 100% task completion under Direct Prompting. Notably, Chainof-Thought prompting—designed to improve reasoning in larger models—significantlydegrades performance in sub-2B models, with Qwen2.5-Coder 0.5B dropping to 12.5%completion under CoT. These findings have practical implications for the deployment ofoffline AI coding assistants in air-gapped systems, embedded development environments,and low-bandwidth regions.