Small Language Models and Spec-Driven Development for High-Accuracy Agentic AI Systems

Abstract

Agentic Artificial Intelligence (AI) systems are often designed with large language models (LLMs) that are assumed to achieve better performance on all tasks as they grow larger. This paper contradicts this belief by showing that small language models (SLMs), when combined with specification-driven development, are more accurate, consistent, and cost-effective than agent design in standard operations. We present a hybrid architecture in which a lightweight dispatcher takes in structured tasks, which are routed to specialist LoRA fine-tuned SLMs. The output generated is fed to a deterministic specification validator for verification. An LLM serves as a fallback for tasks that are out-of-distribution or too complex. This framework was evaluated on four representative agentic tasks: date extraction, JSON formatting, arithmetic reasoning, and schema-constrained tool calling. According to experimental results, the SLM-first consistently outperforms the LLM-only baseline significantly, showing accuracy improvement of 7.8% to 13.2%. Moreover, it achieves 7 times lower inference latency and almost an order of magnitude lower operational cost. Additionally, the proposed approach reports an output consistency of 99.8% as compared to the LLM baseline, which merely achieves 92%. This makes it suitable for production environments that demand predictable and reliable behavior. The findings suggest that specialized SLMs with explicit specifications and selective LLM fallback are a practical, scalable, and low-energy foundation for next-gen high-accuracy agentic AI systems.