Activity-Based Costing Framework for Total Cost of Ownership Analysis of LLM Services

Abstract

The proliferation of Large Language Models (LLMs) requires robust financial planning, yet traditional software cost models cannot capture the unique economics of generative AI. This paper presents a formal Total Cost of Ownership (TCO) framework grounded in Activity-Based Costing (ABC). We deliver a dual-layered accounting architecture: a mathematically tractable, provably convex engine for technical infrastructure costs, alongside a comprehensive, modular taxonomy blueprint for human operational, governance, and organizational cost pools. The framework systematically incorporates LLM-specific cost drivers, including token consumption, Retrieval-Augmented Generation (RAG) operations, and agentic inference steps. To demonstrate practical applicability, the framework is subjected to a multi-faceted empirical validation programme: the technical infrastructure submodel is validated on a 12-month production RAG chatbot deployment, yielding a predictive formula that links high-level business metrics directly to infrastructure expenses with sub-5% forecasting error, and is further exercised on a second institutional deployment through a synthetically augmented dataset of 95,150 user sessions. Complementary cross-case scenario simulations—spanning regulated, high-scale API, and autonomous agentic regimes—together with hybrid local–cloud serving and retrieval-versus-fine-tuning comparisons, illustrate the framework’s structural generalizability across distinct governance, throughput, and multi-step execution regimes. By integrating time-varying vendor prices, stochastic uncertainty, and hybrid deployment extensions, this work provides a rigorous, transparent decision-support tool for the strategic financial management of LLM-based services.