Design and Deployment of Custom GPT Solutions Using Azure OpenAI Service

Abstract

Enterprise adoption of Large Language Models demands evaluation beyond model accuracy, encompassing cost, latency, compliance, and update adaptability. This paper introduces a Quantitative Evaluation Framework (QEF) to benchmark GPT deployment strategies within Azure OpenAI infrastructure. Five configurations baseline inference, prompt engineering, supervised fine-tuning, Retrieval-Augmented Generation (RAG), and a hybrid architecture were evaluated against 500 domain-specific queries drawn from 12,000 enterprise policy documents. Six metrics were measured: Factual Accuracy Score, Hallucination Rate, Mean Latency, Cost per 1,000 Queries, Update Flexibility Index, and a composite Enterprise Deployment Efficiency Index. Results confirm statistically significant performance differences across configurations (p < 0.001). RAG achieved the optimal balance between factual grounding and cost efficiency, reducing hallucination by over 70% relative to baseline. A governance-integrated Deployment Decision Framework is proposed, aligning technical, economic, and regulatory dimensions for enterprise AI architecture selection.