A COMPREHENSIVE OVERVIEW OF LARGE LANGUAGE MODELS AND GPT FAMILY

Abstract

The field of natural language processing (NLP) underwent a sea change with the introduction of large language models (LLMs). The NLP systems have progressed from simple rule-based systems to sophisticated transformer-based generative pre-trained models. These models have been pre-trained on vast amounts of data and encompass tens to hundreds of billions of parameters. They demonstrate excellent emergent abilities, including complex reasoning, instruction following and in-context learning. This paper presents a comprehensive overview of the LLMs with a detailed discussion of one of the most widely used LLM families, namely, the GPT model family. The architecture of the LLMs is explored to understand the nuances of the modern approach to natural language processing. The GPT family is discussed in detail to understand its rapid evolution over a short period. The various application domains of LLMs and the challenges posed by LLMs are enumerated. This paper aims to serve as a foundational resource for researchers and practitioners navigating the rapidly evolving field of large language models.