Nov 2026· Computer Science and Information Technology· 0 citations· 76 references
TL;DR
Comparison of GPT-4, BERT (bidirectional encoder representations from transformers), Gemini, and DeepSeek large language models (LLM), focusing on architectures, training methodologies, and real-world applications reveals GPT-4 excels in natural language generation and complex reasoning, supporting up to 128K tokens with moderate latency and higher costs making it effective for conversational artificial intelligence (AI).
Abstract
This review provides comparative analysis of GPT-4, BERT (bidirectional encoder representations from transformers), Gemini, and DeepSeek large language models (LLM), focusing architectures, training methodologies, and real-world applications. The primary research question is: How do these models differ in design, strengths, limitations, and potential areas for enhancement? By addressing this question, the study aims to provide insights into the trade-offs and future directions for optimizing LLM performance and deployment. The analysis reveals GPT-4 excels in natural language generation and complex reasoning, supporting up to 128K tokens with moderate latency and higher costs making it effective for conversational artificial intelligence (AI). BERT excels bidirectional contextual understanding with smaller computational overhead and broad open-source adoption, effective for text classification. Gemini demonstrates superior multimodal integration processing text, image, audio, and code with context lengths up to 1M tokens, enabling cross-domain adaptability. DeepSeek excels in specialized domains like finance and programming, is optimized for efficiency and supports extended context windows exceeding 200K tokens. However, all models face challenges related to computational cost, hallucinations, and ethical concerns, necessitating further improvements. Despite advancements, LLMs continue to grapple with issues such as data bias, model interpretability, and responsible AI deployment. Future research should focus on hybrid model approaches, domain-specific fine-tuning, and transparency to mitigate risks while maximizing the transformative potential of LLMs in real-world applications.
Large language models (LLMs) are built on the classic Transformer architecture and have become a core driving force for the rapid development of modern artificial intelligence. This paper presents a systematic review of LLMs, elaborating on their fundamental working principles, mainstream open-source models, effective lightweight optimization methods, retrieval-augmented generation frameworks and key human-value-aligned technologies. Nowadays, LLMs have been widely applied in practice. Typical scenarios include intelligent text generation, professional knowledge-based question answering and automated code generation, delivering remarkable value to both industries and academia. However, their large-scale industrial application is still restricted by multiple challenges. The major issues involve content hallucination, poor model interpretability, excessive computing resource consumption, potential ethical risks and unsatisfactory multimodal integration capability. This paper also forecasts the future development directions of LLMs, such as lightweight deployment on edge devices, safety-focused human value alignment, in-depth cross-modal fusion and customized large models for vertical industries. Additionally, it collects a number of representative cases, which can offer solid references and practical guidance for relevant researchers and engineering practitioners to carry out further studies.
: In an era of increasing demand for robust multilingual natural language processing, leveraging advanced transfer learning techniques has become essential. This paper explores the application of the GPT-2 model using a comprehensive Serbian dataset of 750 million tokens. By employing meticulous data preprocessing, effective tokenization, and precise hyperparameter optimization with Optuna, the model's performance in language tasks is significantly improved. These findings underscore the model's adaptability to diverse linguistic contexts, facilitating deployment in real-world applications. The significant performance improvements highlight broader applicability in multilingual AI environments. The paper addresses key challenges such as data heterogeneity and computational efficiency, providing insights and proposing strategies for future research. By overcoming these challenges, the research demonstrates the transformative potential of refined GPT-2 models in multilingual AI. The advancements made lay a solid foundation for further exploration and refinement of multilingual language models, paving the way for more inclusive and accurate AI-driven communication tools.
Large Language Models (LLMs) have emerged as a transformative technology in artificial intelligence, significantly advancing natural language understanding, generation, and reasoning capabilities. This survey reviews the evolution of language models from early statistical approaches to modern Transformer-based architectures and summarizes key developments, including attention mechanisms, scaling laws, alignment techniques, and efficient inference methods. The paper further explores the growing impact of LLMs on everyday life and a wide range of application domains, including healthcare, finance, education, agriculture, marketing, software engineering, and scientific research. To provide a systematic perspective, LLM applications are categorized according to their maturity level and integration across major artificial intelligence subfields, such as natural language processing, multimodal learning, intelligent decision support, autonomous agents, and knowledge-based systems. The survey highlights how these models enhance automation, data-driven decision-making, personalized services, and human–AI interaction across both consumer and industrial environments. Despite their remarkable capabilities, LLMs face several critical challenges, including high computational costs, limited interpretability, hallucinations, privacy and security risks, ethical concerns, and environmental sustainability issues. Existing mitigation approaches and recent advancements are reviewed to assess their effectiveness and limitations. Finally, the paper outlines key future research directions, including trustworthy and explainable AI, efficient model architectures, domain-specific adaptation, multimodal intelligence, and human-centered alignment. This survey provides a comprehensive overview of the current landscape, challenges, and future prospects of LLMs, serving as a valuable reference for researchers and practitioners.
P. Peykani, V. Charles, Ali Emrouznejad et al.· Archives of Computational Me...· 0 citations
As an important trend in the development of artificial intelligence, the large language model (LLM) is committed to building a two-way human-computer interaction, which has excellent performance in dialogue, real-time feedback, task execution, and so on. Research LLM architectures and their trade-offs in efficiency and understanding. Based on the references, this paper summarizes the basic theoretical framework of LLM technology, including its definition, key features, development history, and core technologies. Then, the existing literature is quantitatively analyzed, and the research hotspots of LLM technology are analyzed by using Citespace bibliometric tools. Based on the LLM, this paper mainly classifies its technical architecture, which is divided into pure decoder, encoder-decoder, and sparse hybrid expert. To reflect its interactive ability, this paper supplements it from two aspects: dialogue depth and multimodal support, and makes a comprehensive comparison of several existing mainstream LLMs. LLM can deal with complex decision problems, and is an important support for intelligent decision technology by facing the human-computer interaction mechanism to realize dynamic adjustment and self-optimization.
Jingyang Guo· Advances in Engineering Tech...· 0 citations
As the context window of Large Language Models (LLMs) continues to expand, the data required to effectively train and evaluate these capabilities remains underexplored. With existing research primarily focuses on architectural optimization, there is a need for a systematic, data-centric review. This survey bridges this gap by investigating the data foundations of Long-Context Language Models (LCMs). We begin by examining current data strategies alongside their strengths and limitations, mapping the required data to desired model capabilities. Building on this, we explore how targeted training data designs drive core, often interconnected skills such as retrieval, reasoning, and aggregation. Furthermore, we analyze the evaluation landscape, illustrating how selecting appropriate benchmarks is crucial for probing capability boundaries and guiding effective model selection. Finally, we synthesize actionable guidelines for data construction and outline critical future directions to propel the advancement of long-context language models, including quantifying data quality, establishing scaling laws for length distributions, and developing dynamic evaluation frameworks.
Zechen Sun, Yuyang Sun, Zhao-yu Su et al.· Transactions of the Associat...· 0 citations
The design of transformer-based Large Language Models (LLMs) is being radically changed through new architectures that are able to overcome scalability limitations of previous designs, including Mixture-of-Experts (MoE), Multi-Head Latent Attention (MLA), and Multi-Token Prediction (MTP). As an open-weighted model released at the end of 2024, which has both state of the art architectural transparency and production scale efficiency, DeepSeeek-V3 represents the ultimate testing ground for investigating these modern technologies. This paper provides a comprehensive analysis of the architectural structure of DeepSeek-V3 based upon information from the DeepSeek-V3 Technical Report, industry benchmarking data and independent latency testing, to demonstrate how various techniques can be used to optimize training while still providing competitive performance in code generation and mathematical reasoning. In addition, latency testing conducted on a Distilled version of DeepSeek-V3, with approximately 14 billion parameters, running on a T4 GPU, reveals that although significant improvements have been made in optimizing latency there remains substantial barriers to deploying these models. Through this context, this research will serve as a reference document for practitioners and researchers who wish to understand current trends and challenges in increasing accessibility to high performance AI models.
Yassine Zouhdi, B. Hdioud· EPJ Web of Conferences· 0 citations
