Large language models (LLMs) are built on the classic Transformer architecture and have become a core driving force for the rapid development of modern artificial intelligence. This paper presents a systematic review of LLMs, elaborating on their fundamental working principles, mainstream open-source models, effective lightweight optimization methods, retrieval-augmented generation frameworks and key human-value-aligned technologies. Nowadays, LLMs have been widely applied in practice. Typical scenarios include intelligent text generation, professional knowledge-based question answering and automated code generation, delivering remarkable value to both industries and academia. However, their large-scale industrial application is still restricted by multiple challenges. The major issues involve content hallucination, poor model interpretability, excessive computing resource consumption, potential ethical risks and unsatisfactory multimodal integration capability. This paper also forecasts the future development directions of LLMs, such as lightweight deployment on edge devices, safety-focused human value alignment, in-depth cross-modal fusion and customized large models for vertical industries. Additionally, it collects a number of representative cases, which can offer solid references and practical guidance for relevant researchers and engineering practitioners to carry out further studies.
This survey reviews the evolution of language models from early statistical approaches to modern Transformer-based architectures and summarizes key developments, including attention mechanisms, scaling laws, alignment techniques, and efficient inference methods.
P. Peykani, V. Charles, Ali Emrouznejad et al.· Archives of Computational Me...· 0 citations
Comparison of GPT-4, BERT (bidirectional encoder representations from transformers), Gemini, and DeepSeek large language models (LLM), focusing on architectures, training methodologies, and real-world applications reveals GPT-4 excels in natural language generation and complex reasoning, supporting up to 128K tokens with moderate latency and higher costs making it effective for conversational artificial intelligence (AI).
Kavish Sanghvi, Aparna S. Sharma, Surbhi Hooda· Computer Science and Informa...· 0 citations
This paper summarizes the basic theoretical framework of LLM technology, including its definition, key features, development history, and core technologies, and classifies its technical architecture, which is divided into pure decoder, encoder-decoder, and sparse hybrid expert.
Jingyang Guo· Advances in Engineering Tech...· 0 citations
: In an era of increasing demand for robust multilingual natural language processing, leveraging advanced transfer learning techniques has become essential. This paper explores the application of the GPT-2 model using a comprehensive Serbian dataset of 750 million tokens. By employing meticulous data preprocessing, effective tokenization, and precise hyperparameter optimization with Optuna, the model's performance in language tasks is significantly improved. These findings underscore the model's adaptability to diverse linguistic contexts, facilitating deployment in real-world applications. The significant performance improvements highlight broader applicability in multilingual AI environments. The paper addresses key challenges such as data heterogeneity and computational efficiency, providing insights and proposing strategies for future research. By overcoming these challenges, the research demonstrates the transformative potential of refined GPT-2 models in multilingual AI. The advancements made lay a solid foundation for further exploration and refinement of multilingual language models, paving the way for more inclusive and accurate AI-driven communication tools.
Large language models (LLMs) have achieved remarkable success in high-resource languages, yet their performance on Traditional Mongolian remains highly limited. A primary bottleneck is the absence of a systematic evaluation framework, which precludes quantitative comparison and obscures directions for model optimization. In this paper, we introduce TM-Bench, the first comprehensive benchmark for LLMs on Traditional Mongolian. TM-Bench adopts a hybrid construction strategy consisting of human-verified Translation-based Adaptation, Expert-Original Authoring, and Semi-automated Synthesis. It comprises 18,357 instances spanning five tasks across both natural language understanding and generation to evaluate models' reasoning, knowledge application, and linguistic proficiency. We conduct systematic evaluations across representative model families. The results show that on understanding tasks, model performance lags significantly behind high-resource languages, with only a few models performing slightly above the random baseline. For generation tasks, both automatic metrics and double-blind human evaluations reveal severe semantic collapse, failing to generate coherent text and often producing unreadable gibberish. These findings underscore the critical role of TM-Bench as a foundational infrastructure for evaluating LLMs in Traditional Mongolian and catalyzing future model optimization. Our benchmark and code are available at https://github.com/gao1948083886/TM-Bench.
Zhenjie Gao, Feilong Bao, Aruukhan Bai et al.· Annual International ACM SIG...· 0 citations
Natural Language Processing (NLP) has become a cornerstone of artificial intelligence, enabling machines to process, understand, and generate human language. With the increasing adoption of machine learning, deep learning, and large-scale transformer models, NLP has made significant progress in the past decade. Modern NLP systems are used in applications ranging from machine translation, sentiment analysis, chatbots, and automated summarization to knowledge extraction and conversational agents. Transformer-based architectures and large language models (LLMs) have drastically improved context understanding, semantic representation, and generation quality. This paper provides a comprehensive survey of NLP, discussing its components, historical evolution, applications, datasets, evaluation metrics, recent advancements, and challenges. A detailed literature review based on studies from 2022–2026 is presented, highlighting the role of transformer models, deep learning architectures, and emerging trends such as multi-lingual NLP, domain-specific models, and ethical considerations. Finally, the paper explores future research directions to address low-resource languages, model fairness, and real-world applicability.
Ankita Vijay Shinde, Sunil Tanaji Salunkhe,, Bharati Bhaskar Khandagale· International Journal of Adv...· 0 citations