Jun 2026· Advances in Engineering Technology Research· 0 citations· 26 references
TL;DR
This paper summarizes the basic theoretical framework of LLM technology, including its definition, key features, development history, and core technologies, and classifies its technical architecture, which is divided into pure decoder, encoder-decoder, and sparse hybrid expert.
Abstract
As an important trend in the development of artificial intelligence, the large language model (LLM) is committed to building a two-way human-computer interaction, which has excellent performance in dialogue, real-time feedback, task execution, and so on. Research LLM architectures and their trade-offs in efficiency and understanding. Based on the references, this paper summarizes the basic theoretical framework of LLM technology, including its definition, key features, development history, and core technologies. Then, the existing literature is quantitatively analyzed, and the research hotspots of LLM technology are analyzed by using Citespace bibliometric tools. Based on the LLM, this paper mainly classifies its technical architecture, which is divided into pure decoder, encoder-decoder, and sparse hybrid expert. To reflect its interactive ability, this paper supplements it from two aspects: dialogue depth and multimodal support, and makes a comprehensive comparison of several existing mainstream LLMs. LLM can deal with complex decision problems, and is an important support for intelligent decision technology by facing the human-computer interaction mechanism to realize dynamic adjustment and self-optimization.
Large language models (LLMs) are built on the classic Transformer architecture and have become a core driving force for the rapid development of modern artificial intelligence. This paper presents a systematic review of LLMs, elaborating on their fundamental working principles, mainstream open-source models, effective lightweight optimization methods, retrieval-augmented generation frameworks and key human-value-aligned technologies. Nowadays, LLMs have been widely applied in practice. Typical scenarios include intelligent text generation, professional knowledge-based question answering and automated code generation, delivering remarkable value to both industries and academia. However, their large-scale industrial application is still restricted by multiple challenges. The major issues involve content hallucination, poor model interpretability, excessive computing resource consumption, potential ethical risks and unsatisfactory multimodal integration capability. This paper also forecasts the future development directions of LLMs, such as lightweight deployment on edge devices, safety-focused human value alignment, in-depth cross-modal fusion and customized large models for vertical industries. Additionally, it collects a number of representative cases, which can offer solid references and practical guidance for relevant researchers and engineering practitioners to carry out further studies.
Comparison of GPT-4, BERT (bidirectional encoder representations from transformers), Gemini, and DeepSeek large language models (LLM), focusing on architectures, training methodologies, and real-world applications reveals GPT-4 excels in natural language generation and complex reasoning, supporting up to 128K tokens with moderate latency and higher costs making it effective for conversational artificial intelligence (AI).
Kavish Sanghvi, Aparna S. Sharma, Surbhi Hooda· Computer Science and Informa...· 0 citations
In recent years, large language models (LLMs) have achieved significant results in natural language processing. They are applied to various tasks, including text generation, question answering, automatic summarization, code generation, and complex reasoning. With the increasingly complex real scenarios, the length of input text that models need to deal with also grows. Thus, the long-context processing ability of language models has gradually become an important factor in evaluating the practicability of LLMs. This paper gives an introduction to the long-context processing ability of large language models. It first introduces the background of large language models and the basic concept of long-context processing. It then summarizes the main technical methods of long-context modeling, such as improving positional encoding, training stage expansion, inference-stage optimization, and architecture-level innovation. Third, the paper also discusses the use of long-context ability in long-document question answering, long-text summarization, multi-document integration, code understanding and long-context evaluation tasks. Then, summarize the current main challenges and prospects of research work. This paper argues that the ability of long context should not only come from increasing the context window, but also from the ability of the model to locate, integrate and reason about important information in long text.
Jun Wu· Applied and Computational En...· 0 citations
Abstract--Natural Language Processing (NLP) has emerged as a major branch of Artificial Intelligence (AI) that allows computers to effectively understand, interpret, and generate human language. The recent advances in machine learning, deep learning and transformer-based architectures have considerably improved the performance of NLP systems on a wide range of applications. This paper presents a comprehensive review of the evolution of NLP from traditional rule-based approaches to modern transformer models including BERT and GPT. It covers the major methodologies including text preprocessing, feature representation, machine learning, deep learning and transformer-based language modelling. Moreover, the study elaborates on the use of NLP in healthcare, education, business, finance, customer service, social media, and intelligent communication and highlights its role in enhancing automation, decision-making, and human–computer interaction. In addition, the paper discusses the major challenges faced by current NLP systems, including language ambiguity, multilingual processing, computational complexity, model bias, privacy, and explainability. Finally, future research directions, including lightweight language models, multilingual NLP, explainable AI, and multimodal intelligence, are presented. The findings demonstrate that NLP continues to transform intelligent systems and is expected to play an increasingly significant role in the development of next-generation AI technologies.
P. Kalaiselvi· International Journal of Eme...· 0 citations
This survey reviews the evolution of language models from early statistical approaches to modern Transformer-based architectures and summarizes key developments, including attention mechanisms, scaling laws, alignment techniques, and efficient inference methods.
P. Peykani, V. Charles, Ali Emrouznejad et al.· Archives of Computational Me...· 0 citations
Large Vision-Language Models (LVLMs) have become essential for advancing the integration of visual and linguistic information. While existing benchmarks have laid a solid foundation for evaluation, they are often static, resource-intensive to build, and limited in adaptability. In comparison, automatic evaluation has shown promise in the textual domain, but the visual modality remains far less explored. To advance this frontier, in this work, we introduce AutoDavis, a first-of-its-kind automatic and dynamic evaluation protocol that enables on-demand benchmarking of LVLMs across specific capability dimensions. AutoDavis leverages text-to-image models to generate relevant image samples and then utilizes LVLMs to orchestrate visual question-answering (VQA) tasks, completing the evaluation process efficiently and flexibly. To ensure data diversity, our framework employs a hierarchical aspect-driven generation process enhanced with semantic graph-based constraints. To safeguard reliability, the framework incorporates a self-validation mechanism to detect and correct errors, along with an error-driven adjustment module to mitigate potential bias. Through an extensive evaluation of 11 popular LVLMs across five demanded user inputs (i.e., evaluation capabilities), the framework shows effectiveness and reliability, offering a new paradigm for dynamic benchmarking of multimodal intelligence. View website for code and data.
Han Bao, Yue Huang, Yanbo Wang et al.· Proceedings of the 32nd ACM...· 0 citations