LogPISA: An Improved Pre-Training and Tuning Pipeline for Log Understanding With Invariant and Semantic-Aware Objectives

FlanBC: A Semantic-Structural Sequence Labeling Framework for Log Parsing

Log parsing converts raw system logs into structured templates and is a key preprocessing step for Artificial Intelligence for IT Operations (AIOps). Existing parsers face a practical trade-off: rule-based methods offer high throughput but limited adaptability across heterogeneous log sources, whereas Large Language Model (LLM)-based parsers achieve broader semantic coverage at the cost of inference latency, privacy exposure, and cloud dependency. This paper presents FlanBC, a log parsing framework that formulates template extraction as a BIO (Beginning, Inside, Outside) sequence-labeling task and integrates a Flan-T5 semantic encoder, Bidirectional Long Short-Term Memory (BiLSTM) layers for local sequential modeling, and a Conditional Random Field (CRF) decoder for structured label prediction. Log-specific preprocessing and a subword-to-token alignment mechanism adapt the general-purpose encoder to semi-structured log data. A layer-freezing strategy reduces the number of parameters updated during training. The framework supports local inference without external API dependency. Experiments on three benchmark datasets from LogHub (HDFS, BGL, OpenStack) under a supervised random-split setup evaluate parsing accuracy, training efficiency, statistical stability across random seeds, and component contributions. FlanBC achieves a Group Accuracy of 99.32% on HDFS and 98.47% on BGL, with an inference throughput of 700+ logs/s on a consumer-grade GPU. On OpenStack, performance is lower (GA = 92.54%), reflecting the challenge that diverse natural-language-like logs pose for compact encoder-based models. Under a stricter template-disjoint split that prevents template overlap between training and test sets, FlanBC achieves an average Group Accuracy of 91.14%, indicating that the model generalizes to unseen templates beyond in-distribution recognition. Ablation results indicate that the semantic encoder, BiLSTM module, and CRF decoder each contribute to prediction accuracy. These findings suggest that domain-adapted semantic encoders combined with structured decoding offer a practical accuracy–efficiency balance for log parsing in settings where local, cloud-free inference is preferred.

Enhancing blocking for entity matching with pre-trained language model and masked autoencoder

AdaptiReason: Adaptive Chain-of-Thought Compression for Efficient Large Language Model Inference

Large Language Models (LLMs) trained using Chain-of-Thought (CoT) supervision have achieved state-of-the-art performance on complex reasoning tasks. However, the generation of long reasoning chains introduces substantial computational overhead during inference, limiting their deployment in low-latency and resource-constrained environments. This paper proposes AdaptiReason, a novel framework that dynamically compresses intermediate reasoning steps based on task difficulty and model confidence without requiring retraining of the underlying base model. AdaptiReason employs a lightweight difficulty estimator to determine the appropriate reasoning depth for each input, followed by a learned token-pruning policy that eliminates redundant or low-information reasoning steps. Experimental evaluation on the MATH, GSM8K, and ARC-Challenge benchmarks demonstrates that AdaptiReason reduces the average number of generated tokens by 3.7× while preserving 98.2% of the baseline reasoning accuracy. Furthermore, the proposed framework is model-agnostic and can be seamlessly integrated with instruction-tuned LLMs without requiring access to model parameters, relying solely on output logits for adaptive reasoning compression. The results demonstrate that AdaptiReason significantly improves inference efficiency while maintaining high reasoning performance, making it suitable for real-time and resource-constrained LLM applications.

DeepSeek-V3: Architecture and Optimizations-A Practical Review

The design of transformer-based Large Language Models (LLMs) is being radically changed through new architectures that are able to overcome scalability limitations of previous designs, including Mixture-of-Experts (MoE), Multi-Head Latent Attention (MLA), and Multi-Token Prediction (MTP). As an open-weighted model released at the end of 2024, which has both state of the art architectural transparency and production scale efficiency, DeepSeeek-V3 represents the ultimate testing ground for investigating these modern technologies. This paper provides a comprehensive analysis of the architectural structure of DeepSeek-V3 based upon information from the DeepSeek-V3 Technical Report, industry benchmarking data and independent latency testing, to demonstrate how various techniques can be used to optimize training while still providing competitive performance in code generation and mathematical reasoning. In addition, latency testing conducted on a Distilled version of DeepSeek-V3, with approximately 14 billion parameters, running on a T4 GPU, reveals that although significant improvements have been made in optimizing latency there remains substantial barriers to deploying these models. Through this context, this research will serve as a reference document for practitioners and researchers who wish to understand current trends and challenges in increasing accessibility to high performance AI models.

AutoDavis: Automatic and Dynamic Evaluation Protocol of Large Vision-Language Models on Visual Question-Answering

Large Vision-Language Models (LVLMs) have become essential for advancing the integration of visual and linguistic information. While existing benchmarks have laid a solid foundation for evaluation, they are often static, resource-intensive to build, and limited in adaptability. In comparison, automatic evaluation has shown promise in the textual domain, but the visual modality remains far less explored. To advance this frontier, in this work, we introduce AutoDavis, a first-of-its-kind automatic and dynamic evaluation protocol that enables on-demand benchmarking of LVLMs across specific capability dimensions. AutoDavis leverages text-to-image models to generate relevant image samples and then utilizes LVLMs to orchestrate visual question-answering (VQA) tasks, completing the evaluation process efficiently and flexibly. To ensure data diversity, our framework employs a hierarchical aspect-driven generation process enhanced with semantic graph-based constraints. To safeguard reliability, the framework incorporates a self-validation mechanism to detect and correct errors, along with an error-driven adjustment module to mitigate potential bias. Through an extensive evaluation of 11 popular LVLMs across five demanded user inputs (i.e., evaluation capabilities), the framework shows effectiveness and reliability, offering a new paradigm for dynamic benchmarking of multimodal intelligence. View website for code and data.

LoKiFormer: Locality-aware Attention with Decoupled Knowledge Memory for Efficient Large Language Model Pretraining

Large language models (LLMs) have achieved remarkable breakthroughs across various applications. However, their architectures remain inefficient in pretraining due to two main limitations: (i) self-attention lacks an explicit inductive bias for locality, leading to redundant modeling of sequence-internal local information; (ii) mixture-of-experts (MoE) implicitly couples knowledge storage with computational pathways, hindering flexible access to sequence-external global knowledge. To overcome these limitations, we propose LoKiFormer, a novel LLM architecture that augments the standard decoder with two dedicated modules: 1) Local Fusion Attention (LFA), which incorporates a convolutional fusion to attention, explicitly capturing local patterns and allowing the attention to operate on more informative representations; 2) Knowledge Memory Module (KMM), which introduces a parametric key-value memory that explicitly stores global knowledge in addressable slots, decoupling storage from computation and enabling direct knowledge retrieval. Together, these modules enable LoKiFormer to achieve more efficient and effective integration of information at both levels. Experimental results show that LoKiFormer converges 1.33x faster in pre-training than baseline models, underscoring its superiority over existing LLM architectures.