Interpretable Column Annotation with LLM-Symbolized Decision Process Materialization

Interpretable Unsupervised Community Detection with LLM-Symbolized Structured Processes

LUCID, an LLM-guided, interpretable, training-free, and unsupervised community detection method, designed as a four-stage pipeline that achieves state-of-the-art performance and consistently outperforms leading unsupervised and semi-supervised baselines.

Representation, Retrieval, and Decision Space: A Case Study on Diagnosing LLM Failures in Column Type Annotation

FlanBC: A Semantic-Structural Sequence Labeling Framework for Log Parsing

Log parsing converts raw system logs into structured templates and is a key preprocessing step for Artificial Intelligence for IT Operations (AIOps). Existing parsers face a practical trade-off: rule-based methods offer high throughput but limited adaptability across heterogeneous log sources, whereas Large Language Model (LLM)-based parsers achieve broader semantic coverage at the cost of inference latency, privacy exposure, and cloud dependency. This paper presents FlanBC, a log parsing framework that formulates template extraction as a BIO (Beginning, Inside, Outside) sequence-labeling task and integrates a Flan-T5 semantic encoder, Bidirectional Long Short-Term Memory (BiLSTM) layers for local sequential modeling, and a Conditional Random Field (CRF) decoder for structured label prediction. Log-specific preprocessing and a subword-to-token alignment mechanism adapt the general-purpose encoder to semi-structured log data. A layer-freezing strategy reduces the number of parameters updated during training. The framework supports local inference without external API dependency. Experiments on three benchmark datasets from LogHub (HDFS, BGL, OpenStack) under a supervised random-split setup evaluate parsing accuracy, training efficiency, statistical stability across random seeds, and component contributions. FlanBC achieves a Group Accuracy of 99.32% on HDFS and 98.47% on BGL, with an inference throughput of 700+ logs/s on a consumer-grade GPU. On OpenStack, performance is lower (GA = 92.54%), reflecting the challenge that diverse natural-language-like logs pose for compact encoder-based models. Under a stricter template-disjoint split that prevents template overlap between training and test sets, FlanBC achieves an average Group Accuracy of 91.14%, indicating that the model generalizes to unseen templates beyond in-distribution recognition. Ablation results indicate that the semantic encoder, BiLSTM module, and CRF decoder each contribute to prediction accuracy. These findings suggest that domain-adapted semantic encoders combined with structured decoding offer a practical accuracy–efficiency balance for log parsing in settings where local, cloud-free inference is preferred.

Automatic Domain Classification of Tabular Datasets Using Large Language Models

It is asserted that the present contribution consists of an interpretable domain palette, a constructed benchmark of diverse tabular datasets, and reproducible code and data to enable further research on domain discovery and domain-aware tooling for tabular data.

ChartAnno: Evaluating MLLMs for Chart Annotation Generation

Multimodal large language models (MLLMs) have made significant progress in chart understanding, generation, and editing, but their ability to annotate existing charts remains underexplored. Annotating charts is a common yet challenging communicative task, requiring models to infer intended messages, interpret chart semantics, and place appropriate textual or graphical elements. To address this gap, we introduce ChartAnno, a benchmark for evaluating MLLMs on chart annotation generation. It contains 1,200 real-world charts with paired code and annotation instructions across three levels of instruction specificity. We evaluate 10 representative MLLMs under two primary input settings: (1) chart code alone and (2) both chart code and chart image, and further include a chart image-only ablation study. Results show that proprietary models remain stronger overall, although large-scale open-source models narrow the gap. More specific instructions improve annotation quality, while inferring abstract intent remains most difficult for current MLLMs. Providing chart images brings limited overall gains, with improvements mainly appearing in design-related metrics. These findings highlight chart annotation generation as a challenging task requiring semantic grounding and effective annotation design. Code and data will be released in a future version.

Improving LLM-based event extraction with annotation guidelines

Event extraction constitutes a foundational task in information extraction, but reliance on laborious and expensive human annotations severely restricts the availability of training datasets. While recent works have explored Large Language Models (LLMs) as example-driven (or zero-shot) annotators, they are substantially outperformed by supervised techniques on structured extraction tasks, such as event detection and argument extraction, possibly on account of underspecified task instructions. In this work, we investigate to which extent LLMs can benefit from dataset-specific, detailed annotation guidelines that more precisely represent the dataset's underlying (human) annotation procedures. To this end, we propose a guideline-based, three-stage LLM annotation framework for event extraction that incorporates detailed event annotation guidelines and supports multiple LLM annotators to improve robustness. Using the comprehensive and well-documented ACE 2005 English Annotation Guidelines for Events as a reference document, we evaluate four LLMs across three guideline-compliant benchmark datasets. Our findings indicate that, depending on model choice, guideline specificity, and the dataset's relative label accuracy, employing detailed guidelines can considerably boost event extraction performance, gaining up to 6.7 F1 points over commonly used bare-minimum instructions, with particularly remarkable improvements for reasoning-based models. Furthermore, we demonstrate that augmenting existing datasets with LLM-generated argument annotations can improve argument extraction performance under soft-matching evaluation. Overall, our experiments emphasize the importance of annotation guidelines (as well as their specificity) for LLM-based annotations, providing valuable insights on leveraging LLMs as guideline-compliant annotators.