The impact of Software Model Slicing on Software Model Completion with Large Language Models

Abstract

Large language models (LLMs) have recently shown promising results in supporting software model completion. Despite this, an open issue is what to provide as context to the LLM and how that influences completion correctness. We refer to the selection of model parts given as context to the LLM as software model slicing for LLM-based software model completion. Improving completion correctness directly benefits modelers by reducing manual corrections and additional prompting, while slicing models into smaller parts lowers token usage, reducing inference time. In this paper, we formalize model slicing in a task-independent, metamodel-agnostic manner based on graph representations of software models and instantiate different representative slicing approaches, including radius-based, modularization-based, and LLM-based slicing. Thereupon, we systematically evaluate the influence of model slicing and compare the different approaches against identity and random baselines on 1,000 real-world models from two public datasets using open-source LLMs. Our results show that appropriate model slicing significantly improves completion correctness while simultaneously reducing token usage across several structural and semantic evaluation metrics. This is particularly noteworthy as reducing context might be expected to remove relevant information and degrade performance. In particular, radius-based and modularization-based slicing outperform both the identity and random baselines. These findings establish model slicing as a key factor in LLM-based model completion and provide guidance for effective context selection for other modeling tasks.