Improving TTP Mapping Accuracy in CTI Reports Using Structured RAG Query Configuration
Abstract
Accurately mapping attack behaviors described in Cyber Threat Intelligence (CTI) reports to the Tactics, Techniques, and Procedures (TTPs) of the MITRE ATT&CK framework is a critical challenge for responding to cyber threats and enhancing cyber resilience. However, traditional Large Language Model (LLM) and Retrieval-Augmented Generation (RAG) approaches face significant limitations. Specifically, the simple segmentation of unstructured CTI text leads to context fragmentation and the inclusion of extraneous descriptive details, which ultimately degrades the accuracy of TTP mapping. To solve these limitations, this paper proposes a methodology for constructing RAG queries based on structured fields. We extract attack behaviors from CTI reports as discrete events and organize them into seven fields (four required and three optional) representing the core components of the attack, which are designed to facilitate effective mapping to the MITRE ATT&CK framework. Our approach utilizes an LLM to perform event extraction and constructs optimized RAG queries through field combinations, thereby enhancing semantic alignment during the retrieval process. Experimental results demonstrate that the proposed method improves the F1-score by 0.249 compared to the baseline TTPFShot, achieving a maximum F1-score of 0.489 with the domain-specific model SecureBERT2. Moreover, while structured queries improve precision by constraining the retrieval scope to mitigate retrieval noise and reduce false positives, the effect of additional fields varies depending on the characteristics of the embedding model. Specifically, domain-specific models reach their peak performance with concise field configurations, whereas general-purpose models peak with the configuration integrating all fields. These results indicate that designing field combinations suited to the characteristics of the embedding model is essential, and that the proposed methodology provides a practical framework for high-precision TTP identification in complex CTI environments.