This study constructs a new annotated benchmark based on AHP and proposes the first end-to-end approach that enables LLMs to perform the complete AHP workflow, and shows that the method significantly improves alignment with expert judgments.
Abstract
LLMs are increasingly employed in a wide range of decision-making tasks. However, the opacity of their internal reasoning makes it difficult to validate or interpret their outputs, and the need for interpretability becomes especially critical in high-stakes settings. This study examines the decision-making capabilities of LLMs through the Analytic Hierarchy Process (AHP), a classical and widely used multicriteria decision-making framework. We construct a new annotated benchmark based on AHP and propose the first end-to-end approach that enables LLMs to perform the complete AHP workflow. Experiments in real-world decision problems in the legal and higher-education ranking domains show that our method significantly improves alignment with expert judgments.
Large language models (LLMs) are increasingly useful as front-end tools for decision support, particularly when extensive textual reports must be read and transformed into a structured analytical form. However, summarisation and decision making are different tasks: summarisation is primarily epistemic compression, wher...
A systematic literature review analyzing 111 papers from scientific databases on integrating AI with MCDM/A methods offers both a conceptual consolidation for scholars and a structured foundation for the design of next-generation, human-centered intelligent decision support systems (IDSS).
Bruno Cicciú, E. Frej, A. D. de Almeida· IMA Journal of Management Ma...· 1 citation
[1]. Arh, T., & Blaži, B. J. (2007). Application of Multi-Attribute Decision Making Approach to Learning Management Systems Evaluation . JOURNAL OF COMPUTERS, 2(10), 28-37. [2]. BÂRA, A., BOTHA, I., LUNGU, I., & OPREA, S. V. (2013). Decision Support System in National Power Companies A Practical Example (Part I) . Data...
H. Singh· International Journal of Sci...· 0 citations
Put forward by Thomas L. Saaty in the 1970s, the Analytic Hierarchy Process (AHP) has become one of the most widely used multi‑criteria decision‑making tools around the world. Despite its solid theoretical foundation and broad application scope, the conventional AHP workflow requires constructing full pairwise comparis...
Jia-Hui Zhao· Advances in Social Behavior...· 0 citations
This work proposes integrating and advancing LLM and optimization modeling to empower organizational decision-makers to model and solve such complex problems without requiring deep expertise in optimization, thereby enabling much more widespread improved decision-making and increasing by orders of magnitude the benefit...
Segev Wasserkrug, Léonard Boussioux, D. den Hertog et al.· 0 citations
Several multicriteria methods have been created for use in problem situations that are considered complex because multiple actors and/or organisations are involved. Such methods are easily combined with problem shaping and structuring methods to reduce complexity when a situation is new, ill‐ or not well‐structured....
M. Norese· Journal of Multi-Criteria De...· 0 citations
