LAMDA: Large Language Model as Decision Analyst
TL;DR
This work designs a workflow that prompts LLMs to output elements of an influence diagram and resolves issues through verification and regeneration and applies LAMDA to discussions by groups of disease control experts on a hypothetical pandemic to demonstrate its real-world applicability.
Abstract
Influence diagrams address the challenges of decision-making under risk by structuring information, decisions and values, while clearly depicting uncertainties and probabilistic dependencies. However, constructing an influence diagram requires expertise in decision analysis and is further complicated by the need to process large amounts of contextual information. This work focuses on the construction of influence diagrams from natural language input by leveraging large language models (LLMs). We design a workflow that prompts LLMs to output elements of an influence diagram and resolves issues through verification and regeneration. We also construct a new dataset of typical decision problems under risk. Evaluations using this dataset demonstrate that our framework effectively identifies key factors and relationships in natural language, making better decisions than standalone LLMs and LLMs enhanced with standard techniques such as chain-of-thought (CoT). Finally, we apply LAMDA to discussions by groups of disease control experts on a hypothetical pandemic to demonstrate its real-world applicability. Overall, the method effectively synthesizes unstructured text into an influence diagram that, while subject to human review and refinement, enhances information processing and supports decision-making.