From Results to Practice: A Human-LLM Collaboration Framework for Spanning Causal Machine Learning Knowledge Boundaries

Abstract

Organizations are beginning to deploy causal machine learning (causal ML) models for estimating average treatment effects and treatment effect differences for various subgroups, as well as identifying causal mechanisms driving treatment effects. For instance, in domains such as health care and marketing, causal ML is being used to identify who is most likely to benefit from or be receptive to treatments or interventions, including medical treatments and marketing campaigns. However, while causal ML models can produce highly accurate results when causal assumptions are met, the outputs can be confusing to work with. For instance, it is easy to conflate risk scores with conditional average treatment effects (CATEs), as risk scores from predictive models identify who is most likely to experience an outcome while CATEs from causal models identify who is most likely to benefit (or not benefit) from a treatment. A practitioner reviewing causal ML results may not fully understand such nuances and, thus, may not effectively convert relevant causal ML outputs into practice. Thus, a particular challenge relevant to causal ML, and extensible to any complex set of digital artifact outputs, is that effective practice transformation requires effective boundary spanning. To address this challenge, we propose a framework that conceptualizes large language models (LLMs) as dynamic boundary spanning objects. The framework leverages different forms of human-LLM collaborations throughout the causal ML boundary spanning process, toward the goal of successful practice transformation. We further provide an example of how the framework can be applied to a case of complex causal ML results. We ultimately seek to extend boundary spanning theory and provide a pathway for translating causal ML results across syntactic, semantic, and pragmatic boundaries.