Comparing large language models and prompting techniques to control an embodied persuasive coach

Abstract

Embodied agents play a crucial role in applications designed to influence user behaviour. Traditionally, crafting these agents requires considerable human effort. Large Language Models (LLMs) might open the possibility of creating agents capable of autonomous, open-ended interactions without labour-intensive development processes. We explore creating an embodied agent to coach users in real-time through slow and deep breathing exercises using an LLM. The LLM exploits a text-based context to generate a composition of predefined behaviours for interacting with the user through both verbal and nonverbal communication. This context includes essential details like the user’s respiratory rate to monitor the exercise. First, we assess the feasibility of this approach, showing that the LLM-based coach generates believable contingent behaviour compositions. Then, we conduct a 4 × 2 × 2 study evaluating how four different LLMs and two prompting techniques (reasoning- and learning-oriented) affect latency, appropriateness, and reliability of the behaviour compositions. Our findings show that more recent LLM models, combined with appropriate prompting techniques, improve appropriateness and reliability over previous models. However, our two studies also identify challenges, including latency, inappropriate feedback, and hallucinations. We discuss the limitations of using only LLMs to create persuasive agents capable of real-time user interactions, suggesting a need for hybrid approaches.