Probing Large Language Models for Autonomous Driving Behavior
TL;DR
Findings characterize model-level behavioral priors relevant to LLM choice and prompt design in AV applications, and show that LLM decisions reflect a mix of model characteristics, linguistic framing, and scenario context.
Abstract
As large language models (LLMs) are increasingly integrated into autonomous vehicles (AV), understanding their reasoning and behavioral tendencies becomes essential. Trained on vast datasets, LLMs carry behavioral priors and social biases that may shape their driving decisions. Without such insight, developers may struggle to align models for AV needs. To address this, we probe prompt-conditioned high-level action choices of LLMs, each with 1,500 contextual variants. Three widely used LLMs are evaluated with multilingual prompts to select from predefined behavioral options ordered by aggressiveness. An Ordered Logit Model quantifies how contextual factors influence decisions, complemented by thematic analysis to reveal underlying reasoning tendencies. Results show that LLM decisions reflect a mix of model characteristics, linguistic framing, and scenario context. Across conditions, models remain sensitive to rider urgency, traffic complexity, and road-user types. GPT is more conservative, while DeepSeek and LLaMA act more assertively, especially in vehicle interactions. Prompt language also matters. Chinese and French prompts are associated with more assertive behavior than English, with French strongest. Across scenarios, all models shift toward more protective behavior when vulnerable road users appear, reducing aggressiveness and prioritizing safety and smoother flow. These findings characterize model-level behavioral priors relevant to LLM choice and prompt design in AV applications.