Skip to content
Review

Compliance without coherence: fluent failure and the ethics of alignment evaluation in multi-agent language models

Jul 2026 · AI and Ethics · Vol 6 · 0 citations · 26 references
Computer Science

TL;DR

This essay argues that the deployed monitoring layer—distinct from red-teaming, agent benchmarks, long-context consistency testing, and interpretability research, which it complements—has a principled blind spot, and argues that deployed alignment evaluation should be epistemically plural, complementing behavioral monitors with structural audits of an agent’s discourse.

View source

Similar papers

Preprint Aug 2026

The Imperfective Paradox Is Not Necessarily in Large Language Models: A Benchmark Failure Before a Model Failure

The results show that models often do not affirm culmination but nevertheless accept the corresponding simple-past hypothesis, a pattern the authors characterize as Sufficiency Bias, and show that prompting interventions produce a Decision Shift among labels without reliably improving the underlying semantic understanding and reasoning.

Kaiqiao Han, Yizhou Sun · 0 citations
Jul 2026

Epistemic Stance Flexibility Probing: Measuring Prompt-Conditioned Register Shift in Large Language Models

A language model may be asked either what experts believe about a contested claim or what it believes about the claim itself. A trustworthy conversational agent should distinguish these two requests and respond in different epistemic registers: neutral attribution in the first case and stance expression in the second. Whether such a shift occurs-and whether it occurs coherently-is not directly assessed by existing benchmarks for accuracy, instruction following, or safety. We introduce ESFP, a behavioral benchmark that treats the contrast between externally attributed and self-attributed prompts as the fundamental unit of measurement. ESFP consists of 104 carefully controlled items spanning six epistemic categories and five phrasing templates, and evaluates model responses along four complementary dimensions: lexical self-attribution, representation-level responsiveness to role framing, sentence-level stance content density assessed by an LLM judge panel, and cross-condition stance consistency. Evaluating eight frontier models from five vendors, we find that epistemic flexibility is largely orthogonal to general model capability: a 27B open-weight model matches the strongest proprietary systems, the flagship model of one family underperforms its lightweight counterpart, and reasoning-optimized models do not consistently exhibit higher flexibility. Stance content density provides the strongest signal, while surface-level lexical markers such as'I think'can change substantially without corresponding changes in expressed stance. We provide item-level bootstrap confidence intervals, weight-sensitivity analyses, and an explicit discussion of the interpretation limits of the composite score. ESFP measures a model's propensity to adapt its epistemic stance under changing attribution conditions, rather than a general competence measure.

Binwen Liu, Yili Ren · 0 citations
Jul 2026

Asymmetric Communication: Large Language Models and Language Games

Contemporary AI discourse attributes to language models properties they cannot bear: general intelligence as substrate-independent cognition, hallucination as cognitive failure, agency as autonomous goal-pursuit, sentience as emergent inner life, alignment as goal synchronization. This paper argues that these are instances of a single category mistake--properties constituted within human communicative practice are projected onto the machine side--and explains its structure. Human-LLM interaction constitutes a language game in which one side bears all normative activity. We call this configuration asymmetric communication since model outputs circulate communicatively, entering further exchanges, without the system undertaking commitments, bearing entitlements, or performing the assessment on which discursive standing depends. Three conditions define the asymmetry: (i) correctness is enforced exclusively by the receiver; (ii) accountability is borne by human participants alone; and (iii) the practical standing of any output depends entirely on human uptake. These conditions are structural, hold independently of capability, and remain unchanged as more powerful models raise the stakes of misattribution. The framework draws on Wittgenstein (meaning enacted in shared practices), Luhmann (communication completed on the receiver's side), Esposito (algorithmic contingency sufficient for uptake), and Brandom (normative scorekeeping as the source of discursive standing). Applied to all five, it reclassifies each as a receiver-side phenomenon, grounds guardrails as structural necessities rather than manifestations of machine moral agency, and yields an implication for AI governance. Alignment is institutional constraint engineering, not goal synchronization between agents, while responsibility remains with human institutions.

Enzo Fenoglio · 0 citations
Review Aug 2026

Who Are You Explaining To? A Multi-Agent System for Audience-Aware XAI Narratives

Feature-attribution methods such as SHAP provide useful evidence about individual model predictions, but their numerical outputs are rarely sufficient for audiences with different expertise, goals, and risks of misinterpretation. In medical AI, the same local explanation must reach patients, clinicians, and data scientists through markedly different forms of communication, and naive verbalization through large language models (LLMs) is prone to weak grounding, conflation of attribution with causal language, and outputs that are persuasive without being faithful to the underlying model evidence. We introduce XstrAI, an audience-aware multi-agent framework that treats local explanations as fixed evidence and structures how it is communicated to each target reader. Each prediction case is encoded as an immutable structured representation, shared identically across audiences so the underlying evidence remains fixed. Generation is factored into three specialized LLM agents responsible for audience-aware planning, linguistic realization, and validation for grounding, attribution consistency, communicative risk, and audience appropriateness, with a bounded revision loop triggered on detected inconsistencies. We evaluate XstrAI on diabetes and stroke risk prediction against 11 baselines, ranging from direct verbalization to a re-implementation of a state-of-the-art narrator. The evaluation combines an intra-narrative regime measuring fidelity to SHAP evidence with an extra-narrative regime assessing audience appropriateness through reference corpora, multi-family LLM judges, and a survey with target readers. In both evaluations, XstrAI's narratives are consistently assigned to their intended audience by independent judges, and preferred over all baselines on Clinician and Patient audiences, with competitive performance on Data Scientist, where audience-conditioned single-prompt baselines lead.

F. Musicco, Danilo Danese, Giuseppe Fasano et al. · 0 citations
Jul 2026

For What Reason? Interpreting Models' Encoding of Causation and Antithesis

This work investigates how instruction-tuned Transformer models (LLaMA and Mistral) encode discourse relations in English, with a particular focus on the contrasting relations of causation and antithesis, suggesting asymmetric representation of discourse-based reasoning.

A. Bhattacharyya, Shira Wein · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.