Collaborate and explain on-the-fly: knowledge-based reasoning and learning in ad hoc teamwork

Abstract

Ad hoc teamwork requires an agent to collaborate with previously unknown teammates (human, AI) without prior coordination. State of the art methods address this primar- ily as a learning problem, using large datasets of prior observations to model teammate behavior and determine the actions of the ad hoc agent. Such data and the computa- tional resources required are often unavailable in complex domains, and the resulting models are opaque and difficult to revise as the domain, team composition, or agents’ capabilities change. Motivated by these challenges, this thesis introduces KAT, an architecture that inte- grates knowledge-based reasoning and data-driven learning, enabling an ad hoc agent to reason, adapt, and explain its decisions using substantially fewer resources than purely data-driven methods. Given a goal, the ad hoc agent determines its actions through non-monotonic logical reasoning with prior domain-specific commonsense knowledge, models learned and revised rapidly to predict the behavior of other agents, and future tasks anticipated using generic knowledge of similar situations in a pre- trained Large Language Model. The agent further uses a combination of a pretrained Large Language Model and decision-tree induction to incrementally acquire and revise knowledge, in the form of objects, actions, and axioms governing domain dynamics, from natural language descriptions and observations of other agents’ behavior; and generates relational descriptions as on-demand explanations of its own decisions and beliefs, and those of other agents, in response to different types of questions. We ground and experimentally evaluate KAT in a variety of environments with different levels of complexity, demonstrating reliable, efficient, transparent, and scal- able performance: a substantial improvement over purely knowledge-based baselines, and comparable or better performance than purely data-driven baselines, while using orders of magnitude fewer resources.