Relational Reinforcement Learning with Graph-Based Reward Shaping

Abstract

This paper proposes a novel approach to reinforcement learning (RL) that leverages relational reasoning through the incorporation of graph-based reward shaping. Traditional RL algorithms often struggle in environments exhibiting complex relationships between entities, leading to inefficient learning and suboptimal policies. Our method addresses this limitation by enabling the agent to explicitly learn and represent these relationships, utilizing a graph structure to encode contextual information. The core idea is to shape the reward function based on the agent's understanding of these relationships, guiding it towards more effective exploration and exploitation. This approach is demonstrated through a theoretical framework outlining the algorithm and key formulas, along with a conceptual explanation of its implementation. The potential for enhanced learning efficiency and policy quality in environments with inherent relational complexity is highlighted. The algorithm's performance is expected to improve by considering the interactions between entities within an environment, rather than solely focusing on individual state-action pairs.