Deep Graph Reinforcement Learning with Hierarchical Policy Learning

Abstract

This paper presents a novel approach to Deep Graph Reinforcement Learning (DGRL) that leverages hierarchical policy learning to enhance both sample efficiency and learning speed. Traditional DGRL methods often suffer from slow exploration and high computational demands, particularly when dealing with large, complex graphs. Our proposed framework addresses these challenges by decomposing the reinforcement learning problem into a hierarchy of policies. Higher-level policies define abstract goals, while lower-level policies execute actions to achieve those goals. This hierarchical structure facilitates more focused exploration, reduces the search space, and ultimately accelerates convergence. We demonstrate the effectiveness of this approach through theoretical analysis and a comprehensive evaluation. The core claim of this work is that hierarchical policy learning can significantly improve the sample efficiency and learning speed of deep graph reinforcement learning algorithms. The key mechanism is the decomposition of the learning problem into a hierarchy of policies, allowing for more efficient exploration and faster convergence.