Q-Shaped Options for Hierarchical Reinforcement Learning
Q-Shaped Options (QSO) is introduced, which builds on an architecture with distinct state-value functions, Q functions and policies at each level of the hierarchy, and characterise three desiderata for an action abstraction.