Multi-Objective Reinforcement Learning with Pareto Dominance-Based Exploration

Abstract

Multi-objective reinforcement learning (MORL) presents significant challenges due to the difficulty in balancing multiple, often conflicting, objectives. Traditional exploration methods often fail to adequately explore the complex multi-objective action space, leading to suboptimal solutions. This paper proposes a novel exploration strategy for MORL that leverages Pareto dominance to efficiently navigate this space. The core idea is to guide the agent's exploration towards regions where it can achieve better trade-offs between objectives, as defined by Pareto dominance. We formulate the exploration process using a dominance-based ranking of solutions, allowing the agent to systematically identify and target superior solutions. This approach aims to mitigate the "curse of dimensionality" inherent in MORL and improve the convergence towards optimal multi-objective policies. The theoretical framework outlines the key concepts and provides a basis for further research in this area.