AHRL-PM: Asynchronous Hierarchical Reinforcement Learning Framework for Enhanced Portfolio Management.

Abstract

Effective portfolio management (PM) is a cornerstone of financial strategy, yet it is often challenged by the uncertainties and the high dimensionality of market data. Traditional PM techniques, whether model-based or model-free, frequently fail to address these complexities, resulting in suboptimal asset allocation. This study introduces an asynchronous hierarchical reinforcement learning (AHRL-PM) framework aimed at enhancing active PM. The proposed AHRL-PM approach employs a two-tiered agent system to navigate market complexities. The first-layer agent selects top-performing stocks monthly based on alpha factors, while the second-layer agent adjusts asset weights using market fundamentals and technical indicators, dynamically rebalancing the portfolio in response to market shifts on a daily basis. We rigorously evaluated our AHRL-PM framework through extensive experiments in six globally representative markets. Performance was assessed using two key metrics: annualized return (AR) and Sharpe ratio (SR). Our model consistently outperformed traditional benchmarks on both metrics across all the markets, demonstrating robustness and versatility in PM. Furthermore, ablation studies comparing the AHRL-PM model to a synchronous model and single-layer reinforcement learning (RL) approaches validated its superior performance in terms of profitability and risk-adjusted returns, while also highlighting significant reductions in training time and appropriate portfolio weight adjustments to respond to market dynamics and uncertainties, underscoring the model's efficiency and practical applicability.