An Improved Deep Reinforcement Learning Routing Algorithm in LEO Networks
In recent years, the satellite network has shown significant development, especially in LEO orbit. This development is shown with the launch of satellites in massive numbers, such as Starlink-SpaceX, Iridium, OneWeb, and Kuiper. Characteristics changing LEO satellites in dynamic and different conditions, making satellites a challenge in realizing the objective of developing LEO satellites. The goal is to serve internet needs, navigation, research, and defense, where every objective has different delays, QoS, and load balance issues. Various studies on satellite routing design have introduced methods, such as using the shortest path, minimum hop, RL, and DRL methods, to overcome issues. In particular, MQRP, RL, and QLRA algorithms are capable of well defining various attributes or criteria, such as delay, multi-QoS parameters, and load balance, but the algorithms experience constraints in the face of fast and deep large-scale dynamics and finding the shortest path. To address this challenge, an Improved DRL routing algorithm that uses the BFS and MADM methods is proposed. The BFS method can find the shortest path with minimum hops, combined with the ability of adaptation and scalability from DRL and multi-attribute weighting in taking the decision of next hop selection from MADM. Based on the results of the simulation, it clearly shows that the Improved DRL outperforms DRL. Improved DRL has a gap between DRL regarding end-to-end delay, convergence time, number of hops, and packet loss, which are approximately $500 ~\text{ms}, 1500 ~\text{ms}, 85$ hops, and 40%.