Risk-Aware Hierarchical Meta-Reinforcement Learning with Quantile Regression LSTM Framework for Adaptive Task Prioritization and Congestion Avoidance Offloading in Fog–Cloud Systems
TL;DR
A graph network with a reinforcement learning framework to avoid congestion and schedule tasks with a low makespan in fog–IoT systems and demonstrates the improved ability to ensure reliable and effective fog–cloud allocation in response to dynamically changing workloads.
Abstract
Fog–cloud systems allow distributed and latency-sensitive IoT applications to share edge, fog, and cloud resources for efficient computation, storing and delivering services. Nonetheless, the current methods of task scheduling and resource allocation tend to be based on deterministic prediction, heuristic schedules, or response optimization, restricting risk sensitivity and responsiveness to dynamic workloads. To address these issues, we propose a graph network with a reinforcement learning framework to avoid congestion and schedule tasks with a low makespan in fog–IoT systems. The work starts with real-time monitoring of queue states, delay, bandwidth, energy and deadline properties of upcoming IoT tasks. Quantile Regression Long Short-Term Memory (QR-LSTM) predicts a normal task flow and congestion based on the task queue. The congestion tasks are further scheduled using a Delay-Aware Influence Reinforced-Graph Neural Network (DAIR-GNN). Then, Topology-Sensitive Resource Influence Propagation is used to map and analyze the relationship between the task sender and receiver within a network. After that, Dual-Stage Risk-Aware Hierarchical Policy (DRHP) combines Proximal Policy Optimization (PPO) to select the kinds of sources, like fog or cloud, based on the topology. Similarly, Model-Agnostic Meta-Learning (MAML) with Soft Actor–Critic allocates the resources of each tasks within the selected server. Both RL models are trained using Few-Shot Adaptive Policy Transfer to make better predictions. Lastly, Confidence-Uncertainty Regulated Exploration (CURE) is used to compute the prediction score for task allocation improvement. The proposed framework achieves a variance ratio of 80.6%, latency is 0.626 s, and the success rate is 98% in the prediction of congestion, scheduling and allocation of resources. These results demonstrate the improved ability to ensure reliable and effective fog–cloud allocation in response to dynamically changing workloads.
