Serverless computing has emerged as a promising paradigm for executing scientific workflows characterized by complex task dependencies, data-intensive operations, and high computational demands. However, most existing scheduling approaches assume homogeneous, single-region environments and primarily focus on isolated function execution. These assumptions overlook two critical challenges: (i) the inherent data-parallel nature of workflow tasks, and (ii) the heterogeneity of computing resources across geo-distributed serverless platforms. In this paper, we address these limitations by proposing a novel scheduling framework for geo-distributed serverless environments that explicitly models intra-function data parallelism, heterogeneous abstract resources, and regional concurrency constraints. We formulate a makespan minimization problem in which each function can either execute entirely on a single high-capacity resource or be partitioned across multiple heterogeneous lower-capacity resources, subject to region-specific concurrency limits.To solve this problem, we design a Deep Q-Network (DQN)-based scheduler augmented with two auxiliary heuristics. The first, Critical Workload First, prioritizes high-workload functions through an exhaustive split-deployment search over heterogeneous abstract serverless resources. The second, Load-Aware Heuristic, selects execution regions using a weighted load metric combined with penalty-based resource assignment. We evaluate our approach on five representative scientific workflows BWA, Montage, Inspiral, CyberShake, and SIPHT and using real-world round-trip time measurements from Azure Function deployments across three geo-distributed regions. Experimental results demonstrate that our DQN-based scheduler reduces makespan by up to 29.13% compared to state-of-the-art approaches.
Shubhradeep Roy, Suvarthi Sarkar, Koushik Mukka et al.· IEEE International Conferenc...· 0 citations
Serverless computing is a cost-effective, on-demand paradigm for deploying application workflows, but it faces a key challenge in efficient resource allocation. Platforms must provision resources that satisfy Service Level Objectives (SLOs) while minimizing operational cost. In real workflows, execution time depends on input size, but most approaches are input-agnostic, causing SLO violations for large inputs and overprovisioning for small ones. Recent input-aware schedulers still rely on static averages along the critical path, failing to capture dynamic, input-dependent dataflow across stages. In this work, we propose a Correlation-Aware Proportional Critical Path (C-PCP) method that employs a dynamic, input-aware weighting model derived from online profiling data. A correlation matrix captures statistical dependencies between the input and output sizes of sequential functions, enabling dynamic critical path computation, more accurate sub-SLO allocation, and improved resource profile assignment. Additionally, we develop a dynamic programming based theoretical lower bound to provide a rigorous performance benchmark. For experimental validation, workflow task profiling data is collected from execution traces on a Knative-based testbed deployed over Kubernetes, ensuring realistic modeling of workflow behavior. We evaluate the proposed approach on diverse real-world and synthetic workflows, demonstrating consistent improvements over state-of-the-art methods in both SLO satisfaction and cost efficiency.
G. V. Reddy, Chandrashekhar Dharmarajan, Suvarthi Sarkar et al.· IEEE International Conferenc...· 0 citations
Container placement in data centers must simultaneously minimize power consumption and maximize affinity preferences, while satisfying multi-resource capacity and anti-affinity constraints. Traditional approaches typically rely on fixed rules, which lack adaptability to dynamic cluster states and are difficult to extend for adaptive decision-making. On the other hand, meta-heuristic methods, although more flexible, are often computationally expensive, slower and prone to getting trapped in local optima. In this work, we propose an adaptive and efficient approach based on an LLM-driven multi-agent collaboration framework, where four specialized agents operate in a closed-loop ReAct cycle at each placement step. A Power Consumption Agent and an Affinity Agent debate over competing objectives, while a Placement Agent resolves conflicts using Tolerance-Gated Arbitration (TGA). A Rearrangement Agent further refines decisions through the Monotone Improvement Rule (MIR) in a post-placement refinement phase. All the agents reason over deterministic, feasibility-filtered candidate tables provided by the environment, ensuring that every proposed action inherently satisfies hard constraints. Based on experimental evaluation using the Google Cluster Trace with a configuration of 100 applications and 25 machines, the proposed framework consumes 8.36% less power and achieves 39.30% higher affinity than the power-greedy baseline, 7.12% less power and 10.85% higher affinity than the best domain-specific heuristic (HOP-CAPM), and 4.26% less power and 1.83% higher affinity than the meta-heuristic approach (CPAAP), averaged over three independent runs. Ablation studies confirm that each architectural component, the multi-round debate mechanism, the TGA in the Placement Agent, and the MIR in the Rearrangement Agent, contributes meaningfully to the overall performance, with removal of any single component degrading both power consumption and affinity satisfaction.
Suvarthi Sarkar, Abhishek Hazra, Akshat Mittal et al.· IEEE International Conferenc...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
