The extent to which an agentic framework can optimize and run an HPC scaling study with a low latency network in Amazon Web Services, accurately transform HPC job specifications between workload managers, and design and run an entire biosciences workflow is assessed.
Abstract
The means to execute and orchestrate software components has changed from human-written code to descriptive prose. In high performance computing, this transition is represented in application orchestration, workload management, and system monitoring and debugging, to name a few. The underlying means to enable descriptive definition of tasks is the use of the Large Language Model with associated tool functions and resources. A combination of a model with access to such resources, modeled in software, encompasses an autonomous framework. As fully automated and agentic frameworks are developed for science, it is important to assess reliability and strategies scoped to specific tasks. In this work, we assess the extent to which an agentic framework can optimize and run an HPC scaling study with a low latency network in Amazon Web Services, accurately transform HPC job specifications between workload managers, and design and run an entire biosciences workflow. We find that the framework completes all three tasks while surfacing task-specific failure modes. In the scaling study, agents deploy and optimize applications but monitor running jobs inefficiently, preferring conservative fixed waits over event subscriptions. In job translation, they convert specifications between Slurm and Flux with high accuracy, with processor-affinity flags the most common error. In the bioscience workflow, the agent reproduces an expert-written variant-calling pipeline almost exactly -- agreeing with the reference call set in 18 of 19 completed runs -- and reaches this result through many distinct yet functionally equivalent workflow implementations. This information is invaluable moving forward to developing multi-cluster setups with scheduling and transformation handled by agents.
The emergence of modern agents powered by large language models has created a demand for executing long-horizon, autonomous workflows in various domains that require significant computational resources. While High Performance Computing clusters provide the ideal infrastructure for these computation-intensive workloads, traditional HPC job schedulers such as Slurm are not designed for dynamic, agentic workflows characterized by unpredictable task durations, external API calls, and fault tolerance requirements of modern agents. This work presents RASER, a user-space framework that enables seamless execution of agentic workflows on production HPC clusters by extending Slurm's internal primitives. RASER introduces agentic job arrays with work stealing via shared filesystem queues, user-space checkpointing through application-level state serialization combined with Slurm requeue, and Apptainer container-based isolation without requiring any image modifications. Evaluations demonstrate that RASER reduces makespan by nearly 39% compared to static partitioning while achieving near-full CPU utilization. RASER provides resilience against preemption and failures while maintaining minimal checkpoint/restore overhead. It requires no kernel privileges or external database infrastructure, making it an accessible solution for deploying agentic workflows on existing HPC infrastructure.
This paper presents a hierarchical, dynamic architecture and software to discover resources across diverse cloud, edge, and HPC systems and exemplifies the importance of careful coordination between agents, discovery tools, and infrastructure for agentic science.
An integrated management framework powered by Tapis is shown how a unified UI-driven workflow streamlines the transition from initial evaluation to deployment, ensuring operational consistency and reproducibility without manual script porting.
Manikya Swathi Vallabhajosyula, Gautam Gururaj Molakalmuru, Samuel Khuvis et al.· Practice and Experience in A...· 0 citations
This paper presents a systematic, practice-driven evaluation of WebAssembly (WASM) as an execution substrate for cloud-native workloads orchestrated through Docker and Kubernetes using runwasi. We develop a reproducible workflow that compiles Rust and Go/TinyGo applications to WASM modules, applies Ahead-of-Time (AOT) optimizations per engine and architecture, and packages minimal OCI artifacts for multi-architecture deployment (amd64, arm64, riscv64). Through controlled experiments on heterogeneous nodes, we benchmark eight CPU-bound workloads, reporting mean, median, and confidence intervals to ensure statistical robustness. The results show that WASM achieves latencies close to native on amd64/arm64, with WasmEdge leading under AOT and Wasmtime offering strong compatibility and stability. Artifact sizes are substantially smaller than traditional containers, improving pull times and packing density. While riscv64 execution is feasible, it remains less mature and more sensitive to engine and build choices. Overall, the findings demonstrate that WASM is a viable and efficient option for Cloud Continuum environments when deployments rely on AOT-optimized artifacts, RuntimeClass-based scheduling, and workloads aligned with the WASI feature set supported by the selected engines.
Álvaro Vázquez-Rodríguez, David Vila-Pérez, Carlos Giraldo-Rodíguez et al.· International Conference on...· 0 citations
In HPC system management, the transition to modular architectures like OpenCHAMI requires standardization to ensure long-term interoperability. However, the consensus-building process required by open-source projects often operates on a different timeline than the deadline-driven operational needs of site administrators. This creates a synchronization challenge: teams must deliver functional software for immediate machine bring-up while ensuring alignment with future community standards. This paper details how the OpenCHAMI project utilized Fabrica, a custom code generator, to bridge this gap during the development of a Field Replaceable Unit Inventory Service. Fabrica decouples the API’s structural standards from its application logic, allowing the team to generate working prototypes for immediate testing at Los Alamos National Laboratory while retaining the flexibility to regenerate the service to match pending architectural decisions. We demonstrate how this approach enables parallel workflows and discuss its implications for improving Research Software Engineering productivity and facilitating AI-assisted development.
B. McDonald, Alex Lovell-Troy· Practice and Experience in A...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
