Skip to content
Log in
Sponsored AI AM CLAUDE, A Beginner's Guide: Learn the basics of working with AI AI AM CLAUDE, A Beginner's Guide: Learn the basics of working with AI

OpenTela: Unifying Decentralized Computing Resources for Heterogeneous LLM Serving (Operational Systems)

Xiaozhe Yao Youhe Jiang Ilia Badanin Epfl Qinghao H. Mit Binhang Yuan Eiko Yoneki A. Klimovic Eth Zurich
· 0 citations · 72 references

TL;DR

OpenTela is presented, a user-space orchestration overlay that turns existing fragmented HPC clusters into a unified, cross-institutional serving platform and provides a replicable blueprint for other sovereign AI initiatives to harness their own federated GPU infrastructure.

Semantic Scholar
View source

Similar papers

Preprint Aug 2026

FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution

Frontier open-weight models are increasingly available, but serving them still largely assumes datacenter infrastructure. We present FreeToken, an edge-native MoE serving system that treats a personal machine not as a small GPU, but as a unified, elastic inference platform. FreeToken co-designs the full serving stack,...

Shuo Yang, Xiao-yun Fan, Melissa Z. Pan et al. · 3 citations
Book Open access Jun 2026

STREAM: Multi-Tier LLM Inference Middleware with Dual-Channel HPC Token Streaming

STREAM (Smart Tiered Routing Engine for AI Models) addresses the gap between local, HPC, and cloud inference with a three-tier routing architecture combining local, HPC, and cloud inference with a local LLM-based complexity judge.

A. Nassar, Steve Mohr, L. Apanasevich et al. · 0 citations
Open access Aug 2026

CELLServe: An SLO-Aware and Cost Efficient LLMs Serving System for Serverless Computing Environments

CELLServe formalizes SLO-constrained joint resource provisioning as an optimization problem with a dedicated algorithm, and introduces an opportunistic instance merging strategy for decode phase functions to reclaim fragmented resources.

Zejian Wang, Nan Lin, Zinuo Cai et al. · 0 citations
Preprint Aug 2026

OpScale: Operator-level Provisioning and Autoscaling for LLM Serving

OpScale is presented, a practical operator-level orchestration framework of profiling, provisioning, placement, and runtime serving that attains SLOs with up to 36.3% fewer GPUs and 28% less power, or achieves 44% higher throughput under fixed cost budgets.

Xingqi Cui, Chieh-Jan Mike Liang, Ziang T. Tang et al. · 0 citations
Preprint Aug 2026

Multi-tenant Kubernetes Use Cases for AI, Secure Computing and Data Services, and More

This work evaluates Kubernetes deployment on a Hewlett Packard Enterprise (HPE) Cray EX supercomputer with HPE Slingshot interconnect, called Isambard-AI, with co-design use cases that combines KubeRay, Ray, and vLLM to provide a distributed, sandboxed, persistent AI model hosting service targeting multi-tenant confide...

J. Watson, S. Alam, Christopher J. Woods et al. · 0 citations
Conference Jul 2026

Two-level Network Bandwidth Allocation for Multi-tenant AI Clouds in Kubernetes

Compute and memory resources in cloud environments are strictly managed and isolated by the control plane; in contrast, network resources lack equivalent management and isolation mechanisms. This best-effort treatment of networking leads to significant challenges for modern AI workloads, which have diverse and bandwidt...

Sunyanan Choochotkaew, Lionel Jouin, Tatsuhiro Chiba · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.