Book
Open access
Jun 2026
STREAM: Multi-Tier LLM Inference Middleware with Dual-Channel HPC Token Streaming
STREAM (Smart Tiered Routing Engine for AI Models) addresses the gap between local, HPC, and cloud inference with a three-tier routing architecture combining local, HPC, and cloud inference with a local LLM-based complexity judge.
A. Nassar, Steve Mohr, L. Apanasevich et al.
· Practice and Experience in A... · 0 citations