Aug 2026· Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication· pp. 2265-2267· 0 citations· 12 references
Computer Science
TL;DR
Weaver is a low-overhead cross-layer diagnosis framework for GPU kernels that builds a semantic execution graph from operator, kernel timeline, and warp/block evidence, distinguishes blocked and slowed kernels, and reports an interpretable root-cause chain.
Abstract
Modern GPU workloads use batching, asynchronous execution, kernel overlap, and GPU sharing to improve utilization, but these optimizations make kernel-level slowdown hard to diagnose. A target kernel may be blocked by extra synchronization or helper kernels, or slowed by concurrent kernels after it starts. Existing tools provide GPU visibility, but they either remain too heavy for continuous online use or focus on coarse-grained symptoms, leaving fine-grained kernel-level root causes to manual analysis. This poster presents Weaver, a low-overhead cross-layer diagnosis framework for GPU kernels. Weaver builds a semantic execution graph from operator, kernel timeline, and warp/block evidence, distinguishes blocked and slowed kernels, and reports an interpretable root-cause chain. Our prototype evaluation shows that Weaver continuously collects runtime evidence with low overhead and accurately localizes anomalies.
Ghost is an OS-level GPU virtualization layer integrated directly into the open-source GPU driver, using a GPU container abstraction with cgroup -like APIs for compute and memory control and privileged hardware-level scheduling and preemption for dynamic compute resource management.
In modern GPU-based non-unified heterogeneous systems, CPU-GPU communication happens via the PCI bus. Data transfers are affected by startup overhead, which underutilizes the PCI channel bandwidth for small transfers. Modern programming models, such as CUDA and OpenMP, treat each input argument to a compute kernel independently, leading to data movement segmentation and execution slowdowns. This work presents In-Copy Fusion (ICF), a runtime optimization of OpenMP’s accelerator model, implemented in LLVM, that gathers eligible map clauses into staging buffers, orchestrating an optimal transfer pipeline through fusion without modifying application or kernel code. The technique preserves OpenMP semantics with negligible overhead. We evaluate ICF across a range of representative HPC benchmarks and configurations, including varying argument counts, data sizes, and argument-size disparities, as well as real-world benchmarks. The results show that ICF improves effective host-to-device (H2D) bandwidth and reduces end-to-end time relative to the baseline runtime with per-argument transfers across platforms and workloads, achieving an up to 4.8 × speedup.
Dionisis-Odysseas Sotiropoulos, Sara Royuela Alcázar, Eduardo Quiñones et al.· Proceedings of the Internati...· 0 citations
This work categorizes LLM kernel operands into three inter-workgroup sharing patterns and shows that the required optimization strategies differ across categories, from simple per-workgroup pinning to subgroup-aware co-scheduling, highlighting the need for placement-aware kernel programming and smarter architectural support for work and data locality in multi-partition GPUs.
Donghyeon Joo, Sooraj Puthoor, N. Jayasena et al.· arXiv.org· 1 citation
This paper presents a zero-overhead, multi-vendor GPU compilation framework built natively into the Rust compiler (rustc) and LLVM backends, and leverages Rust's rich type system, ownership system, and strict aliasing guarantees to efficiently manage and optimize data transfers through LLVM's Offload infrastructure.
Manuel S. Drehwald, Marcelo Domínguez, Kevin Sala et al.· 0 citations
Modern AI systems depend on specialized accelerator kernels, whose development is complicated by increasingly diverse operators and hardware. LLMs and agentic systems promise to automate this work, but existing evaluations do not show whether their performance transfers across operator sources and hardware platforms, or what such transfer costs. We present KernelGenBench, the first unified multi-source and multi-chip infrastructure for evaluating LLM- and agent-generated Triton kernels. With a common Triton target spanning six hardware platforms, it provides the broadest cross-vendor hardware coverage among existing kernel-generation benchmarks. We report two controlled analytical views: KernelGenBench-MS (Multi-Source) covers 210 operators from PyTorch ATen, production vLLM operators, and proprietary cuBLAS routines, while KernelGenBench-MC (Multi-Chip) evaluates a semantically stable 110-operator subset across six hardware platforms. Our evaluation consumed over 15 billion tokens. Agentic execution improved correctness, but no method dominated across sources and platforms: vLLM posed the strongest correctness challenge, cuBLAS set the highest performance ceiling, and AutoKernel accuracy fell from 87% on NVIDIA to 25% on Iluvatar CoreX. These improvements were costly: specialized agents averaged 4.99 million tokens per successful operator, rising to 6.25 million for CUDA Optimized Skill. The results establish operator source, hardware platform, and agentic scaffold as distinct dimensions of kernel-generation capability, and show that success in a familiar source-hardware setting is not a reliable proxy for deployment readiness.
Pei-Yu Zang, Jian-Hang Tao, Jia-Ling Zhang et al.· 0 citations
A per-layer, contention-aware CPU+GPU row split for transformer prefill built on a fix for MLX's lazy-graph scheduler that accelerates Llama-shaped decoder-block prefill and saves time-to-first-token on a real Qwen2.5-7B checkpoint.
Om Mohite· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
