PTXBench provides an auditable testbed for measuring and improving LLMs'ability to exploit evolving GPU architectures, and shows that architecture-specific PTX capability remains uneven.
Abstract
We introduce PTXBench, a benchmark for evaluating and adapting large language models (LLMs) to use architecture-specific PTX for GPU kernel optimization. PTXBench measures functional correctness, whether selected target instructions execute at runtime, and speedup over frontier libraries across GEMM and attention workloads on H100 and B200 GPUs. Our evaluation shows that architecture-specific PTX capability remains uneven: success rates fall substantially on complex attention backward workloads, and executing the target instructions does not necessarily translate into competitive performance. No evaluated model consistently matches frontier libraries across the suite. We further adapt Qwen3.6-27B using supervised fine-tuning. Repair-conditioned training improves several tasks, but generalization remains uneven; data coverage, balance, and the quality of the reasoning teacher matter in addition to dataset size. PTXBench provides an auditable testbed for measuring and improving LLMs'ability to exploit evolving GPU architectures.
As large language models (LLMs) keep growing in size and complexity, their training frameworks evolve at a rapid pace as well. Therefore, continuous integration (CI) is critical for maintaining the quality and stability of these frameworks. However, unlike traditional software, CI for LLM training frameworks relies on...
Tian-Shuo Qiao, Nai-Qian Zheng, Xiao-Peng Liu et al.· 0 citations
DataKernelBench is introduced, which translates SQL into validated PyTorch TorchPlan programs and evaluates LLMs that optimize either the core tensor-bounded snippet or the full query in CUDA or Triton through execution-guided repair and finds that higher-performing implementations commonly use kernel fusion and execut...
A hierarchical search-space planning framework for GPU kernel optimization that delivers stronger overall implementation validity, sample efficiency, and optimization performance than existing training-free methods, while remaining competitive with the training-based CUDA-L1 without additional model training is propose...
Jing-Hao Wang, Qiqi Gu, Chenpeng Wu et al.· 0 citations
Current GPU Collective Communication Libraries (CCLs) employ predefined schedules optimized for stable environments. Their supported schedules and selection logic are fixed at communicator initialization, which fails to account for evolving runtime conditions, such as workload characteristics and hardware health status...
Rui Ding, Xiandong Lu, Jiajun Wang et al.· Conference on Applications,...· 1 citation
Tile-based programming models, such as Triton and cuTile, aim to simplify high-performance kernel development, but their practical performance, tuning behavior, and usability remain difficult to compare systematically. We present TileBench, a controlled benchmark for evaluating Triton and cuTile on NVIDIA B200 GPUs und...
Bowen Cui, Zhongchun Zhou, Hao Wu et al.· 0 citations
In-Copy Fusion (ICF), a runtime optimization of OpenMP’s accelerator model, implemented in LLVM, that gathers eligible map clauses into staging buffers, orchestrating an optimal transfer pipeline through fusion without modifying application or kernel code, preserves OpenMP semantics with negligible overhead is presente...
Dionisis-Odysseas Sotiropoulos, Sara Royuela Alcázar, Eduardo Quiñones et al.· Proceedings of the Internati...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
