AI Networking Cookbook: Practical recipes for AI-assisted network automation and development
TL;DR
FitFloat is presented, a drop-in floating-point array replacement supporting user-specified precision on GPUs with the goal of reducing storage requirements of scientific applications while maximizing performance over Unified Memory.
Valk, a performance analysis tool that combines data from multiple profilers, shows that when memory bandwidth is increased, kernels become compute bound, and makes three recommendations to fully utilize the GPUs' potential for relational workloads when the memory wall is removed.
One of the main reasons compute hardware becomes obsolete is software bloat: resource requirements increase for every iteration of a software product. Resource constrained VMs are one way to combat software bloat as they post a hard limit on the resources and so force the programmer to be frugal. In this paper we explore the deployment of one such resource constrained VM, Uxn, on GPU. We show that for competitive performance it is essential to make use of the GPU data parallelism. We present an OpenMP-style parallelism API for Uxntal, the stack-based assembly-style language for the Uxn platform. We demonstrate that exemplar code using our API can run at comparable performance even on an integrated GPU. Specifically, our evaluation results show that using this approach improves performance on the compute-intensive Stencil benchmark with 19x and frame rate on the graphics-intensive Bunnymark benchmark with 7x. In practice, all laptops and desktops and even mobile devices have a GPU and our work shows that they can be used to execute frugal workloads effectively.
We study unstable integer sorting on GPUs for arrays whose elements lie in a known integer range. Focusing on counting-sort-based methods that determine the output interval of each value from its frequency and the prefix sums of the frequencies, we propose and evaluate Range-Tiled CDF sort (RT-CDF), which partitions the possible value range into small intervals, called tiles, that fit in shared memory. For each tile, RT-CDF constructs a histogram, computes its prefix sum as a local CDF, and directly generates the output array from the local CDF. We compare RT-CDF against three baselines: CUB DeviceRadixSort, whose processed bit range is restricted to $[0,\lceil\log_2 R\rceil)$ to exploit the known range size $R$; Ref-H-P sort; and an implementation based on the algorithm of Kolonias et al. Experiments on an NVIDIA GeForce RTX 4090 with range sizes from $R=2^7$ to $2^{18}$, input sizes from $n=10^6$ to $10^9$, and uniformly distributed, normally distributed, and all-equal inputs show that RT-CDF outperforms the baselines over a broad set of conditions for small to medium ranges, achieving a maximum speedup of 4.39 over the fastest baseline. For $R=2^{18}$, however, at least one baseline outperforms RT-CDF for every evaluated input size and input distribution, showing that the cost of histogram construction limits the applicability of RT-CDF to larger ranges.
Arbitrary-precision floating-point arithmetic is used to solve ill-conditioned problems in scientific computing, and MPFR and MPC have become the de facto standard libraries for CPU-based computations. However, no publicly available environment provides MPFR/MPC functionality on GPUs. In this study, we developed mpc\_cuda, an arbitrary-precision arithmetic environment for GPUs, by leveraging generative AI (Claude Code) and rerunnable conversion scripts to faithfully port the mini-GMP-based MPFR/MPC libraries to CUDA kernels while reserving a per-thread temporary storage region (a bump arena). This approach accelerates real and complex elementary functions and basic linear algebra on GPUs by approximately 40--105$\times$, while remaining bit-identical to the host. In addition, we implemented compile-time fixed-precision types cu\_freal/cu\_fcomplex, and demonstrate, through benchmark tests involving the GMRES($m$) method and algebraic equation solving on GB10 and H100 GPUs, that the proposed implementation outperforms the existing GPU libraries CUMP and CAMPARY at low-to-medium precision.
Building on NCCL's device-side API, low-latency interfaces for constructing custom collective kernels are developed and used to implement new symmetric collectives in NCCL, demonstrating benefits for both AI inference and traditional HPC workloads.
This work proposes SAI, a mechanism that virtualizes shared memory into the L2 cache to improve GPU performance for AI applications and introduces an L2 cache management strategy that integrates associativity-based virtual page allocation and a replacement information table, reducing page-swapping overhead while preserving L2 cache performance.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.