AI and HPC infrastructure increasingly serves workload portfolios that combine dense tensor computation, sparse kernels, large memory footprints, and communication-intensive collectives. Supporting these portfolios requires coordinated choices across accelerators, memory tiers, scale-up fabrics, and cluster networks. The resulting Cross-layer Heterogeneous System (XHS) design space is difficult to explore: hardware choices change legal task mappings, while rack power, switch radix, cabling, and cost constraints invalidate many candidates. We present CHASE, an application-driven framework that searches physically feasible XHS architectures through the workloads they must execute. CHASE represents candidates as hierarchical typed graphs and rejects designs that violate deployment constraints. It avoids intractable joint hardware-mapping search with a decoupled two-level loop: an inner mapper translates hardware-independent workload DAGs into topology-aware event traces, a calibrated event-driven simulator evaluates each mapping, and an outer telemetry-guided optimizer evolves the hardware graph. We evaluate CHASE on sparse-computing and LLM workloads. Its mapper remains within 6.06% of exhaustive optima while reducing mapping time by 60.5% on average relative to PEFT. Compute-model errors average 4.4-7.5%, and communication validation reproduces key trends across physical platforms. The outer search reaches near-global optima within 64 iterations. End-to-end case studies show that sparse workloads favor criticality-aware heterogeneous pods, whereas LLM inference favors scale-up islands; the resulting designs deliver 6.20$\times$ and 2.12$\times$ geomean speedups, respectively, while reducing cost and power relative to the baselines.
Yuchen Fan, Minghong Sun, Jikui Ma et al.· arXiv.org· 0 citations
H3-Attn is proposed, an Attention-efficient 3D DRAM PNM processor for low-batch LLM inference that features a hybrid head parallelism for Attention processing, whereby various optimized Attention mechanisms with spatial tiled FlashAttention can be flexibly enabled with fully leveraged 3D DRAM PNM bandwidth.
Yaolei Li, Wenbin Jia, Zhanchen Zhao et al.· International Symposium on L...· 0 citations
Diffusion models have shown marked advancements in 2-D generation and have also become focal points in 3-D generation via consistent multiview image generation. However, the computation and memory demands hinder their real-time deployment on mobile and edge devices. Moreover, the reduction of diffusion timesteps leads to a decrease in interstep similarity, thereby making previous methods ineffective in optimizing computation. The varied layer quantization sensitivity and optimal format in diffusion models also present challenges for traditional quantization methods to achieve efficient memory compression. To address these issues, we first introduce <inline-formula> <tex-math notation="LaTeX">${M}^{3}$ </tex-math></inline-formula> quantization, a mixed-precision, mixed-format, and mixed-granularity quantization framework that allocates optimal precision and format to different data with a unified FP8 computation flow for efficient implementation. Building upon <inline-formula> <tex-math notation="LaTeX">${M}^{3}$ </tex-math></inline-formula> quantization, we present TriM-D, a hardware accelerator designed to optimize computation. It features sparsity-aware dual-branched FP8-MAC units that introduce a dual-branch path and an efficient approximate adder to fully utilize bit-level sparsity of <inline-formula> <tex-math notation="LaTeX">${M}^{3}$ </tex-math></inline-formula>-quantized data. Such an FP8-MAC scheme preserves both efficiency and numerical accuracy. Our experiments demonstrate that TriM-D significantly outperforms the state-of-the-art (SOTA) diffusion accelerator Cambricon-D (Camb-D), achieving an average reduction of 56.5% in memory access and 80.6% in computation cost. In addition, TriM-D provides a <inline-formula> <tex-math notation="LaTeX">$55\times $ </tex-math></inline-formula>, <inline-formula> <tex-math notation="LaTeX">$6.72\times $ </tex-math></inline-formula> improvement in energy efficiency and <inline-formula> <tex-math notation="LaTeX">$3.69\times $ </tex-math></inline-formula>, <inline-formula> <tex-math notation="LaTeX">$2.67\times $ </tex-math></inline-formula> speedups over NVIDIA A100 and Camb-D.
Wenxun Wang, Li-Kai Ma, Chen Tang et al.· IEEE Transactions on Compute...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
