In modern GPU-based non-unified heterogeneous systems, CPU-GPU communication happens via the PCI bus. Data transfers are affected by startup overhead, which underutilizes the PCI channel bandwidth for small transfers. Modern programming models, such as CUDA and OpenMP, treat each input argument to a compute kernel independently, leading to data movement segmentation and execution slowdowns. This work presents In-Copy Fusion (ICF), a runtime optimization of OpenMP’s accelerator model, implemented in LLVM, that gathers eligible map clauses into staging buffers, orchestrating an optimal transfer pipeline through fusion without modifying application or kernel code. The technique preserves OpenMP semantics with negligible overhead. We evaluate ICF across a range of representative HPC benchmarks and configurations, including varying argument counts, data sizes, and argument-size disparities, as well as real-world benchmarks. The results show that ICF improves effective host-to-device (H2D) bandwidth and reduces end-to-end time relative to the baseline runtime with per-argument transfers across platforms and workloads, achieving an up to 4.8 × speedup.
Dionisis-Odysseas Sotiropoulos, Sara Royuela Alcázar, Eduardo Quiñones et al.· Proceedings of the Internati...· 0 citations
On-device model adaptation is essential to enable lifelong personalization on resource-constrained hardware, but compute, power, and memory limitations of such devices make end-to-end backpropagation impractical for modern deep neural networks. This work proposes a heterogeneous adaptation pipeline that repurposes a commercial edge AI inference accelerator, Hailo-8L, for frozen-backbone feature extraction during on-device training. The computational graph is partitioned so that the pre-trained backbone is quantized to INT8 and run on the accelerator, while only a lightweight FP32 classification head is fine-tuned on the host CPU, enabling frequent, energy-efficient in-field updates with most weights remaining fixed. Across multiple architectures and datasets, this pipeline achieves up to 15.4x faster wall-clock training time compared to a Raspberry Pi 5 CPU baseline, offers competitive throughput in favorable settings, and consistently reduces energy per sample. Post-training quantization restoration is shown to be crucial for preserving the quality of accelerator-generated features and mitigating accuracy loss in quantization-sensitive architectures. Overall, the results demonstrate a practical approach to efficient on-device adaptation using inference-oriented edge accelerators. The implementation is available at https://github.com/MatPiech/accelerator-training.
M. Piechocki, Alessandro Capotondi, Marek Kraft· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
