Skip to content
Log in

Author

Engelbert Delgado

View on Semantic Scholar

2 papers indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

#large language models Open access Aug 2026

ToshLLM: local LLM inference on Intel Macs with AMD GPUs

A native SwiftUI application that runs large language models locally on Intel Macs with AMD GPUs, a configuration mainstream inference stacks leave unsupported or incorrect. Beyond packaging, it contributes original work to the Metal backend of llama.cpp: ToshGEMM, a manually tiled matrix multiply that replaces the simdgroup-matrix path AMD GPUs do not provide. FA-AMD, flash-attention decode, tile and prefill kernels written for AMD, where the upstream vectorised kernel miscompiles. A wave64 port for GCN and Vega: reductions, quantized decode, batched mat-vec and prefill on 64-wide simdgroups. Multi-GPU tensor parallelism with a butterfly all-reduce and a per-batch choice between peer copies over Infinity Fabric and event hand-off. A reimplementation of TurboQuant KV cache compression and a speculative decoding planner. Patches apply on top of llama.cpp and stable-diffusion.cpp, which remain under the copyright and MIT licence of their own authors.

Engelbert Delgado · 0 citations
#large language models Open access Aug 2026

ToshLLM: local LLM inference on Intel Macs with AMD GPUs

A native SwiftUI application that runs large language models locally on Intel Macs with AMD GPUs, a configuration mainstream inference stacks leave unsupported or incorrect. Beyond packaging, it contributes original work to the Metal backend of llama.cpp: ToshGEMM, a manually tiled matrix multiply that replaces the simdgroup-matrix path AMD GPUs do not provide. FA-AMD, flash-attention decode, tile and prefill kernels written for AMD, where the upstream vectorised kernel miscompiles. A wave64 port for GCN and Vega: reductions, quantized decode, batched mat-vec and prefill on 64-wide simdgroups. Multi-GPU tensor parallelism with a butterfly all-reduce and a per-batch choice between peer copies over Infinity Fabric and event hand-off. A reimplementation of TurboQuant KV cache compression and a speculative decoding planner. Patches apply on top of llama.cpp and stable-diffusion.cpp, which remain under the copyright and MIT licence of their own authors.

Engelbert Delgado · 0 citations