Author

Douglas Orr

h-index 1 View on Semantic Scholar

We have 2 of 6 papers

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

Preprint Aug 2026

Llama-Mobile: Efficient 2.7-Bit Quantization of VLMs

Deploying vision-language models (VLMs) on mobile devices is challenging due to their significant memory and compute requirements. We present a framework for quantizing VLMs for efficient inference on resource-constrained hardware. Our approach combines a quantization pipeline that uses the model itself to generate training data and does not require access to the training setup, with a novel 2.7-bit-per-parameter format supporting efficient execution on Arm CPUs. We validate our approach by compressing the Llama 3.2 11B Vision Instruct model to 3.7 GB with 8-bit activations, preserving strong performance on a set of standard visual question answering tasks.

Luka Ribar, Jeevan Bhoot, Douglas Orr · 0 citations

#natural language process... Jul 2026

Studying quantization trade-offs for efficient inference deployment in machine translation

The experiments show that the trade-off between inference efficiency and translation quality depends not only on the quantization format, but also on the choice of text chunking strategy, as well as on the choice of text chunking strategy.

Jim Zhao, Sohir Maskey, Koen Oostermeijer et al. · arXiv.org · 0 citations