Jewon Lee

Quantize the Target, Quantize the Drafter: Efficient Inference with Qwen3.5-4B

This report describes our approach to the Efficient Qwen Competition, where the goal is to enable low-latency serving of Qwen3.5-4B on a resource-constrained NVIDIA A10G GPU. Our system combines a quantized target model with speculative decoding. To recover accuracy, we apply quantization-aware distillation to the targ...