Quantization reduces the memory and bandwidth demands of large language model (LLM) inference but can degrade accuracy; offloading higher-precision data can instead stall generation. We introduce StrataFlux, a framework that challenges this tradeoff by recovering accuracy lost to low-bit weight and key-value (KV) cache...
Jake Hyun, Zi-Yu Zhao, Jae-sun Seo et al.· 0 citations
Machine unlearning is needed when data must be removed because of deletion requests, outdated records, or data-quality concerns, while retraining from scratch can be costly. Certified machine unlearning methods provide mathematical guarantees, while deployed systems release concrete finite-precision artifacts produced...
Zi-Yu Zhao, Xin-Yu Wang, Xiao-Wen Chang et al.· 0 citations
Given a fixed set of pending deletion requests, retraining from scratch after each request is prohibitive, so a prescribed request-wise policy processes them sequentially. The resulting terminal model can depend on their order. Rather than prescribing an ordering rule, we study the permutation objective induced by the...
Xin-Yu Wang, Zi-Yu Zhao, Yi-Xuan He et al.· 0 citations
Results show that ASR self-speculation depends on token prediction, audio-position tracking, and draft cost, and AnchorDraft, which teaches the draft to track the audio position during training without changing the inference graph, improves end-to-end speed at both tested target scales.
OJBKQ is proposed, a layer-wise PTQ method that formulates weight quantization as a joint optimization problem over activations and weights, yielding a multiple-right-hand-side box-constrained integer least squares (BILS) problem per layer.
Xin-Yu Wang, Zi-Yu Zhao, Peng Lu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.