An orthogonal update framework tailored to minimax optimization, which equips orthogonal updates with an adaptive stepsize rule driven by accumulated gradient norms, yielding an effectively parameter-free method for nonconvex-strongly-concave minimax optimization.
Zhi-Wei Zhai, Xin-Yu Wang, Wen-Jing Yan et al.· 0 citations
Kronecker-preconditioned optimizers recompute a cached curvature eigendecomposition only periodically, and implementations and adaptive methods control this through the refresh frequency. We show that frequency leaves part of a schedule free: at a fixed number of refreshes, when they occur changes held-out loss at the...
Xin-Yu Wang, Zhi-Wei Zhai, Yu Gu et al.· 0 citations
This work presents Whisper-Flash, a two-layer drafter built on a property of speech recognition: the words still to be written have already been spoken, which reads Whisper's encoded audio and accepted decoder states and proposes eight tokens in a single forward pass.
Hua-Peng Zhou, Hua-Yu Wang, Junkai Wu et al.· 0 citations
Machine unlearning is needed when data must be removed because of deletion requests, outdated records, or data-quality concerns, while retraining from scratch can be costly. Certified machine unlearning methods provide mathematical guarantees, while deployed systems release concrete finite-precision artifacts produced...
Zi-Yu Zhao, Xin-Yu Wang, Xiao-Wen Chang et al.· 0 citations
Given a fixed set of pending deletion requests, retraining from scratch after each request is prohibitive, so a prescribed request-wise policy processes them sequentially. The resulting terminal model can depend on their order. Rather than prescribing an ordering rule, we study the permutation objective induced by the...
Xin-Yu Wang, Zi-Yu Zhao, Yi-Xuan He et al.· 0 citations
FTC, a sequential structured compression framework that adapts the approximation to the current compressed model while jointly exploiting the native Q/K/V head structure under a fixed storage budget, achieves the lowest WikiText-2 perplexity among the compared methods.
Jian Chen, Xin-Yu Wang, Tian-Shuo Yan et al.· 0 citations
TreeSpark is introduced, which reads a parent-conditioned distribution from the drafter's existing Markov head at negligible cost, calibrates it into an edge-acceptance estimate, and lets path survival govern everything else: best-first expansion, per-round stopping, and a load-adaptive serving policy.
Results show that ASR self-speculation depends on token prediction, audio-position tracking, and draft cost, and AnchorDraft, which teaches the draft to track the audio position during training without changing the inference graph, improves end-to-end speed at both tested target scales.