OnPTQ is proposed, an on-policy framework that calibrates on trajectories visited by the current quantized policy, and derives a Decision--Consequence bound linking behavioral deviation to current policy discrepancy and action-conditioned future-value span.
Abstract
Post-training quantization (PTQ) lowers deployment cost for multimodal large language models, but calibration typically reconstructs fixed sequences with local objectives. This overlooks autoregressive feedback: a quantization-induced token change redirects the prefix and changes future states. Yet on-policy coverage alone is insufficient because many decision mismatches barely affect future generation. We propose OnPTQ, an on-policy framework that calibrates on trajectories visited by the current quantized policy. On shared prefixes, OnPTQ identifies quantization-eroded boundaries, evaluates competing tokens through short counterfactual rollouts, and combines current discrepancy with branch consequence into a Decision--Consequence risk. The risk prioritizes critical states, while context anchoring and trajectory refresh preserve multimodal behavior and keep calibration aligned with the updated policy. We further derive a Decision--Consequence bound linking behavioral deviation to current policy discrepancy and action-conditioned future-value span. Across vision--language and omni-modal Qwen models under multiple low-bit settings, OnPTQ improves downstream performance and yields fewer correctness flips against the corresponding Dense/FP16 references, without changing the deployed inference graph.
By coupling QAD's stable low-bit initialization with OPD's on-policy reasoning recovery, the framework provides a comprehensive sub-3-bit solution that preserves broad capabilities while restoring long-form reasoning.
Yuan-Teng Chen, Zhi-Lei Liu, Pei-Song Wang et al.· 0 citations
The proposed Uncertainty-Calibrated MOPD (UCMOPD), which addresses limitations of standard MOPD through two complementary mechanisms, improves the general-capability average over standard MOPD by $4.48\% and $7.86\% while maintaining vertical-domain performance.
Zi-Yuan Liu, Jiao Ou, Jian Liang et al.· 0 citations
PreDE (Predict Before You Deploy), a policy-calibrated framework for predicting quantization-induced task degradation from offline action deviations, supports policy-specific behavioral calibration for quantization configuration selection while identifying candidates that require closed-loop evaluation.
Q-Target Pretrained Transformers is proposed, which keeps the context-conditioned Transformer architecture but replaces behavior cloning with a Bellman-style Q-target objective, and learns to use rewards and transitions in the context to estimate action values, rather than simply imitating the behavior policy.
Yi-Chen Lin, Xu-Yuan Xiong, Xue Wang et al.· 0 citations
Reasoning post-training draws on offline demonstrations and policy-generated rollouts, yet conventional supervised fine-tuning (SFT) followed by reinforcement learning (RL) separates their objectives and limits continued reuse of collected responses. We propose \textbf{Cross-Behavior Policy Optimization} (CBPO), which...
Du Cheng, Jun-Yu Lu, Jing-Fan Zhang et al.· 0 citations
A unified temporal-credit view of token-level OPD and sequence-level OPD is established, showing that practical token-level OPD can be interpreted as a temporal approximation to the sequence-level reverse-KL gradient.
Shi-Qi Liu, Ze-Yu He, Le-Tian Tao et al.· 3 citations