We prove Shor's conjecture: an orthogonal measurement maximizes the information obtainable from any two quantum states in finite dimension. We then show that an error constraint can require three outcomes. For a class of full-rank qubit pairs, the best orthogonal basis switches abruptly as the error penalty increases....
The scale and expense of pre-training language models make efficient hyperparameter tuning essential, yet a principled guidance is still missing. In this work, we analyze language model pre-training dynamics from a local landscape geometry perspective. Our study reveals two distinct phases. In Phase I, sharpness of the...
Zhan-Peng Zhou, Yu-Han Sun, Bing-Rui Li et al.· 1 citation
A larger-batch configuration reduces time-to-target only when its throughput gain exceeds its samples-to-target penalty, and a larger-batch configuration reduces time-to-target only when its throughput gain exceeds its samples-to-target penalty.
Zi-Niu Li, Jinbo Wang, Guan-Hua Huang et al.· 0 citations
Extracting total correlations from a quantum system usually requires reconstructing its state, whereas many experiments access only a few measurement settings. A possible shortcut is to add the mutual informations obtained from complementary measurements; in dimensions above two, however, this procedure can count the s...
This work proposes Mixture of Activations (MoA), a token-adaptive FFN design that mixes a dictionary of activation functions using lightweight input-dependent gates while sharing the same linear projections, suggesting that token-adaptive activation mixing is a simple and effective mechanism for improving FFN expressiv...
