It is demonstrated that pre-training can induce an implicit bias with a clear statistical advantage over random initialization, enabling feature learning from scarce fine-tuning data.
Flow matching (FM) has recently emerged as a promising framework for generative modeling due to its conceptual simplicity and strong empirical performance. In FM, samples are transported along a vector field parameterized by a neural network, inducing a probability path that evolves from a simple noise distribution to...
Tail-Corrected Top-$k$ On-Policy Distillation (TT-OPD) is proposed, which preserves the advantages of TK-OPD, including rich distributional supervision and low computational cost, while providing an unbiased estimator of the gradient of the reverse KL divergence.
Lin-Jian Meng, Si-Yuan Gan, Yu-Hang Li et al.· 0 citations
An unsupervised framework for functional data that spans both univariate and multivariate cases, with emphasis on the multivariate setting, and factorizes the latent space into separate amplitude and phase embeddings derived from all channels is proposed.
Pei-Da Wu, X. Xiong, Peng-Cheng Zeng· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Multivariate time series anomaly detection (MTAD) is crucial for ensuring the safe and reliable operation of complex systems. Many existing methods learn normal patterns by training reconstruction or forecasting models on predominantly normal data. However, a large portion of these approaches rely on deterministic mode...
W. Koo, Jaeyeong Lee, Taeseong Yoon et al.· 0 citations
What determines the unavoidable sample cost of learning cyclic causal structure? For cyclic linear non-Gaussian models, we study exact condensation recovery from observational data: identifying the strongly connected component (SCC) partition and all edges between components. We establish the first information-theoreti...
Non-negative matrix factorization (NMF) is a singular statistical model whose Bayesian asymptotics are governed by the real log canonical threshold (RLCT). We study the local geometry of the factorization map and derive an upper bound for the RLCT of NMF. Let $H$ be the model inner dimension and $H_0$ the non-negative...
Testing whether two populations of networks share the same edge probabilities is a basic problem in network inference. How hard it is depends on the norm used to measure the difference. For the inhomogeneous Erd\H{o}s--R\'enyi (IER) model, the optimal sample complexity is known for every integer $L_r$ norm and for $1\l...
Inverse Optimal Transport (OT) based methods for representation learning learn representations such that the global OT coupling between a pair of data marginals in the representation space, concentrates on the positive pairs. This is in contrast to previous methods that primarily focused on pairwise matching. However,...
Ngoc-Hai Nguyen, Thuan Nguyen, Prakash Ishwar et al.· 0 citations
Estimating causal effects with continuous treatments in observational studies is challenging due to confounding, model misspecification, and high-dimensional covariates. We propose the Weighted Spline-Expanded Network (WSENet), an end-to-end neural framework that addresses these challenges by combining covariate balanc...
Shu-Cheng Liu, Chan Park, Guan-Hua Chen· 0 citations
This review presents a unified Bayesian formulation of KD that formulates teacher predictions as prior information and establishes a rigorous connection to uncertainty quantification, providing a principled interpretation of how teacher information is incorporated into student learning.
Lu-Yang Fang, Hao-Ran Lu, Jia-Zhang Cai et al.· 0 citations
This work introduces SPatial coHErent risk control for REplay (SPHERE), a general replay-allocation method applicable across a broad range of learning settings and demonstrates that SPHERE improves accuracy and reduces forgetting across noisy-label vision tasks, continual language-model instruction tuning, and code-gen...
Hai-Xiang Sun, Jie-Fu Zhang, Yi He et al.· 0 citations
Jennifer Neville did not want to go into computer science—but that’s exactly where she landed. Neville discusses the starts and stops that led to her professional sweet spot and her work identifying “surprising failures” making it hard for AI to handle complexity. The post What AI gets wrong and what failure teaches us appeared first on Microsoft Research.