Bayesian model selection couples a discrete model indicator with a model-specific continuous parameter space. We introduce structured dimension-matched variational transdimensional inference (SM-VTI) for finite enumerable model spaces. A rooted construction graph expresses a model as a sequence of local stop/child decisions. Each typed edge compiles a declared scientific parent-child edit into an exact native-coordinate dimension-matching lifting; an edge-conditioned flow then learns the residual continuous transport. The resulting local policy and conditional flow define one direct joint variational distribution, without embedding every model in a saturated maximum-dimensional surrogate. We derive its exact path density and optimize the joint reverse-KL objective. On a controlled 15-model target, SM-VTI-Joint recovers terminal masses, local actions, and nonlinear conditional geometry. On a 128-model misspecified robust variable-selection problem, a 10-data-set nearly parameter-matched affine comparison with AVTI shows stronger early model-mass recovery and competitive final joint accuracy under the same target-evaluation budget.
Stochastic gradient Markov chain Monte Carlo (SGMCMC) methods enable scalable Bayesian inference, but their performance depends strongly on hyperparameters such as the step size, mini-batch size, and number of leapfrog steps. Since most SGMCMC algorithms lack a Metropolis-Hastings acceptance rate, standard acceptance-based tuning methods are not directly applicable. We propose HyperMC, a multi-fidelity tuning framework that combines Hyperband-style resource allocation with kernel Stein discrepancy (KSD) evaluation. By running multiple successive-halving brackets, HyperMC balances broad exploration of a continuous hyperparameter space with increasingly accurate evaluation of promising configurations under a fixed computational budget. We further introduce Robust HyperMC, which uses global grid initialization followed by elite-guided local refinement to reduce sensitivity to random candidate generation and noisy finite-budget evaluations. Under suitable approximation and concentration conditions for the estimated KSD, we establish that the successive-halving component selects a near-optimal configuration among the sampled candidates with high probability and derive a sufficient computational budget for successful selection. Experiments on logistic regression, probabilistic matrix factorization, and Bayesian neural networks show that HyperMC improves posterior approximation or predictive calibration relative to MAMBA, grid search, and heuristic baselines, while Robust HyperMC yields more stable and reproducible tuning results.
Min-Chin Tan, Xiyun Jiao· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
