We consider the problem of high-dimensional inference with the lasso estimator. Different methods including'double selection'techniques and multiple versions of the'debiased lasso'have been proposed for this task with noticeable success. However, most guarantees assume strong hypotheses on the underlying data process and the errors in the linear regression model, such as subgaussian designs and independence between errors and the data itself. We show that'standardizing'one's dataset -- a natural procedure in practical penalized regression -- leads to the same results under much weaker hypotheses, paying only a small price for not assuming light tails. The key technical point allowed by this step is exploiting the concentration properties of self-normalized processes. Importantly, we prove our results for two different methods closely related to the'debiased lasso'. The second method performs valid inference even for a misspecified linear model, under mild sparsity conditions similar to the'double selection'literature.
We consider statistical inference based on the LASSO (cf. Tibshirani (J. Roy. Statist. Soc. Ser. B, Methodol. 58 (1996) 267–288)) in high dimensional regression problems. It is well known that the LASSO produces biased estimator of the regression parameter. The bias problem is further exacerbated when the LASSO has the variable selection consistency property (cf. Lahiri (Ann. Statist. 49 (2021) 820–844)). The centered and scaled point estimators are dominated by the bias term and fail to converge to any non-degenerate limit distribution. In contrast, we show that the LASSO interval estimators based on the Bootstrap are surprisingly accurate. Here, we consider two variants of the Bootstrap, namely, the Residual Bootstrap and the Perturbation Bootstrap, and show that under some regularity conditions, the Bootstrap approximations generated by both variants automatically adjust for the effects of the bias and are second order correct. This is an important finding as centered and scaled LASSO estimators under variable selection consistency fail to have a non-degenerate limit distribution and the Bootstrap approximations provide a viable way of constructing valid confidence intervals. Building on these second order results, we next consider construction of two-sided symmetric Bootstrap confidence intervals and show that with suitable choices of the studentized pivotal quantities, Bootstrap based two-sided symmetric confidence intervals attain a level of accuracy O(n−2) where n is the sample size. Thus, even with penalization and with diverging model parameter dimension, the performance of the Bootstrap here remains comparable to its finite dimensional counterpart in the traditional Smooth Function model (cf. Hall (The Bootstrap and Edgeworth Expansion (1992) Springer-Verlag)). We also establish similar results for the heteroscedastic case for the Perturbation Bootstrap and report results from a moderate simulation study in support of the theoretical findings.
Debraj Das, A. Chatterjee, S. Lahiri· Bernoulli· 0 citations
The recent work of Sarkar and Zhang (2025) introduced Positive Tail Dependence Under the Null (PTDN) and developed Generalized Shifted Benjamini-Hochberg (BH) procedures for two-sided Gaussian $z$- and $t$-testing under known covariance structures. This paper develops further consequences of that framework. First, we derive explicit dependence-adaptive lower and upper bounds for the FDR of the original BH procedure in terms of the conditional variance parameters $\tau_i=1-R_i^2$, where $R_i^2$ is the squared multiple correlation between the $i$th statistic and the remaining coordinates. These bounds recover the exact BH FDR under independence and provide finite-sample, covariance-specific information complementary to generic bounds. We also identify conditions under which the coordinate-specific calibration of shifted BH can provide a rejection advantage over the original BH procedure. Second, we consider the practically important setting in which the covariance matrix is unknown but an independent Wishart estimator is available. Using simultaneous lower confidence bounds for the $\tau_i$'s, we construct a confidence-bound shifted BH procedure and establish finite-sample FDR control. To our knowledge, this is the first shifted-BH-type procedure with a finite-sample guarantee for two-sided Gaussian mean testing under a completely unknown covariance matrix estimated independently. Numerical studies illustrate the behavior of the covariance-adaptive bounds, the potential advantage of shifted BH over BH, and the performance of confidence-bound shifting under unknown covariance.
We introduce a flexible model for covariate-dependent multiple testing which can be encoded using a nonparametric Gaussian mixture model. Weight-localized predictive recursion (PRx), a new development in the methodology of Newton's predictive recursion algorithm, is then leveraged to estimate the components of this mixture model, allowing for recovery of the covariate-localized false discovery rate $\text{Pr}(H_i = 0|z_i,x_i)$ using a single, unified algorithm. This quantity represents the most direct extension of Efron's local false discovery rate to the covariate-dependent setting, and admits provable Bayesian FDR control properties under simple rejection rules. We introduce several procedures for estimating and thresholding the local false discovery rate, and show using various simulations and a real-data example that our procedures lead to increased power, tighter Bayesian FDR control, and more interpretable rejections. We furthermore show that this holds for fixed and randomized hypothesis labels, indicating that our proposed methods perform well under both frequentist and Bayesian interpretations of multiple testing.
In real-world applications, data are often error-contaminated; naively applying conventional methods without accommodating the measurement error effects often yields inconsistent estimates. Biased results can be further exacerbated by the ultrahigh-dimensionality of covariates. Focusing on the widely used function-on-scalar linear regression model, this article develops new methods for simultaneous parameter estimation and variable selection with error-prone covariates that can be ultrahigh-dimensional. The proposed framework provides flexibility to handle different types of measurement error models. We rigorously establish asymptotic properties of the proposed estimators under mild conditions. Notably, the convergence rates and limiting distributions of the proposed estimators depend on the nature of measurement error. Our findings highlight the significant differences of settings with ultrahigh dimensions compared to scenarios with finite dimensions, as well as the drastically different influence of different measurement error processes. For efficient computation, we design algorithms with data-driven tuning. We evaluate the finite sample performance of the proposed method through simulation studies and a real data application, demonstrating its effectiveness in addressing the challenges posed by error-contaminated and ultrahigh-dimensional of covariates.
A neural network is trained on simulated datasets drawn from a prior over a distribution family, using single independent draws of the root T_n - T(F) scored by the pinball loss, a proper scoring rule whose population minimizer is the posterior-predictive law of the root.
Estimating the first stage of an instrumental variables (IV) model with the least absolute shrinkage and selection operator (LASSO) requires choosing a dictionary of technical instruments and a penalty level. First-order asymptotic theory offers no guidance on these choices, as any consistent implementation yields a structural parameter estimator with the same limiting distribution. In finite samples, however, these choices can have a substantial impact on the resulting structural parameter estimate. Working in a model with a single endogenous regressor and homoskedastic Gaussian errors, we use first- and second-order Stein identities to derive the approximate mean squared error (AMSE) of the instrumental-variables LASSO (IV-LASSO) estimator, which can be consistently estimated and used to rank a prespecified list of dictionary-penalty candidates. The AMSE reveals a bias-variance trade-off: more complex first-stage fits better approximate the conditional mean of the endogenous variable but are also more correlated with the structural errors, with complexity measured by the degrees of freedom of the LASSO fit. The weight on this bias rises with the endogeneity of the regressor, a quantity that neither plug-in nor cross-validation penalty rules take into account. Despite the AMSE being derived in a Gaussian model, penalty selection by minimizing the feasible AMSE criterion delivers up to a one-third lower mean squared error compared to cross-validation and plug-in penalty rules in Gaussian and non-Gaussian simulation designs calibrated to the data of Gilchrist and Sands (2016).
Yu-Kun Ma, Manu Navjeevan, Bogdan Salahub· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
