Oracle high-dimensional $M$-estimation using smooth reparameterization for sparsity

Abstract

This paper establishes a unified non-linear regularization framework for high-dimensional $M$-estimation, encompassing both linear models and Cox's proportional hazards models. Rather than relying on traditional additive non-convex penalties, the proposed paradigm embeds sparsity directly into the transformation for the physical parameter $\theta=\phi(\beta)$ using a smooth ($C^2$) component-wise"ReParametrization map for Sparsity (RePS)"$\phi$, and the penalty term is $\lambda \Vert \beta \Vert_1$ rather than $\lambda \Vert \theta \Vert_1$. This structural formulation dynamically adapts to local parameter scales, suppressing high-dimensional noise while simultaneously recovering unbiased oracle asymptotic normality under the large-sample limit. Through the Primal-Dual Witness method, we establish a unified oracle equivalence for both model classes under a liberated micro-penalty scaling regime $\lambda \ll n^{-1/2}$, accommodating model-specific structures such as the shift-invariance in survival analysis. Extensive Monte Carlo simulations demonstrate that the proposed framework consistently achieves superior false-positive control and high 95\% confidence interval coverage in high-dimensional linear models, while also delivering performance comparable or superior in all aspects to state-of-the-art methods like SCAD and MCP in proportional hazards models.