Parameter-Independent Feature Ranking with Volume-Integrated Sharma–Mittal Entropy: Kernel-Based Estimation, Theoretical Properties and Empirical Validation

Correlation-Sensitive Adaptive LASSO for High-Dimensional Data: A Redundancy-Aware Regularization Approach

In multivariate statistical analysis, accurate modeling of the covariance structure is critical for high-dimensional data analysis, variable selection, and regularization. In high-dimensional settings, strong inter-variable correlation and redundancy are key factors limiting the performance of classical sparsity-based methods. While LASSO and its variants provide effective tools for coefficient shrinkage and variable selection, they may select redundant variables and produce unnecessarily complex models in highly correlated settings. In this study, a Correlation-Sensitive Adaptive LASSO (CDA-LASSO) method is proposed to address these limitations. The proposed approach is based on a hybrid weighting mechanism that makes the penalty term sensitive not only to initial coefficient magnitudes but also to the correlation structure between variables. This structure incorporates correlation-based redundancy information and imposes stronger penalties on predictors with higher directed redundancy scores. Under fixed-dimensional regularity conditions, the bounded correlation multiplier is shown to preserve the selection consistency and oracle limiting distribution of Adaptive LASSO. The method was evaluated through 14 high-dimensional simulation scenarios covering different sample sizes, dimensionalities, sparsity levels, correlation strengths, support structures, and normal or heavy-tailed errors. The results indicate that the Max and kMean variants generally reduce the false discovery rate and model size relative to LASSO and Elastic Net while maintaining broadly comparable predictive performance. Numerical improvements over Adaptive LASSO were also observed in several scenarios, although these differences were not uniformly statistically significant. Under very high correlation, reductions in false discoveries were sometimes accompanied by modest decreases in the true positive rate. The real-world Riboflavin analysis further showed that the CDA-LASSO variants produced smaller models than LASSO and Elastic Net while retaining comparable prediction errors. Overall, CDA-LASSO directly incorporates the internal correlation structure of the data into the penalty weights without requiring a predefined graphical structure and provides a practical methodological extension for more controlled and parsimonious variable selection in high-dimensional correlated settings.

Cluster-Based Dimensionality Reduction by Nonparametric Distributional Screening

We consider dimensionality reduction for high-dimensional observations accompanied by a supplied partition into two or more clusters. The objective is not to construct a low-rank projection, but to retain an interpretable subset of the original coordinates that preserves the distributional information distinguishing the clusters. For each coordinate, the proposed procedure compares the cluster-specific empirical distribution functions through a several-sample Kolmogorov-Smirnov separation statistic. We formalize the resulting marginal cluster support and establish simultaneous finite-sample concentration over all coordinates, explicit bounds for false inclusions and omissions, and exact support recovery when the minimum distributional separation dominates the high-dimensional stochastic error. We also quantify the dimension inflation induced by using an unadjusted testing level and give a familywise-error-controlled version. Under a conditional sufficiency condition, sure screening preserves the full-data posterior cluster probabilities, mutual information, and Bayes risk; an additional result characterizes robustness to imperfectly estimated cluster labels. The procedure is invariant to strictly increasing coordinate transformations and can retain low-variance cluster signals that principal components may discard. We further develop average dual information, a criterion combining partition agreement after transformation with structural coverage of cluster-relevant coordinates, and derive its basic properties and consistency. Simulations illustrate the theory, the interpretability of the selected coordinates, and the distinction between cluster-directed screening and variance-directed projection.

Multi-kernel spectral clustering: Entrywise eigenvector perturbation bounds and exact recovery

Kernel spectral clustering with a single bandwidth can be inadequate for data exhibiting multiple characteristic pairwise-distance scales, a problem particularly prevalent in the high-dimensional regime. We address this issue through a multi-kernel formulation that aggregates kernels with different bandwidths. The bandwidths are selected as prescribed empirical quantiles of the pairwise squared distances, thereby capturing the relevant distance scales without requiring prior population-scale information. We develop a rigorous theoretical analysis of the resulting method under a general high-dimensional, multi-scale mixture model with heterogeneous cluster centers and covariance geometries. We construct a blockwise constant, low-rank informative approximation to the empirical multi-kernel matrix and establish row-wise $\ell_{2,\infty}$ perturbation bounds for its leading spectral components, as well as for the associated normalized Laplacian matrix. These bounds yield observation-level control of the spectral embedding, which is more informative than conventional global eigenspace perturbation estimates. Under suitable eigen-gap and cluster-separation conditions, we show that approximate $K$-means applied to the multi-kernel spectral embedding achieves exact recovery with high probability.

What Does Goodness Measure? A Likelihood-Ratio Account of Forward-Forward Learning

Bridging extrinsic and intrinsic variable importance

Simulations show that VIMP and MPLOCO agree most closely when the fitted learner is well aligned with the data-generating mechanism, and clarify when intrinsic and extrinsic importance can be interpreted similarly and when they provide complementary information.

Robust Functional Regression via FPCA, Copula-Scale Design, and Multivariate Bernstein Smoothing

We introduce a robust nonparametric regression framework for functional covariates that combines functional principal component analysis (FPCA), marginal copula-scale normalization, bounded-score M-estimation, and multivariate Bernstein smoothing. The proposed procedure reduces the infinite-dimensional functional predictor to a low-dimensional score representation, transforms the retained scores onto the compact unit cube, and estimates a conditional M-functional through a smoothly aggregated system of local estimating equations. This construction is designed to accommodate nonlinear regression structure, heavy-tailed score distributions, and response contamination while limiting the influence of extreme observations. Under suitable regularity and undersmoothing conditions, we establish pointwise and uniform consistency, derive explicit convergence rates, and prove asymptotic normality. The limiting variance contains an explicit Bernstein concentration factor that plays a role analogous to the integrated squared kernel in classical nonparametric regression. The analysis also clarifies the interaction among the projection dimension, the Bernstein resolution, the empirical copula transformation, and the effective local sample size. The finite-sample performance of the method is examined through simulations involving heavy-tailed functional scores, Student-t errors, nonlinear regression effects, and increasing response contamination. The proposed estimator exhibits strong overall predictive performance and good robustness, with particularly favorable behavior under absolute-error criteria.