This work extends the orthogonal polynomial kernel paradigm by introducing a novel family based on discrete Hermite I polynomials, a class of $q$-orthogonal polynomials that generalize classical Hermite polynomials through a deformation parameter $q$.
Abstract
The performance of Support Vector Machines (SVMs) critically depends on the kernel function choice, which enables implicit mapping of data into high-dimensional feature spaces. While classical kernels like Radial Basis Function (RBF) remain popular, orthogonal polynomial kernels offer mathematically interpretable alternatives that can incorporate structured prior knowledge. This work extends the orthogonal polynomial kernel paradigm by introducing a novel family based on discrete $q$-Hermite I polynomials, a class of $q$-orthogonal polynomials that generalize classical Hermite polynomials through a deformation parameter $q$. We formally define the q-Hermite kernel and establish its validity under Mercer's theorem. The kernel's inherent boundedness properties naturally prevent annihilation and explosion effects without requiring explicit scaling mechanisms. Extensive experiments across 20 benchmark datasets demonstrate that the proposed kernel achieves competitive performance compared to both classical kernels and other orthogonal polynomial kernels, while offering advantages in numerical stability and computational simplicity. Our results confirm that $q$-orthogonal polynomials constitute a promising direction for kernel design, bridging mathematical elegance with practical machine learning applications, that provides conceptual and algorithmic resources that may be further extended to emerging quantum computing paradigms. To facilitate full reproducibility, we provide the complete implementation and experimental pipeline in an open-access GitHub repository at https://github.com/Kokechacho/SVMs-QSVMs.
This work introduces a general kernel-based encoder-decoder framework for operator learning that separates observation, representation, learning, and reconstruction, and develops this framework for multi-input, multi-output operator learning, where operators map between products of potentially distinct function spaces.
Adrien Weihs, Chun-Yang Liao, Jingmin Sun et al.· 0 citations
Kernel methods, which embed data distributions into a reproducing kernel hilbert space (RKHS) via positive-definite similarity measures, continue to play an important role. However, learning a good, generalizable kernel for high-dimensional and heterogeneous data under temporal or regional distribution shift remains challenging. To address these issues, we propose SpectraMancer, which learns kernels directly in the Fourier spectral domain induced by multilevel circulant matrices, thereby enabling generalizable kernel learning for complex data. SpectraMancer embeds all shift-invariant candidates into a common multilevel order via randomized multilevel circulant matrices, which yields a fixed Fourier diagonalization and turns inverses, products, and gradients into elementwise fast Fourier transform (FFT) operations. To the best of our knowledge, this is the first kernel-learning approach that exploits randomized multilevel circulant matrices for joint diagonalization across kernels. SpectraMancer further enforces scale invariance via kernel double centering and Frobenius normalization, reduces spectral variance through antithetic phase pairing with quasi-Monte Carlo draws, and optimizes a solver-free spectral risk proxy (SRP) for bandwidth weighting without repeated inner solves. Experimental results show that SpectraMancer improves spectrum-aware kernel selection and predictive performance across diverse benchmarks.
Li-Zhong Ding, Jiarun Fu, Qiuning Wei et al.· IEEE Transactions on Neural...· 0 citations
Approximating the input-output behavior of a multivariable black-box function from limited data is challenging when blind to the importance of its inputs and their interactions. We introduce total sensitivity kernels (TSKs), a method based on families of weighted ANOVA kernels that learn and adapt to this multivariable structure. TSKs parameterize the weights on each multivariable component of the target function by factors for each input. We propose learning these factors directly from function evaluations by selecting the reproducing kernel Hilbert space (RKHS) in which the target function has minimum norm. Under suitable conditions, we show that this norm-minimization problem admits a unique solution, and we establish consistency of a finite-data formulation based on minimum-norm interpolation. The learned TSK factors characterize the participation of individual inputs across interactions and main effects, providing a kernel-dependent notion of input sensitivity related to total Sobol indices. Numerical experiments demonstrate that adapting the kernel to learned multivariable structure can substantially improve approximation accuracy over a standard product kernel.
A bottleneck in the theory of kernel methods in machine learning is the storage requirement. To ameliorate this, a standard trick is to replace the kernel with an explicit feature map. Perhaps, the most well known example is the Gaussian kernel which can be expressed in terms of the Fourier features. Analytically, the kernel $K$ can be expressed in terms of an integral expression that involves a possibly asymmetric kernel $G$ representing the feature map. Numerically, one needs to approximate this integral by a suitable numerical integration scheme, typically Monte Carlo. In this paper, we demonstrate that the eigenvalues of $K$ are approximated much better by the eigenvalues of the kernel obtained by discretizing the integral using suitable quadrature formulas instead. We illustrate this fact in the case of the Gaussian kernel and neural tangent kernels on the unit sphere of a four-dimensional Euclidean space corresponding to the sigmoid and ReLU activation functions.
A. Cloninger, Q. L. Le Gia, H. Mhaskar· 0 citations
We show that, up to isotropic scaling, the Gaussian RBF reproducing kernel Hilbert space (RKHS) is asymptotically isometric to Euclidean space in the large bandwidth limit. This strongly suggests that kernel-based constructions reliant on metric properties of the RKHS will yield results for Gaussian RBF kernels that similarly approach those of linear kernels for large bandwidths. The asymptotic behavior of Gaussian CKA can be understood in this light. We further consider kernel PCA, showing that Gaussian RBF eigenvalues, eigenprojections, and principal components all converge to those of classical (linear) PCA as bandwidth $\sigma \rightarrow \infty$. For a given data representation, both the RKHS feature embeddings and the orthogonal PCA eigenframes of the two kernel types differ asymptotically by a geometric similarity transformation, up to a residual of size $O \left (\frac{\rho}{\sigma} \right )^2$, where $\rho$ is a measure of geometric eccentricity of the representation, equal to the ratio of maximum to median pairwise distance between data examples. Experiments over a diverse collection of data sets demonstrate that $\rho$ provides a simple and reliable predictor of dataset-specific convergence behavior in the top principal directions.
It is concluded that remaining instabilities are attributable to numerical issues, providing a unifying validity foundation for operator-informed kernel methods.
J. Moser, C. Albert, S. Ranftl· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
