Geometric Supercharger: Data-Informed Hyperbolic Initialization of MLP-based Sparse Neural Networks
Sparse neural networks can substantially reduce computational and memory costs, yet their performance remains highly sensitive to the choice of sparse topology, particularly at extreme sparsity, where they struggle to outperform fully connected models. Existing sparse initialization methods are predominantly structure-...
Thomas Adler, Diego Cerretti, Carlo Vittorio Cannistraci
· Preprints.org · 0 citations