Layerwise Decoupling for Stable Structured Sparsification of Fully Connected Layers
It is proved that the constrained decoupled objective is equivalent at optimality to a specific joint penalty on the inner and outer weights, for any positively homogeneous activation, and thus admits a clean projected and proximal formulation.
Charles Kulick, Armenak Petrosyan, Tang Sui
· 0 citations