Best Proximity Learning: Stable Optimization of Deep Neural Networks Under Non-Convex Constraints

Abstract

Best Proximity Learning (BPL) is presented as an optimisation framework for training deep neural networks under non-convex and constrained conditions. The framework reformulates parameter updating as the minimisation of a proximity gap between the current parameters and their mapped counterparts, rather than requiring an exact fixed point. On this basis, the study develops the Best Proximity Gradient Descent (BPGD) algorithm, which combines stochastic gradient updates, proximity projection, adaptive thresholding, and constraint-aware mappings. Conceptual convergence principles are formulated for expected proximal contraction, cyclic mappings, and adaptive learning rates. The method is evaluated on image classification, adversarial training, federated learning with non-identically distributed client data, and language modelling. The reported experiments use CIFAR-10, CIFAR-100, MNIST, and WikiText-103 with established neural architectures and compare BPGD with SGD, Adam, AMSGrad, and proximal gradient descent. Across the reported tasks, BPGD demonstrates comparatively stable loss trajectories and favourable accuracy, robustness, and perplexity values, while maintaining a per-epoch computational cost close to that of SGD. The proximity-based formulation is also discussed in relation to recurrent networks, convolutional models, transformers, and constrained parameter spaces. These findings suggest that proximity minimisation may provide a useful basis for stabilising optimisation when fixed-point assumptions are difficult to satisfy. However, stronger mathematical proofs, complete ablation studies, consistent experimental specifications, and independently reproducible implementations are required to establish the method’s broader reliability and scalability.