The Spiking Actor Network Soft Actor Critic (SANSAC) is proposed to address the use of RL frameworks in continuous environments, designed as a framework that can be implemented on neuromorphic hardware.
Abstract
Reinforcement learning (RL) algorithms have made strides over the past decade applying them to a wide range of problems and control tasks. However, the deployment of RL on neuromorphic hardware for continuous control tasks remains under-validated. Namely it is unclear whether replacing a conventional actor network with a spiking neural network (SNN) affects the performance of an agent before any hardware-specific benefits manifest. We provide a systematic validation of a minimal, neuromorphically viable spiking actor variant of Soft Actor-Critic (SAC) on conventional hardware, establishing a baseline for future neuromorphic RL research. In this paper, we propose the Spiking Actor Network Soft Actor Critic (SANSAC) to address the use of RL frameworks in continuous environments, designed as a framework that can be implemented on neuromorphic hardware. We compare a traditional Soft Actor Critic (SAC) network to SANSAC in a traditional computer. We demonstrate the near equivalent performance of SANSAC and SAC, while addressing the impact of hidden dimensions. Our results demonstrate the viability of SNN based algorithms in complex continuous environments, as well as competitive performance to traditional neural networks in traditional computers, providing a basis to continue exploring the use of SNNs in continuous RL frameworks.
It is aimed at proving that SNNs have potential in such areas as computer vision, robotics, and speech recognition, and their role in overcoming the barrier between artificial and biological neural systems is proved.
Mesala Sravani, K. Kumari, S. M. Reddy· International Journal of Unc...· 0 citations
This work proposes a noisy group neuron (NGN) model, which incorporates population-level synchronous resetting and neural stochasticity as fundamental computational mechanisms, and develops the NGN method as a framework that combines the NGN model with backpropagation learning based on mean-field dynamics.
Yajie Zhai, Yanmei Kang, Meng Li et al.· 0 citations
This work introduces their symmetric counterpart by replacing adaptation with slow self-excitation, motivated by intrinsic calcium-mediated membrane currents, and derives and validate a mean-field neural mass model that remains stable while retaining working-memory functionality.
D. Depannemaecker, Adrien D’hollande, G. Casagrande et al.· Nature Communications· 0 citations
Neuromorphic computing is closely associated with spiking neuronal networks. However, an alternative class of so-called"rate-based"models arising from computational neuroscience and machine learning forgoes spiking interactions and instead relies on continuous coupling between neurons. Existing neuromorphic implementations designed around spike-based interactions are not well-suited for emulating such models. Here view the distributed simulation of these models as message-passing algorithms on parallel hardware. Leveraging prior art in numerical algorithms and distributed simulation, we outline steps that enable the design of efficient digital neuromorphic accelerators for non-spiking neuronal models. In particular, we show that multi-bit packets, rather than spikes, are the most efficient communication strategy in packet-switched networks and that compared to basic numerical integration methods, higher-order differential equation solvers decrease both computation and communication costs while achieving lower numerical error, but that these benefits are ultimately limited by arithmetic precision. Using our proposed design principles, we convert an existing neuromorphic architecture into a distributed numerical solver - a spikeless neuromorphic system - for continuously-coupled neuronal models. We thereby demonstrate that our theoretical considerations indeed translate into practical advantages, namely reduced energy consumption and delay.
Jakob Jordan, Ole Richter, Cong Li et al.· 0 citations
This paper integrates reinforcement learning with reward-modulated, bio-inspired SNNs that employ reward-modulated spike-timing-dependent plasticity (R-STDP) to address real-time control tasks and reconstructs the task-oriented state space and reward mechanism to realize a cross-task control framework.
Yijing Fan, Chun Zhao, Lin Zhang· Simulation (San Diego, Calif...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
