KalWeNN: Kalman-Filtered Memory for Weightless Neural Network Classifiers

Abstract

Weightless Neural Networks (WNNs) such as WiSARD learn by storing information directly in RAM-based memory structures, enabling fast training and efficient hardware implementation. However, conventional WiSARD memories store only binary activations or occurrence counts, providing no measure of confidence in the stored information. This paper introduces KalWeNN (Kalman Weightless Neural Network), a weightless classifier that replaces the traditional counting memory with Kalman filter-based memory cells. Each memory address maintains both an estimate of its association with a class and the uncertainty of that estimate. During training, address activations are treated as noisy observations and incorporated through recursive Kalman update. We evaluate KalWeNN on twelve classification datasets under two architectural configurations and compare it with a standard WiSARD implementation. Results show that KalWeNN consistently matches or improves classification accuracy, achieving approximately 93% accuracy on MNIST. More importantly, it substantially improves run-to-run stability, often reducing performance variability by a factor of four to five. These gains are obtained with only modest increases in memory usage and execution time, preserving the lightweight nature of weightless neural networks.