Retrieval-Centric Deep Learning in Growing Nonparametric Neural Networks
We investigate a general-purpose layer for deep learning that, instead of compressing arbitrary-size training data into fixed-size weight matrices, stores a new pair of key-value representations for every data point during training, and retrieves and recombines these representations through an attention mechanism at in...
Maximilian Schlegel, Rajai Nasser, Seijin Kobayashi et al.
· 0 citations