Information-Bottlenecked Variational Autoencoder for Top-N Recommendation via Gating Mechanism
Abstract
Generative models for top- \(N\) recommendation have garnered significant attention, with Variational Autoencoder (VAE) emerging as a promising approach for modeling user preferences. Yet, traditional VAE-based models encounter two major challenges: simplistic priors may cause posterior collapse, resulting in ineffective latent structures; meanwhile, limited decoder capabilities hinder the processing of complex data and the extraction of task-relevant information, potentially affecting generative performance. To address these issues, we propose VAEinfox, short for Information-Bottlenecked Variational Autoencoder via Gating Mechanism, a simple yet effective framework that mitigates posterior collapse in VAE. VAEinfox introduces an Adaptive Compressed Representation (ACR) module based on the Information Bottleneck (IB) principle to efficiently and adaptively compress and select latent variables, mitigating posterior collapse. It leverages the Relaxed Bernoulli distribution for differentiable compressed representations and employs a dynamic compression probability selection mechanism with a trainable soft mask activation function to automatically adjust latent variable sparsity during training. Then, a joint training objective maximizes mutual information using the Jensen-Shannon estimator to minimize latent redundancy. In particular, the generative module in VAEinfox incorporates a Mixture-of-Experts (MoE) paradigm, enhancing representation learning and the decoder's ability to process complex data and extract task-relevant information. These dual enhancements improve the model's generative performance while also enhancing its universality and efficiency, particularly when handling implicit feedback. Extensive experiments on eight real-world datasets demonstrate that the proposed framework achieves better results than existing competitors and tackles the specified challenges in top- \(N\) recommendation tasks.