Sep 2026· Proceedings of the 20th ACM Conference on Recommender Systems· pp. 125-134· 0 citations· 16 references
Abstract
Existing sequential recommendation models rely on absolute positional encodings and fixed context windows, producing two structural failure modes: out-of-distribution degradation on histories longer than the training window, and hard context limits that discard long-range interactions. We present CONGA (COntinual Neural Gated Architecture), which addresses these limitations through three contributions: (1) Rotary Positional Embeddings (RoPE) with a norm-preserving property (\(\Vert \mathcal {R}_m\Vert _F = \sqrt {d}\) for any sequence length), accelerated by custom CUDA kernels; (2) KromHC multi-stream fusion with exact doubly-stochastic mixing via Kronecker-product parametrization, where ablation confirms the expressivity gain arises from balanced gradient flow rather than additional parameters, together with a data-adaptive stream selection mechanism that prevents overfitting on sparse corpora; and (3) TITANS neural associative memory adapted to discrete-item recommendation – the first such proof-of-concept – via a two-phase training protocol with a structural forgetting-prevention property: the base encoder is frozen, preserving its short-sequence predictions, while Phase 2 only adds a learned memory term. Evaluated under a rigorous full-ranking protocol across four benchmarks (ML-1M, Beauty, Yelp, Steam), CONGA achieves state-of-the-art performance on long-history benchmarks, with gains up to \(+18.7\%\) HR@5 on ML-1M – the densest, longest-history dataset where all three failure modes are simultaneously active – and consistent improvements on long-history benchmarks, with competitive results on short-history datasets where the RoPE backbone is the primary driver.
Transformers have redefined sequential recommendation by effectively modeling dynamic user behaviors and long-range dependencies. However, they remain inherently inefficient: standard architectures operate at a fixed rate, allocating comparable computation to every item in a user’s history regardless of its information...
Dwipam Katariya, T. Caputo, Akshat Shreemali et al.· Proceedings of the 20th ACM...· 1 citation
This work introduces Structured Matrix Attention via a family of causal masks with structured long-range routing whose row supports have VC-dimension, which makes VC-dimension an explicit knob governing access-pattern complexity, prefill cost, and decoding memory.
E. Anand, Abdullah Ateyeh, Archer Wang et al.· arXiv.org· 3 citations
The contribution is a system-level integration that makes long-term, multi-interest, and multimodal modeling jointly deployable in a real-time production pipeline, together with the engineering practices required to sustain it.
Yong-Kang Fu, Bei-Ning Bao, Yu Jiang et al.· 0 citations
Continual parameter-efficient fine-tuning for large language models (LLMs) must balance retention of previously acquired knowledge, adaptation to new tasks, and strict parameter budgets. We present \textbf{ChainLoRA}, a replay-free continual merging framework built on chain-updated task-vector geometry. From a paramete...
Hang Yin, Hao-Zhe Wang, Yu-Hua Luo et al.· 0 citations
Session-based recommendation (SBR) infers user interest from limited interaction sequences, yet current models show a consistent performance drop when longer histories are available. This degradation is associated with decoder designs that overemphasize recent items while underutilizing distant, yet semantically releva...
Lin-Jiang Guo, N. Bisht, Shi-Qing Wu et al.· IEEE Transactions on Knowled...· 0 citations
Sequential recommendation (SR) aims to predict a user’s next interaction by modeling temporal dependencies in historical behavior sequences. However, modeling long sequences introduces two challenges: longer histories often include noisy interactions irrelevant to a user’s core interests, and increasing sequence length...
Woo-Seung Kang, Minje Kim, Suwon Lee et al.· Proceedings of the 20th ACM...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.