Skip to content
Book Open access

CONGA: Continual Neural Gated Architecture for Long-History Sequential Recommendation

Sep 2026 · Proceedings of the 20th ACM Conference on Recommender Systems · pp. 125-134 · 0 citations · 16 references

Abstract

Existing sequential recommendation models rely on absolute positional encodings and fixed context windows, producing two structural failure modes: out-of-distribution degradation on histories longer than the training window, and hard context limits that discard long-range interactions. We present CONGA (COntinual Neural Gated Architecture), which addresses these limitations through three contributions: (1) Rotary Positional Embeddings (RoPE) with a norm-preserving property (\(\Vert \mathcal {R}_m\Vert _F = \sqrt {d}\) for any sequence length), accelerated by custom CUDA kernels; (2) KromHC multi-stream fusion with exact doubly-stochastic mixing via Kronecker-product parametrization, where ablation confirms the expressivity gain arises from balanced gradient flow rather than additional parameters, together with a data-adaptive stream selection mechanism that prevents overfitting on sparse corpora; and (3) TITANS neural associative memory adapted to discrete-item recommendation – the first such proof-of-concept – via a two-phase training protocol with a structural forgetting-prevention property: the base encoder is frozen, preserving its short-sequence predictions, while Phase 2 only adds a learned memory term. Evaluated under a rigorous full-ranking protocol across four benchmarks (ML-1M, Beauty, Yelp, Steam), CONGA achieves state-of-the-art performance on long-history benchmarks, with gains up to \(+18.7\%\) HR@5 on ML-1M – the densest, longest-history dataset where all three failure modes are simultaneously active – and consistent improvements on long-history benchmarks, with competitive results on short-history datasets where the RoPE backbone is the primary driver.

Read PDF

Similar papers

Book Open access Sep 2026

DP-Rec: Towards Dynamic Patching for Efficient Long-Sequence Recommendation

Transformers have redefined sequential recommendation by effectively modeling dynamic user behaviors and long-range dependencies. However, they remain inherently inefficient: standard architectures operate at a fixed rate, allocating comparable computation to every item in a user’s history regardless of its information...

Dwipam Katariya, T. Caputo, Akshat Shreemali et al. · 1 citation

SMat-Attention: Structured Long-Context Sequence Modeling

This work introduces Structured Matrix Attention via a family of causal masks with structured long-range routing whose row supports have VC-dimension, which makes VC-dimension an explicit knob governing access-pattern complexity, prefill cost, and decoding memory.

E. Anand, Abdullah Ateyeh, Archer Wang et al. · 3 citations
Preprint Sep 2026

MuSeR: Scalable Long-sequence Recommendation with Multi-interest Modeling

The contribution is a system-level integration that makes long-term, multi-interest, and multimodal modeling jointly deployable in a real-time production pipeline, together with the engineering practices required to sustain it.

Yong-Kang Fu, Bei-Ning Bao, Yu Jiang et al. · 0 citations
#machine learning Preprint Sep 2026

ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs

Continual parameter-efficient fine-tuning for large language models (LLMs) must balance retention of previously acquired knowledge, adaptation to new tasks, and strict parameter budgets. We present \textbf{ChainLoRA}, a replay-free continual merging framework built on chain-updated task-vector geometry. From a paramete...

Hang Yin, Hao-Zhe Wang, Yu-Hua Luo et al. · 0 citations
Open access Oct 2026

MIDA: Multi-Interest Decoding with Adaptive Aggregation for Session-based Recommendation

Session-based recommendation (SBR) infers user interest from limited interaction sequences, yet current models show a consistent performance drop when longer histories are available. This degradation is associated with decoder designs that overemphasize recent items while underutilizing distant, yet semantically releva...

Lin-Jiang Guo, N. Bisht, Shi-Qing Wu et al. · 0 citations
Book Open access Sep 2026

Information-Aware Long Sequence Compression for Sequential Recommendation

Sequential recommendation (SR) aims to predict a user’s next interaction by modeling temporal dependencies in historical behavior sequences. However, modeling long sequences introduces two challenges: longer histories often include noisy interactions irrelevant to a user’s core interests, and increasing sequence length...

Woo-Seung Kang, Minje Kim, Suwon Lee et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.