Skip to content
Book Open access

OffTrail: A Benchmark for Natural-Language Context in Recommendation beyond Interaction Histories

Nov 2026 · Proceedings of the 35th ACM International Conference on Information and Knowledge Management · 1 citation · 40 references

Abstract

Recommender systems are increasingly expected to combine interaction histories with language. In many cases, a user's next choice may depend on a temporary situation---such as their current mood, plans, or constraints---that is not visible in past behavior. Existing benchmarks, however, mostly evaluate average next-item accuracy, broad instruction following, or content understanding; they do not isolate recommendation episodes in which external situation context is plausibly needed. We introduce e OffTrail, a benchmark built on MovieLens for situation-aware recommendation beyond interaction histories. It contains two complementary families of episodes: observed off-trail choices that are real user transitions but hard to infer from prior behavior, and neighbor-supported alternative continuations that represent plausible futures for similar recent histories. Each episode packages a user history, target item, item metadata, and generated life-situation context. The contexts are treated as plausible external circumstances rather than ground-truth user intentions. OffTrail supports controlled evaluation across sequential models, LLM-based recommenders, and hybrid systems by keeping the recommendation episode fixed while exposing or withholding the situation context. By releasing the artifacts, construction pipeline, and evaluation protocol, OffTrail provides a reproducible testbed for studying whether natural-language context improves ranking when interaction history is weak or ambiguous. Benchmark: https://doi.org/10.5281/zenodo.20560655.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.