Jul 2026· International Conference on the Theory of Information Retrieval· 0 citations· 53 references
Computer Science
TL;DR
This work demonstrates that PFNs, which are originally trained for classification, successfully outperform classification baselines on ranking data and introduces a novel sampling and inference scheme to obtain pairwise predictions from PFNs' native pointwise architecture, analogous to pairwise LTR.
Abstract
Learning to rank (LTR) traditionally requires large-scale training data to generalize effectively. In low-data domains where expert annotation is scarce, the performance of LTR methods degrades sharply. Foundation models have alleviated similar data dependencies in other domains via in-context learning, but a foundation model for ranking with tabular features has not been explored yet. We propose prior-data fitted networks (PFNs) as a strong method for ranking in low-data settings. First, we demonstrate that PFNs, which are originally trained for classification, successfully outperform classification baselines on ranking data. Next, we evaluate PFNs as rankers, showing that they surpass state-of-the-art tuned baselines in low-data regimes. We introduce a novel sampling and inference scheme to obtain pairwise predictions from PFNs' native pointwise architecture, analogous to pairwise LTR. To address the limited context window of the transformers underlying PFNs, we propose a dynamic support set selection strategy for queries that scales PFNs beyond random subsampling. Our experimental results show that PFNs are an effective foundation model for ranking that provides significant gains when data is limited.
RecPFN is pretrained entirely on synthetic clickstream environments sampled from a broad structural causal prior, enabling it to amortize Bayesian-style inference from a small support set, and provides a practical path toward generalizable, data-efficient recommenders.
En-Zhi Tan, Jia-Xiang Lim, Bryan Lijie Chew et al.· Annual International ACM SIG...· 0 citations
A task-centric, retrieval-based perspective is offered for how TFMs generalize: it is believed that tabular in-context generalization is largely retrieval-based, and good models are those that learn to identify relevant examples in the provided context and aggregate them well.
Nour Shaheen, Junwei Ma, Alex Labach et al.· 1 citation
This tutorial presents a systematic overview of this emerging paradigm of tabular foundation models, which treats tables as a common representation that can capture information from tabular data, time series, and graphs within a shared learning framework.
Peng Cui, Xingxuan Zhang, Han-Jia Ye et al.· Proceedings of the 32nd ACM...· 0 citations
In document reranking, rerankers based on Large Language Models (LLMs) demonstrate superior performance but are constrained by high memory consumption and latency. To develop lightweight yet high-performance LLM-based pointwise rerankers through knowledge distillation, we identify two critical limitations: teachers often yield over-smoothed and inaccurate supervision on hard negative samples, thereby hindering the student's optimization; furthermore, traditional methods underutilize the relevance score differences between candidates, which are crucial for ranking tasks. To address these challenges, we propose CRED (Calibrated Relational Enhanced Distillation), which integrates Adaptive Teacher Calibration (ATC) to calibrate teacher predictions and amplify score margins, while employing Preference Relation Alignment (PRA) to align the distributional patterns of relevance score differences, enabling the student to capture precise ranking structures. To support this approach, we also construct FineDistill, a dataset of 1M samples providing fine-grained score supervision. We distill an 8B teacher into a 0.6B pointwise student. Extensive experiments on TREC and BEIR benchmarks show that our model outperforms leading baselines in both performance and generalization.
Qingran Yang, Wenxuan Zhang, Yuting Wang et al.· Annual International ACM SIG...· 0 citations
Experimental results demonstrate that proposed Bayesian domain weighting method could achieve stable and efficient domain weights learning, and identifies optimal mixtures while consuming substantially less data than search-based function-fitting methods, revitalizing optimization-based domain weighting for large-scale applications.
Xiang Yuan, Kai-Qing Lei, Zhenyu Jin et al.· arXiv.org· 0 citations
PreGress is proposed, the first ranking-native pre-training and prompting framework for supporting a wide range of node ranking tasks, and designs lightweight, task-specific prompt modules that adapt a frozen ranking backbone to downstream tasks without full retraining.
Lujie Ban, Jiasheng Shi, Yingli Zhou et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
