Skip to content

PIcsC: Partitioning-Induced Covariate Shift Correction

Jul 2026 · arXiv.org · Vol abs/2607.25441 · 0 citations · 50 references
Computer Science

TL;DR

The results demonstrate that Fisher information provides an effective and unified mechanism for mitigating partition-induced covariate shift across both centralized and distributed learning.

Abstract

Covariate shift across training-data partitions biases model selection and parameter estimation in cross-validation, lifelong learning, and federated learning. We propose \textit{Partition-Induced Covariate-shift Correction} (\texttt{PIcsC}), a Fisher information-based regularization framework that mitigates distribution mismatch between data partitions and a reference distribution. \texttt{PIcsC} approximates partition divergence using the Fisher Information Matrix (FIM) and incorporates the resulting statistic as a regularizer during optimization. The same formulation applies to both centrally partitioned datasets (batches or cross-validation folds) and inherently distributed data (federated clients or decentralized nodes), requiring only partition-local gradient statistics rather than raw data. We further introduce a conditional adaptation mechanism that combines FIM shift with KL divergence to detect significant distribution shifts and activates regularization only when necessary. Experiments on more than 40 datasets demonstrate consistent improvements under both natural and synthetic covariate shift. On fragmented batch and fold settings, \texttt{PIcsC} reduces fragmentation-induced performance degradation by more than 20\% and 25\%, respectively. On seven federated learning benchmarks, it consistently outperforms FedAvg, FedProx, and SCAFFOLD by 3 -5 percentage points without requiring client-specific personalization. These results demonstrate that Fisher information provides an effective and unified mechanism for mitigating partition-induced covariate shift across both centralized and distributed learning.

View source

Similar papers

Preprint Aug 2026

Separating Covariate Shift from Mechanism Change with Two Discriminators: CJSD, a Conditional Discrepancy with an Exact Covariate-Concept Decomposition

The Conditional Jensen-Shannon Discrepancy (CJSD) is proposed, which proves a covariate-null property, a drift-mass law, a one-sided misspecification-control inequality, a one-sided misspecification-control inequality, and a fixed-measure metrization via an identifiability lemma.

Kentaro Oda · 3 citations
#artificial intelligence Preprint Sep 2026

General Quantification of Covariate and Concept Shifts

This paper proposes a key notion: $\gamma^{*}\!$-concept shifts, and derive a general error bound unifying covariate and $\gamma^{*}\!$-concept shifts, which applies to broad loss functions, label spaces, and stochastic labeling and develops estimators for these shifts with concentration guarantees.

Hong-Bo Chen, L. Xia · 0 citations
Preprint Sep 2026

Summary-powered prediction under distribution shift

This work considers a multi-resolution setting in which individual-level data are available from a source population, while the target population is observed only through subgroup summaries, and proposes SAGE, a one-step estimator that updates a source-trained predictor using a gradient estimated from these summaries.

Ivy Zhang, Dominik Rothenhaeusler · 0 citations
#artificial intelligence Preprint Oct 2026

Rethinking Data Augmentation under Covariate Shift: Invariant-Guided Diffusion and Prototype Reweighting

In many industrial applications, 1) tabular data is scarce and imbalanced and thus requires synthetic expansion; 2) input distributions drift between training and deployment (covariate shift); 3) validation sets often diverge from unseen test environments; or 4) standard generative models simply mimic outdated source d...

Hong-Yu Cao, Xin-Yuan Wang, Arun Vignesh Malarkkan et al. · 1 citation
Open access Oct 2026

Transfer Learning Under Group‐Label Shift: A Semiparametric Exponential Tilting Approach

This article studies unsupervised transfer learning for a binary outcome, with labeled source data and unlabeled target covariates, and formulate this structure as group‐label shift (GLS) and pair it with a class‐specific exponential tilting model for the joint density ratio between the target and source populations.

Man-Li Cheng, Subha Maity, Qing-Long Tian et al. · 0 citations
#machine learning Preprint Oct 2026

High-Dimensional Asymptotics and Dataset Selection for Private Transfer Learning

To commit to buying external data or participate in collaborative learning, one must decide whether the additional data will improve prediction enough to justify the cost. This comes with several challenges: (i) the decision often relies only on aggregated statistics available publicly, rather than individual-level dat...

Filip Kovacevic, Edwige Cyffers, Stefano Sarao Mannelli et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.