#artificial intelligence #machine learning Preprint

Generative Support Realignment for Cross-Domain Offline Reinforcement Learning

Minung Kim Jeongmo Kim G. Choi Seungyul Han

May 2026 · 0 citations · 44 references

Computer Science

TL;DR

A performance gap bound is derived that characterizes the interplay between generation error and source--target dynamics gap, providing theoretical guidance for effective coverage expansion and source utilization.

Abstract

Cross-domain offline reinforcement learning learns a target policy from pre-collected source and target datasets with different dynamics. When target data are scarce, effectively compensating for their limited coverage using source data remains challenging due to the discrepancy between domains. We propose Target-aligned Coverage Expansion (TCE), which leverages source states to generatively realign and expand the limited target support while controlling the generation error induced by this expansion. We further derive a performance gap bound that characterizes the interplay between generation error and source--target dynamics gap, providing theoretical guidance for effective coverage expansion and source utilization. Across diverse cross-domain environments, TCE consistently outperforms state-of-the-art baselines, while further analysis empirically supports the guidance provided by our theoretical findings.

DOI arXiv Semantic Scholar

View source

Similar papers

Preprint Aug 2026

CoDrift: Compositional Drifting for Offline Reinforcement Learning

This work proposes CoDrift, a compositional framework for one-step generative policy learning that combines three objective-level fields into a unified policy field that compares favorably with state-of-the-art methods and achieves the best average rank in both settings.

Xiewei Ni, Ruo-Feng Mei, Xiang-Yu Xu · 0 citations

#machine learning Preprint Sep 2026

In-Distribution Imagination for Model-Based Offline Reinforcement Learning

Model-based offline reinforcement learning (MBORL) improves sample efficiency through model-generated trajectories. However, accumulative model error can drive imagined trajectories outside the offline data distribution, leading to unrealistic synthetic data and unstable policy optimization. Many existing methods prima...

Mintae Kim, K. Sreenath · 0 citations

Aug 2026

Dual Advantage-Guided Offline Reinforcement Learning

Offline reinforcement learning aims to learn effective policies from fixed datasets without online interaction, necessitating conservative constraints to mitigate the out-ofdistribution issue. Although existing approaches alleviate this issue through conservative constraints or policy regularization, they still struggl...

Hui-Zhi Wang, Yan Kong · International journal of sof... · 0 citations

#machine learning Preprint Sep 2026

Not All Rollouts Are Worth Learning: On Trajectory Valuation for Post-Training Reinforcement Learning

Dynamic Trajectory Valuation is proposed, a simple and efficient framework that estimates trajectory utility at the mini-batch level and filters detrimental trajectories based solely on gradient information by operating at the optimization level.

Xue-Song Jia, Ziao Yang, Zhan-He Huang et al. · 0 citations

#machine learning Preprint Oct 2026

Score-Calibrated Flow for Sampling from Unnormalized Densities with Applications to Generative Online Reinforcement Learning

Diffusion and flow models provide expressive policy classes for online reinforcement learning (RL), enabling multimodal behaviors and improved performance. However, training these policies remains challenging: the critic specifies the desired policy as an unnormalized Boltzmann density but does not provide direct sampl...

Ze-Yang Li, Yu-Nan Wang, Risheek Garrepalli et al. · 0 citations