Offline reinforcement learning (RL) aims to learn optimal policies from static datasets while enhancing generalization to out-of-distribution (OOD) data. To mitigate overfitting to suboptimal behaviors in offline datasets, existing methods often relax constraints on policy and data or extract informative patterns through data-driven techniques. However, there has been limited exploration into structurally guiding the optimization process toward flatter regions of the solution space that offer better generalization. Motivated by this observation, we present FANS , a generalization-oriented structured network framework that promotes flatter and robust policy learning by guiding the optimization trajectory through modular architectural design. FANS comprises four key components: (1) Residual Blocks, which facilitate compact and expressive representations; (2) Gaussian Activation, which promotes smoother gradients; (3) Layer Normalization, which mitigates overfitting; and (4) Ensemble Modeling, which reduces estimation variance. By integrating FANS into a standard actor-critic framework, we highlight that this remarkably simple architecture achieves superior performance across various tasks compared to many existing advanced methods. Moreover, we validate the effectiveness of FANS in mitigating overestimation and promoting generalization, demonstrating the promising potential of architectural design in advancing offline RL.
Da Wang, Yi Ma, Ting Guo et al.· Neural Information Processin...· 0 citations
Large Language Model (LLM)-driven AutoML agents have shown strong capabilities in constructing end-to-end machine learning pipelines. However, their effectiveness is limited by costly execution-based feedback, which can make the search for high-quality solutions inefficient under restricted computational budgets. We propose PACE (Pre-execution Admission via Code Embeddings), an online-adaptive admission control framework that improves budgeted sample efficiency by estimating candidate utility prior to execution from within-run execution history, without training a separate offline predictor. The core idea of PACE is to leverage latent structure in the solution space as a within-run admission signal. It projects candidate solutions into multi-view semantic embeddings, dynamically organizes executed candidates into clusters, and estimates new candidates by their proximity to historical elite regions. Moreover, PACE aggregates multi-view embeddings via an adaptive reweighting strategy that prioritizes views with higher discriminative power. This enables PACE to bias the agent toward high-potential regions under a limited computational budget while retaining exploration when local structure is weak. We demonstrate that PACE operates as a plug-and-play admission layer for AutoML agents, serving as either an execution gate or a search prior. In the tested settings, it improves the density of elite solutions found within fixed budgets without modifying the underlying agent architecture. Code, configurations, and prompt templates are publicly available at https://github.com/fendss/PACE.
Gangyi Zhao, Hebin Liang, Hongyao Tang et al.· Proceedings of the 32nd ACM...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
