Skip to content
Open access

Effective Exploration for Reinforcement Learning via LLM-generated Fuzzy Rules

Sep 2026 · Symposium on Intelligent Systems and Informatics · pp. 785-790 · 0 citations · 35 references

TL;DR

A novel two-stage hybrid architecture that bridges the semantic reasoning capability of Large Language Models (LLMs) with optimized continuous RL control and autonomously generate, test, and iteratively improve a Fuzzy Inference System (FIS) via natural language critique is proposed.

Abstract

While reinforcement learning (RL) has achieved significant success in autonomous control, its deployment in real-world domains is hampered by a profound lack of sample efficiency and the erratic, high-variance action chattering of early-stage exploration. To bypass the detriments of unguided exploration, we propose a novel two-stage hybrid architecture that bridges the semantic reasoning capability of Large Language Models (LLMs) with optimized continuous RL control. This framework utilizes an automated LLM-driven loop to autonomously generate, test, and iteratively improve a Fuzzy Inference System (FIS) via natural language critique. The resulting fuzzy controller then acts as an exploration filter to gather directed transition data, successfully warm-starting an off-policy RL agent. Proof-of-concept experiments evaluated on the HalfCheetah-v5 environment demonstrate that the warm-started agent outperforms a standard off-policy baseline at every stage of early training. Quantitatively, the proposed method achieves a mean episodic reward improvement of +29.71 at 10,000 steps, expands this margin to +897.05 at 20,000 steps, and maintains a substantial lead of +774.84 at 30,000 steps, effectively accelerating early-stage learning with minimal direct environment interactions. Code is available at https://github.com/ABC-iRobotics/rl-exploration-via-gen-fis.

Read PDF

Similar papers

#machine learning Preprint Oct 2026

Self-Evolution Reasoner: Continuous Optimization via Policy-Intrinsic Exploration and Exploitation

Reinforcement learning with verifiable rewards (RLVR) enables large language models (LLMs) to continuously self-improve their reasoning capabilities. Compared to RLVR's scalar rewards, recent self-supervised on-policy distillation methods leverage denser supervision signals. However, both paradigms suffer from explorat...

Wen-Hang Shi, Yi-Ren Chen, Zhe Zhao et al. · 0 citations
Conference Open access Sep 2026

Reinforcement Learning: A Comprehensive Survey of Foundational Algorithms, Recent Advances, and Future Directions

Reinforcement Learning (RL) has emerged to be one of the most radical ideas in artificial intelligence since it enables the learner to learn the desired behavior through a response to the environment. The survey provides the overall picture of reinforcement learning, its development history beginning with the classical...

N. Ramshankar, D.Elangovan, S. S et al. · 0 citations
#machine learning Preprint Oct 2026

Bellman Meets Lyapunov: Unsupervised Reinforcement Learning via Mastering Chaos

Reinforcement learning (RL) is a powerful paradigm for training agents, yet its success rests on domain expertise of human engineers who design informative reward signals for every new task. Unsupervised RL aims to reduce this engineering with intrinsic motivation (IM): reward signals that emerge from the agent environ...

Tristan A. Shah, Wooyoung Chung, V. Makarenko et al. · 0 citations
#natural language process... Preprint Oct 2026

RSIPO: Recursive Self-Improving Policy Optimization for Agentic Reinforcement Learning via Skill-Policy Co-Evolution

Recent focus in agentic reinforcement learning (RL) seeks to improve credit assignment in long-horizon tasks. One promising direction combines reinforcement learning with verifiable rewards (RLVR) and on-policy self-distillation (OPSD), integrating trajectory-level advantages with dense token-level supervision. This li...

Fan Zhang, Hao Wu, Jin-Yang Wu et al. · 1 citation
#artificial intelligence Preprint Oct 2026

Guided-RLVR: Guided Exploration in Reinforcement Learning with Verifiable Rewards

Reinforcement learning with verifiable rewards (RLVR) has shown strong empirical gains in improving the reasoning ability of large language models (LLMs) on tasks such as mathematics and programming. However, recent studies have shown that RLVR rollouts often collapse onto a narrow set of reasoning trajectories, limiti...

Bo-Sen Yang, Z. Zhang, Wei-Hua Li et al. · 1 citation
Preprint Oct 2026

CARL: Chat Assisted Reinforcement Learning

Failure of applied Reinforcement Learning (RL) to solve a given problem predominantly stems from unaddressed specification gaps, where the intent of human experts in the problem's domain is inaccurately or incompletely translated by AI into mathematical objectives, constraints, and reward functions. Failure can also st...

Sanjiv R. Das, Harshad Khadilkar, Sukrit Mittal et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.