Deep reinforcement learning for separation control in turbulent wind-tunnel flow

Abstract

This work investigates Deep Reinforcement Learning (DRL) as a tool for model-free closed-loop active separation control in a fully turbulent wind tunnel flow over a one-sided diffuser. The agent controls an array of magnetic valves (on/off) that eject compressed air into the boundary layer, while the environmental state is reduced to the signal from a single wall-shear-stress sensor placed near the natural transitory detachment point. The control law is learned in real time using Proximal Policy Optimization. Compared to the standard learning design based on the weighted sum of all rewards following an action, we demonstrate that a horizon aligned with the convective time of the flow leads to faster convergence and a more robust control strategy. The resulting control law corresponds to a low-duty-cycle actuation pattern that yields a forward-flow fraction of approximately $53\%$. This compares favorably with conventional and optimized periodic open-loop control ($\sim 40\%$ and $\sim 51\%$, respectively). The findings of this article indicate that, when embedded into an online experiment, DRL represents an efficient tool to identify robust and interpretable active separation control strategies.