2026· IEEE Control Systems Letters· Vol 10, pp. 2131-2136· 0 citations· 16 references
Computer Science
Abstract
This letter proposes a learning method for real-time solution of receding-horizon mixed-integer nonlinear programs (RH MINLPs) in dynamic energy supply systems. Conventional MINLP solvers incur high computational costs. Although machine learning (ML) approximations can reduce the cost, existing approaches struggle to satisfy temporally coupled hard constraints solely through the forward pass, as required in industrial applications. To address this issue, we propose a two-stage architecture. First, RH MINLP is reformulated via backward recursion to decouple temporal dependencies into local allocation tasks. Second, a specialized neural network (NN) architecture embeds structural correction mechanisms into differentiable layers, thereby satisfying hard constraints during the forward pass without the need for external solvers. The proposed NN comprises a scalable binary-constraint layer and a continuous-allocation layer. A case study on a district cooling system demonstrates that the proposed method enables large-scale end-to-end training and generates feasible operational plans in real time.
In the continuous decision-making process of complex engineering systems, it is often necessary to simultaneously consider long-term performance optimization and instantaneous physical executability. Therefore, the present study proposes a constraint-aware deep reinforcement learning method, CTD3, for the energy management of multi-stack fuel cell hybrid trams, aiming to coordinate long-term constraints with single-step action executability. Based on the multi-energy coupled dynamic structure, stack availability, minimum stable power, and energy storage boundaries are incorporated into the corresponding time-varying feasible domain, and the stack-level operating environment is explicitly modeled. On this basis, the energy management problem is formulated as a constrained Markov decision process (CMDP), and an adaptive Lagrangian constraint adjustment mechanism is introduced into the twin delayed deep deterministic policy gradient (TD3) framework to achieve long-term optimization and constraint maintenance requirements. Action projection and feasible power mapping execution layers are designed to convert system-level continuous actions into executable instructions that satisfy the physical boundaries at the stack level. Simulation experiments are conducted under typical urban line conditions based on a joint Python and MATLAB/Simulink platform. The results indicate that, compared with the standard TD3 algorithm, CTD3 exhibits faster convergence, improved training stability, and a decreasing trend in the overall violation rate of comprehensive constraints. Relative to the finite state machine (FSM), the equivalent hydrogen consumption of CTD3 is reduced by 6.31% and the comprehensive constraint violation rate is maintained within the 2% threshold. Meanwhile, the proposed method can guide the formation of differentiated power allocation among multiple stacks and promote the coordinated distribution of power batteries and supercapacitors. The results demonstrate that CTD3 achieves good overall performance in reducing equivalent hydrogen consumption, controlling the comprehensive constraint violation rate, and coordinating multi-source power allocation under the tested typical operating conditions, thereby providing a feasible constraint-aware reinforcement learning approach for constrained continuous control problems such as energy management of multi-stack fuel cell hybrid trams.
Jinling Ma, Jiye Zhang· International journal of pat...· 0 citations
Many operational problems are constrained sequential decision processes with large, combinatorial action spaces and interdependent feasibility constraints. Mixed-integer linear programs (MILPs) handle such constraints flexibly but scale poorly in stochastic environments. Deep reinforcement learning (DRL) promises scalable decision rules, but existing methods either penalize constraints rather than enforce them, or rely on feasibility mechanisms that break down once constraints interact. We bridge this gap by embedding a differentiable convex optimization module inside the policy: a neural network proposes continuous action targets, a quadratic program projects them onto the relaxed feasible set, and a dual-informed integer mapping restores integrality while preserving feasibility. Given a differentiable simulator, the policy trains end to end from sampled trajectories using pathwise gradients, while handling hard constraints with similar flexibility to MILPs. We show that our feasibility enforcement has bounded error relative to an exact integer projection and ensures the entire feasible action space is reachable. We apply the method to multi-echelon production-inventory planning under shared resource and material constraints. Our policy attains an average optimality gap below 1% on small instances. It further outperforms state-of-the-art echelon base-stock policies by up to 9.75% and a rolling-horizon multi-stage stochastic program by at least 7.7% in larger networks. On an industry-scale case study from ASML, it reduces average cost by up to 3.22% relative to the best-known benchmark policy. The savings are largest where planning is hardest: in tightly capacitated systems with high demand variability. More broadly, our work shows that DRL can deliver economically significant savings in sequential decision problems with interdependent hard constraints, which are widespread in practice.
Patrick Helm, John Doerr, Joren Gijsbrechts et al.· 0 citations
This paper finds that solutions produced at the early search stage of MILP solvers are often structurally close to the solutions found after full-budget search, and proposes a new solver-informed paradigm that shifts the learning target from variable assignment to early-to-final consistency.
Guanli Li, Chengrui Gao, Chenguang Wang et al.· 0 citations
Large traveling salesman problem (TSP) instances require a solver to allocate limited computation while preserving the validity of its outputs. Existing neural--operations-research (OR) hybrids predict guidance without requiring learned transitions to satisfy constraints discovered during search. DualCert introduces \emph{constraint-coupled learning}, in which current degree equations and dynamically separated subtour-elimination constraints (SECs) define each learned transition. At each refinement, the degree equations and selected, strictly satisfied SEC equations, with positive slacks, define an iterate-dependent primal-slack Karush--Kuhn--Tucker (KKT) manifold. Repaired dual variables and violated SEC rows define a local cost field. An exact constrained mirror-descent step maps each finite state to a positive state on the same manifold. Where selected rows and deterministic ties remain fixed, implicit differentiation maps parameter perturbations into the manifold tangent space and reuses the forward constraint operator for the local-cost-field derivative. The terminal edge state allocates computation across Held--Karp ascent, candidate-graph edge tests, and tour construction under a fixed budget. Deterministic verification recomputes original costs and accepts only verified candidate-graph lower bounds and edge decisions. On 1,000 held-out TSP1000 instances, DualCert attains a mean tour-cost gap of \(0.0573\%\) from Lin--Kernighan--Helsgaun version 3 (LKH-3) reference tours in \(9.55\) batch-amortized seconds per instance. It returns a verified candidate-graph lower bound for every instance and achieves \(81.46\%\) edge-decision coverage. The mean gap is \(67.1\%\) smaller than the reported NeuroLKH mean gap. Thus, optimization constraints govern learning, while deterministic verification preserves output validity.
Yancheng Song, Yong-Zhi Qi, Wei Qi et al.· 0 citations
This paper proposes a novel learning-based approach to approximately solve instances of mixed-integer optimization problems. These problems are computationally challenging, as they require jointly determining discrete and continuous decisions while satisfying complex combinatorial constraints. The proposed method relies on a graph-based generative diffusion model that learns the discrete component of mixed-integer optimization problems while integrating a training-free feasibility projection operator directly into the reverse diffusion process to steer intermediate samples toward the feasible set throughout generation. Once the discrete decisions are generated, the remaining optimization reduces to a continuous problem that can be solved efficiently (relative to the original problem) using existing numerical methods. The resulting framework named Constrained Graph Diffusion (CGD), is problem-agnostic and can accommodate a broad class of mixed-integer optimization problems through suitable projection operators. We evaluate CGD on optimal transmission switching for ACOPF and discrete portfolio optimization, demonstrating substantial improvements in feasibility and solution quality over learning-based baselines while achieving speedups of up to $425\times$ over state-of-the-art numerical solvers for MINLPs.
Vincenzo Di Vito, Mehdi Taghizadeh, D. Deka et al.· 0 citations
Sequential reinforcement learning (RL) solvers and global diffusion model (DM) solvers for neural combinatorial optimization exhibit complementary failure modes under an optimization-regret view. The former enjoys small marginal regret in the early construction stage, but suffers from horizon-wise compounding errors with super-linear regret growth; the latter avoids horizon compounding but incurs linear or sublinear regret w.r.t. the dimension of the remaining unsolved subspace. We propose Hybrid Neural Solver for Combinatorial Optimization (HyCO), a hybrid inference algorithm that constructs a solution prefix with an RL solver and adaptively switches to a conditional DM to complete the remaining decisions. To characterize why such hybridization helps, when to trigger the handover, and how to realize it in practice, we first develop a unified error-scaling theoretical framework and prove that, under explicit error-scaling assumptions, i) the hybrid structure achieves strictly lower expected regret than either backbone alone, and ii) there exists a unique optimal trigger step that minimizes the hybrid regret. We then design a lightweight adaptive trigger that combines policy entropy and RL-DM disagreement to detect trajectory-level signals of the regime shift as a practical proxy, since the optimal trigger step is defined at the expected-regret level and is not directly computable on individual trajectories. Experimental results on diverse benchmarks demonstrate that HyCO achieves consistent improvements over both backbones and support the empirical effectiveness of adaptive triggering.
Yuheng Li, Di Yang, Haipeng Chen et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
