Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Jul 27;15(1):27315.
doi: 10.1038/s41598-025-13010-6.

Adaptive distributed stochastic deep reinforcement learning control for voltage and frequency restoration in islanded AC microgrids with communication noise and delay

Nima Mahdian Dehkordi  1 , Vahab Nekoukar  2
Affiliations

Adaptive distributed stochastic deep reinforcement learning control for voltage and frequency restoration in islanded AC microgrids with communication noise and delay

Nima Mahdian Dehkordi et al. Sci Rep. .

Abstract

This paper proposes an adaptive secondary control strategy for islanded AC microgrids (MGs) using Distributed Stochastic Deep Reinforcement Learning (DSDRL), targeting reliable frequency and voltage restoration alongside proportional power sharing. Unlike existing distributed stochastic control methods that often fail under communication noise and variable delays, our approach dynamically adapts to these uncertainties by integrating a control Lyapunov function (CLF)-based nominal model with a deep deterministic policy gradient (DDPG) algorithm. Distinct from prior DRL applications, our method rigorously couples Lyapunov-based stability guarantees with DRL-based gain tuning, ensuring formal stability despite stochastic disturbances and model uncertainties. This hybrid framework facilitates real-time optimization of control policies, contributing to improved resilience and robustness under realistic network conditions. Furthermore, the controller's design is independent of precise distributed generator (DG) parameters, supporting heterogeneous and uncertain systems. MATLAB/SimPowerSystems simulations demonstrate the proposed method's improved performance compared to existing techniques across a range of scenarios involving communication noise, time delays, and load variations. The proposed framework offers a theoretically grounded, scalable, and adaptable solution for secondary control of MGs, paving the way for future extensions to larger systems and secure control applications.

Keywords: Deep deterministic policy gradient (DDPG); distributed generations (DGs); distributed stochastic deep reinforcement learning (DSDRL); microgrids (MGs); power-sharing; secondary control..

PubMed Disclaimer

Conflict of interest statement

Declarations. Competing interests: The authors declare no competing interests.

Figures

Fig. 1
Fig. 1
Block diagram of MG including N DGs.
Algorithm 1
Algorithm 1
DSDRL-based secondary control
Fig. 2
Fig. 2
Topology of MG and its communication graph.
Fig. 3
Fig. 3
Architecture of the actor and critic networks used in DRL agents.
Fig. 4
Fig. 4
Scenario A: trajectories performance of (a) frequencies, (b) real power ratios, and (c) output real powers of DGs under proposed DSDRL protocols with noise and time delay.
Fig. 5
Fig. 5
Scenario A: trajectories performance of (a) voltages and (b) output reactive powers of DGs under proposed DSDRL protocols with noise and time delay.
Fig. 6
Fig. 6
The reward curve of formula image.
Fig. 7
Fig. 7
Scenario B: trajectories performance of (a) frequencies, (b) real power ratios, (c) and output real powers of DGs under proposed DSDRL protocols with noise and time delay.
Fig. 8
Fig. 8
Scenario B: trajectories performance of (a) voltages, (b) output reactive powers, and (c) protocol gains of DGs under proposed DSDRL protocols with noise and time delay.
Fig. 9
Fig. 9
Scenario B: trajectories performance of (a) frequencies, (b) real power ratios, and (c) output real powers of DGs under protocols.
Fig. 10
Fig. 10
Scenario B: trajectories performance of (a) voltages and (b) output reactive powers of DGs under protocols.
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. Guerrero, J. M., Vasquez, J. C., Matas, J., de Vicuna, L. G. & Castilla, M. Hierarchical control of droop-controlled AC and DC microgrids—A general approach toward standardization. IEEE Trans. Ind. Electron.58(1), 158–172 (2011).
    1. Guerrero, J. M., Chandorkar, M., Lee, T.-L. & Loh, P. C. Advanced control architectures for intelligent microgrids—Part I: Decentralized and hierarchical control. IEEE Trans. Ind. Electron.60(4), 1254–1262 (2013).
    1. Sahoo, A. K. et al. Communication-less primary and secondary control in inverter-interfaced AC microgrid: An overview. IEEE Trans. Emerg. Sel. Top. Power Electron.9(5), 5164–5182 (2021).
    1. Poonahela, I. et al. Hierarchical model-predictive droop control for voltage and frequency restoration in AC microgrids. IEEE Open J. Ind. Electron. Soc.4, 85–97 (2023).
    1. Wang, Z., Wu, W. & Zhang, B. A distributed quasi-newton method for droop-free primary frequency control in autonomous microgrids. IEEE Trans. Smart Grid9(3), 2214–2223 (2018).

LinkOut - more resources