Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Feb 24;25(3):413.
doi: 10.3390/e25030413.

Deep Reinforcement Learning-Assisted Optimization for Resource Allocation in Downlink OFDMA Cooperative Systems

Mulugeta Kassaw Tefera  1 , Shengbing Zhang  1 , Zengwang Jin  1
Affiliations

Deep Reinforcement Learning-Assisted Optimization for Resource Allocation in Downlink OFDMA Cooperative Systems

Mulugeta Kassaw Tefera et al. Entropy (Basel). .

Abstract

This paper considers a downlink resource-allocation problem in distributed interference orthogonal frequency-division multiple access (OFDMA) systems under maximal power constraints. As the upcoming fifth-generation (5G) wireless networks are increasingly complex and heterogeneous, it is challenging for resource allocation tasks to optimize the system performance metrics and guarantee user service requests simultaneously. Because of the non-convex optimization problems, using existing approaches to find the optimal resource allocation is computationally expensive. Recently, model-free reinforcement learning (RL) techniques have become alternative approaches in wireless networks to solve non-convex and NP-hard optimization problems. In this paper, we study a deep Q-learning (DQL)-based approach to address the optimization of transmit power control for users in multi-cell interference networks. In particular, we have applied a DQL algorithm for resource allocation to maximize the overall system throughput subject to the maximum power and SINR constraints in a flat frequency channel. We first formulate the optimization problem as a non-cooperative game model, where the multiple BSs compete for spectral efficiencies by improving their achievable utility functions while ensuring the quality of service (QoS) requirements to the corresponding receivers. Then, we develop a DRL-based resource allocation model to maximize the system throughput while satisfying the power and spectral efficiency requirements. In this setting, we define the state-action spaces and the reward function to explore the possible actions and learning outcomes. The numerical simulations demonstrate that the proposed DQL-based scheme outperforms the traditional model-based solution.

Keywords: deep reinforcement learning; distributed optimization; game theory; power control; throughput maximization; wireless interference channel.

PubMed Disclaimer

Conflict of interest statement

The authors declare that there is no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

Figures

Figure 1
Figure 1
An illustration of downlink resource allocation for multi-cell and multiple user systems.
Figure 2
Figure 2
Reinforcement learning for multi-cell OFDMA systems.
Figure 3
Figure 3
Sum-rate vs. transmit power budget for different schemes.
Figure 4
Figure 4
Sum rate vs. power budget ignoring QoS.
Figure 5
Figure 5
The average sum rate vs. the number of user pairs.
See this image and copyright information in PMC

References

    1. Gesbert D., Hanly S., Huang H., Shitz S.S., Simeone O., Yu W. Multi-Cell MIMO Cooperative Networks: A New Look at Interference. IEEE J. Sel. Areas Commun. 2010;28:1380–1408. doi: 10.1109/JSAC.2010.101202. - DOI
    1. Chen S., Zhao T., Chen H.-H., Meng W. Network Densification and Path-Loss Models versus UDN Performance—A Unified Approach. IEEE Trans. Wirel. Commun. 2021;20:4058–4071. doi: 10.1109/TWC.2021.3055549. - DOI
    1. Chami M., Pischella M., Le Ruyet D. Resource allocation for OFDM-based multiuser cooperative underlay cognitive systems. EURASIP J. Wirel. Commun. Netw. 2017;2017:1–15. doi: 10.1186/s13638-017-0958-4. - DOI
    1. Venturino L., Prasad N., Wang X. Coordinated Scheduling and Power Allocation in Downlink Multicell OFDMA Networks. IEEE Trans. Veh. Technol. 2009;58:2835–2848. doi: 10.1109/TVT.2009.2013233. - DOI
    1. Shi Q., Razaviyayn M., Luo Z.Q., He C. An iteratively weighted MMSE approach to distributed sum-utility maximization for a mimo interfering broadcast channel. IEEE Trans. Signal Process. 2011;59:4331–4340. doi: 10.1109/TSP.2011.2147784. - DOI