Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Jul 2;21(13):4553.
doi: 10.3390/s21134553.

A Novel Approach to the Job Shop Scheduling Problem Based on the Deep Q-Network in a Cooperative Multi-Access Edge Computing Ecosystem

Junhyung Moon  1 , Minyeol Yang  1 , Jongpil Jeong  1
Affiliations

A Novel Approach to the Job Shop Scheduling Problem Based on the Deep Q-Network in a Cooperative Multi-Access Edge Computing Ecosystem

Junhyung Moon et al. Sensors (Basel). .

Abstract

In this study, based on multi-access edge computing (MEC), we provided the possibility of cooperating manufacturing processes. We tried to solve the job shop scheduling problem by applying DQN (deep Q-network), a reinforcement learning model, to this method. Here, to alleviate the overload of computing resources, an efficient DQN was used for the experiments using transfer learning data. Additionally, we conducted scheduling studies in the edge computing ecosystem of our manufacturing processes without the help of cloud centers. Cloud computing, an environment in which scheduling processing is performed, has issues sensitive to the manufacturing process in general, such as security issues and communication delay time, and research is being conducted in various fields, such as the introduction of an edge computing system that can replace them. We proposed a method of independently performing scheduling at the edge of the network through cooperative scheduling between edge devices within a multi-access edge computing structure. The proposed framework was evaluated, analyzed, and compared with existing frameworks in terms of providing solutions and services.

Keywords: cooperative scheduling system; deep Q-network; job shop scheduling problem; manufacturing process; multi-access edge computing.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
MEC framework.
Figure 2
Figure 2
Job shop scheduling problem.
Figure 3
Figure 3
Smart factory manufacturing process service provider’s cooperative scheduling solution framework.
Figure 4
Figure 4
Job shop scheduling problem.
Figure 5
Figure 5
Convergence analysis according to the number of M and J.
Figure 6
Figure 6
Convergence analysis according to the difference in parameter ϵ-diff.
Figure 7
Figure 7
Comparison of the proposed method (DQN with transfer learning) with the existing DQN.
Figure 8
Figure 8
Comparison of the proposed method and the acquisition reward value of the traditional DQN.
Figure 9
Figure 9
Comparison of the proposed method and the regression line of the conventional DQN.
See this image and copyright information in PMC

References

    1. Zhang J., Ding G., Zou Y., Qin S., Fu J. Review of job shop scheduling research and its new perspectives under Industry 4.0. J. Intell. Manuf. 2019;30:1809–1830. doi: 10.1007/s10845-017-1350-2. - DOI
    1. Luo S. Dynamic scheduling for flexible job shop with new job insertions by deep reinforcement learning. Appl. Soft Comput. 2020;91:106208. doi: 10.1016/j.asoc.2020.106208. - DOI
    1. Yu W., Liang F., He X., Hatcher W.G., Lu C., Lin J., Yang X. A survey on the edge computing for the Internet of Things. IEEE Access. 2017;6:6900–6919. doi: 10.1109/ACCESS.2017.2778504. - DOI
    1. Angelopoulos A., Michailidis E.T., Nomikos N., Trakadas P., Hatziefremidis A., Voliotis S., Zahariadis T. Tackling faults in the industry 4.0 era—A survey of machine-learning solutions and key aspects. Sensors. 2020;20:109. doi: 10.3390/s20010109. - DOI - PMC - PubMed
    1. Kuhnle A., Schäfer L., Stricker N., Lanza G. Design, implementation and evaluation of reinforcement learning for an adaptive order dispatching in job shop manufacturing systems. Procedia CIRP. 2019;81:234–239. doi: 10.1016/j.procir.2019.03.041. - DOI

LinkOut - more resources