Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Oct 11:17:1285831.
doi: 10.3389/fnbot.2023.1285831. eCollection 2023.

Research on automatic pilot repetition generation method based on deep reinforcement learning

Weijun Pan  1 , Peiyuan Jiang  1 , Yukun Li  1 , Zhuang Wang  1 , Junxiang Huang  2
Affiliations

Research on automatic pilot repetition generation method based on deep reinforcement learning

Weijun Pan et al. Front Neurorobot. .

Abstract

Using computers to replace pilot seats in air traffic control (ATC) simulators is an effective way to improve controller training efficiency and reduce training costs. To achieve this, we propose a deep reinforcement learning model, RoBERTa-RL (RoBERTa with Reinforcement Learning), for generating pilot repetitions. RoBERTa-RL is based on the pre-trained language model RoBERTa and is optimized through transfer learning and reinforcement learning. Transfer learning is used to address the issue of scarce data in the ATC domain, while reinforcement learning algorithms are employed to optimize the RoBERTa model and overcome the limitations in model generalization caused by transfer learning. We selected a real-world area control dataset as the target task training and testing dataset, and a tower control dataset generated based on civil aviation radio land-air communication rules as the test dataset for evaluating model generalization. In terms of the ROUGE evaluation metrics, RoBERTa-RL achieved significant results on the area control dataset with ROUGE-1, ROUGE-2, and ROUGE-L scores of 0.9962, 0.992, and 0.996, respectively. On the tower control dataset, the scores were 0.982, 0.954, and 0.982, respectively. To overcome the limitations of ROUGE in this field, we conducted a detailed evaluation of the proposed model architecture using keyword-based evaluation criteria for the generated repetition instructions. This evaluation criterion calculates various keyword-based metrics based on the segmented results of the repetition instruction text. In the keyword-based evaluation criteria, the constructed model achieved an overall accuracy of 98.8% on the area control dataset and 81.8% on the tower control dataset. In terms of generalization, RoBERTa-RL improved accuracy by 56% compared to the model before improvement and achieved a 47.5% improvement compared to various comparative models. These results indicate that employing reinforcement learning strategies to enhance deep learning algorithms can effectively mitigate the issue of poor generalization in text generation tasks, and this approach holds promise for future application in other related domains.

Keywords: controller training; generalization; reinforcement learning; text generation; transfer learning.

PubMed Disclaimer

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Figures

Figure 1
Figure 1
ATCOs training process.
Figure 2
Figure 2
Core technologies of automatic pilot seat.
Figure 3
Figure 3
The proposed DRL architecture.
Figure 4
Figure 4
UI Interface of the tower control instruction generator.
Figure 5
Figure 5
Distribution of tower control dataset and area control dataset.
Figure 6
Figure 6
RoBERTa-RL (λ = 0) PRG text length distribution.
Figure 7
Figure 7
Reference label length distribution.
Figure 8
Figure 8
RoBERTa-RL (λ=0.5) PRG text length distribution.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Alexandr N., Irina O., Tatyana K., Inessa K., Arina P. (2021). Fine-tuning GPT-3 for Russian text summarization, in Data Science and Intelligent Systems: Proceedings of 5th Computational Methods in Systems and Software 2021 (Springer: ), 748–757.
    1. de Souza J. G., Kozielski M., Mathur P., Chang E., Guerini M., Negri M., et al. . (2018). Generating e-commerce product titles and predicting their quality, in Proceedings of the 11th International Conference on Natural Language Generation (IOP Publishing: ), 233–243.
    1. Devlin J., Chang M.-W., Lee K., Toutanova K. (2018). BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
    1. Drayton J., Coxhead A. (2023). The development, evaluation and application of an aviation radiotelephony specialised technical vocabulary list. English Specific Purposes 69, 51–66. 10.1016/j.esp.2022.10.001 - DOI
    1. Elmadani K. N., Elgezouli M., Showk A. (2020). Bert fine-tuning for Arabic text summarization. arXiv preprint arXiv:2004.14135.