A Comprehensive Survey of Abstractive Text Summarization Based on Deep Learning

Abstract

With the rapid development of the Internet, the massive amount of web textual data has grown exponentially, which has brought considerable challenges to downstream tasks, such as document management, text classification, and information retrieval. Automatic text summarization (ATS) is becoming an extremely important means to solve this problem. The core of ATS is to mine the gist of the original text and automatically generate a concise and readable summary. Recently, to better balance and develop these two aspects, deep learning (DL)-based abstractive summarization models have been developed. At present, for ATS tasks, almost all state-of-the-art (SOTA) models are based on DL architecture. However, a comprehensive literature survey is still lacking in the field of DL-based abstractive text summarization. To fill this gap, this paper provides researchers with a comprehensive survey of DL-based abstractive summarization. We first give an overview of abstractive summarization and DL. Then, we summarize several typical frameworks of abstractive summarization. After that, we also give a comparison of several popular datasets that are commonly used for training, validation, and testing. We further analyze the performance of several typical abstractive summarization systems on common datasets. Finally, we highlight some open challenges in the abstractive summarization task and outline some future research trends. We hope that these explorations will provide researchers with new insights into DL-based abstractive summarization.

Figure 1

A general architecture of DL-based ABS. It is mainly composed of three steps: preprocessing, semantic understanding, and summary generation.

Figure 2

RNN timeline expansion diagram.

Figure 3

Framework of convolutional neural networks.

Figure 4

The  schematic diagram of GNN. The  basic idea of GNN is to embed nodes according to the local neighbourhoods.

Figure 5

The basic encoder-decoder framework. It consists of input layer, hidden layer, and output layer.

Figure 6

The basic encoder-decoder framework with attention mechanisms. The attention mechanism enables the decoder to interact with the input during the decoding process.

Figure 7

The basic hierarchical encoder-decoder architecture. It is mainly divided into sentence level and word level. The word level processes each word token, and the sentence level processes each sentence.

Figure 8

The CNN-based ABS model. It is the most representative ABS model based entirely on CNN.

Figure 9

The framework of the pointer softmax. It utilizes two softmax layers to predict the next generated words: one softmax to predict the location of the word in the source sentence and copy it as output, and the other to predict the word in the shortlist vocabulary.

Figure 10

The framework of the PG model. It utilizes a pointer to copy words from the input document, which helps to accurately reproduce the information while retaining the ability to generate new tokens through the generator.

Figure 11

The overall framework of the FTSum model. It is a dual-attention encoder-decoder model.

Figure 12

The overall framework of the Entailment-aware encoder-decoder model. It uses the attention-based encoder-decoder framework as the infrastructure, and then shares the encoder with the entailment recognition system.

Figure 13

The overall framework of the FASum model. Its encoder and decoder are stacked by Transformer blocks.

Figure 14

The overall framework of the FAR-ASS model.