Name
Title | ||
|---|---|---|
A Latent Variable Model with Hierarchical Structure and GPT-2 for Long Text Generation |
Abstract | ||
|---|---|---|
Variational AutoEncoder (VAE) has made great achievements in the field of text generation. However, the current research mainly focuses on short texts, with little attention paid to long texts (more than 20 words). In this paper, we first propose a hidden-variable model based on the GPT-2 and hierarchical structure to generate long text. We use hierarchical GRU to encode long text to get hidden variables. At the same time, to generate the text better, we combine the hierarchical structure and GPT-2 in the decoder for the first time. Our model improves Perplexity (PPL), Kullback Leibler (KL) divergence, Bilingual Evaluation Understudy (BLEU) score, and Self-BLEU. The experiment indicates that the coherence and diversity of sentences generated by our model are better than the baseline model. |
Year | DOI | Venue |
|---|---|---|
2021 | 10.1007/978-3-030-86383-8_24 | ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING, ICANN 2021, PT V |
Keywords | DocType | Volume |
Latent variable, Text generation, VAE, Hierarchical structure, GPT-2 | Conference | 12895 |
ISSN | Citations | PageRank |
0302-9743 | 0 | 0.34 |
References | Authors | |
0 | 5 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Kun Zhao | 1 | 23 | 5.09 |
| H. Ding | 2 | 0 | 1.69 |
| Kai Ye | 3 | 0 | 1.69 |
| Xiaohui Cui | 4 | 374 | 44.64 |
| Zhongwang Fu | 5 | 0 | 0.34 |