Title | ||
---|---|---|
GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-supervised Learning and Explicit Policy Injection. |
Abstract | ||
---|---|---|
Pre-trained models have proved to be powerful in enhancing task-oriented dialog systems. However, current pre-training methods mainly focus on enhancing dialog understanding and generation tasks while neglecting the exploitation of dialog policy. In this paper, we propose GALAXY, a novel pre-trained dialog model that explicitly learns dialog policy from limited labeled dialogs and large-scale unlabeled dialog corpora via semi-supervised learning. Specifically, we introduce a dialog act prediction task for policy optimization during pre-training and employ a consistency regularization term to refine the learned representation with the help of unlabeled dialogs. We also implement a gating mechanism to weigh suitable unlabeled dialog samples. Empirical results show that GALAXY substantially improves the performance of task-oriented dialog systems, and achieves new state-of-the-art results on benchmark datasets: In-Car, MultiWOZ2.0 and MultiWOZ2.1, improving their end-to-end combined scores by 2.5, 5.3 and 5.5 points, respectively. We also show that GALAXY has a stronger few-shot ability than existing models under various low-resource settings. For reproducibility, we release the code and data at https://github.com/siat-nlp/GALAXY. |
Year | Venue | Keywords |
---|---|---|
2022 | AAAI Conference on Artificial Intelligence | Speech & Natural Language Processing (SNLP) |
DocType | Citations | PageRank |
Conference | 0 | 0.34 |
References | Authors | |
0 | 12 |
Name | Order | Citations | PageRank |
---|---|---|---|
Wanwei He | 1 | 2 | 1.83 |
Yinpei Dai | 2 | 0 | 1.35 |
Yinhe Zheng | 3 | 1 | 3.06 |
Yuchuan Wu | 4 | 0 | 0.68 |
Zheng Cao | 5 | 0 | 0.68 |
Dermot Liu | 6 | 0 | 0.34 |
Peng Jiang | 7 | 87 | 3.87 |
Min Yang | 8 | 77 | 20.41 |
Fei Huang | 9 | 506 | 56.44 |
Luo Si | 10 | 2498 | 169.52 |
Jian Sun | 11 | 0 | 2.70 |
Yongbin Li | 12 | 3 | 7.49 |