Name
Title | ||
|---|---|---|
Neural Grammatical Error Correction Systems With Unsupervised Pre-Training On Synthetic Data |
Abstract | ||
|---|---|---|
Considerable effort has been made to address the data sparsity problem in neural grammatical error correction. In this work, we propose a simple and surprisingly effective unsupervised synthetic error generation method based on confusion sets extracted from a spellchecker to increase the amount of training data. Synthetic data is used to pre-train a Transformer sequence-to-sequence model, which not only improves over a strong baseline trained on authentic error-annotated data, but also enables the development of a practical GEC system in a scenario where little genuine error-annotated data is available. The developed systems placed first in the BEA19 shared task, achieving 69.47 and 64.24 F-0.5 in the restricted and low-resource tracks respectively, both on the W&I+LOCNESS test set. On the popular CoNLL 2014 test set, we report state-of-the-art results of 64.16 M-2 for the submitted system, and 61.30 M-2 for the constrained system trained on the NUCLE and Lang-8 data. |
Year | Venue | Field |
|---|---|---|
2019 | INNOVATIVE USE OF NLP FOR BUILDING EDUCATIONAL APPLICATIONS | Pattern recognition,Computer science,Error detection and correction,Synthetic data,Artificial intelligence |
DocType | Citations | PageRank |
Conference | 0 | 0.34 |
References | Authors | |
0 | 3 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Roman Grundkiewicz | 1 | 109 | 11.75 |
| Marcin Junczys-Dowmunt | 2 | 312 | 24.24 |
| Kenneth Heafield | 3 | 579 | 39.46 |