Name
Title | ||
|---|---|---|
A Comparative Study of Pre-trained Encoders for Low-Resource Named Entity Recognition |
Abstract | ||
|---|---|---|
Pre-trained language models (PLM) are effective components of few-shot named entity recognition (NER) approaches when augmented with continued pre-training on task-specific out-of-domain data or fine-tuning on in-domain data. However, their performance in low-resource scenarios, where such data is not available, remains an open question. We introduce an encoder evaluation framework, and use it to systematically compare the performance of state-of-the-art pre-trained representations on the task of low-resource NER. We analyze a wide range of encoders pre-trained with different strategies, model architectures, intermediate-task fine-tuning, and contrastive learning. Our experimental results across ten benchmark NER datasets in English and German show that encoder performance varies significantly, suggesting that the choice of encoder for a specific low-resource scenario needs to be carefully evaluated. |
Year | DOI | Venue |
|---|---|---|
2022 | 10.18653/v1/2022.repl4nlp-1.6 | PROCEEDINGS OF THE 7TH WORKSHOP ON REPRESENTATION LEARNING FOR NLP |
DocType | Volume | Citations |
Conference | Proceedings of the 7th Workshop on Representation Learning for NLP | 0 |
PageRank | References | Authors |
0.34 | 0 | 5 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Yuxuan Chen | 1 | 0 | 1.01 |
| Jonas Mikkelsen | 2 | 0 | 0.34 |
| Arne Binder | 3 | 0 | 0.34 |
| Christoph Alt | 4 | 3 | 4.10 |
| Leonhard Hennig | 5 | 72 | 10.62 |