Name
Abstract | ||
|---|---|---|
Today when many practitioners run basic NLP on the entire web and large-volume traffic, faster methods are paramount to saving time and energy costs. Recent advances in GPU hardware have led to the emergence of bi-directional LSTMs as a standard method for obtaining per-token vector representations serving as input to labeling tasks such as NER (often followed by prediction in a linear-chain CRF). Though expressive and accurate, these models fail to fully exploit GPU parallelism, limiting their computational efficiency. This paper proposes a faster alternative to Bi-LSTMs for NER: Iterated Dilated Convolutional Neural Networks (ID-CNNs), which have better capacity than traditional CNNs for large context and structured prediction. Unlike LSTMs whose sequential processing on sentences of length N requires O(N) time even in the face of parallelism, ID-CNNs permit fixed-depth convolutions to run in parallel across entire documents. We describe a distinct combination of network structure, parameter sharing and training procedures that enable dramatic 14-20x test-time speedups while retaining accuracy comparable to the Bi-LSTM-CRF. Moreover, ID-CNNs trained to aggregate context from the entire document are even more accurate while maintaining 8x faster test time speeds. |
Year | DOI | Venue |
|---|---|---|
2017 | 10.18653/v1/D17-1283 | empirical methods in natural language processing |
Field | DocType | Volume |
Convolutional neural network,Convolution,Computer science,Structured prediction,Exploit,Artificial intelligence,Iterated function,Machine learning,Limiting,Network structure | Conference | D17-1 |
Citations | PageRank | References |
28 | 0.76 | 27 |
Authors | ||
4 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Emma Strubell | 1 | 82 | 7.10 |
| Patrick Verga | 2 | 97 | 9.11 |
| David Belanger | 3 | 192 | 8.82 |
| Andrew Kachites McCallumzy | 4 | 19203 | 1588.22 |