Name
Abstract | ||
|---|---|---|
Natural Language Processing (NLP) systems commonly leverage bag-of-words co-occurrence techniques to capture semantic and syntactic word relationships. The resulting word-level distributed representations often ignore morphological information, though character-level embeddings have proven valuable to NLP tasks. We propose a new neural language model incorporating both word order and character order in its embedding. The model produces several vector spaces with meaningful substructure, as evidenced by its performance of 85.8% on a recent word-analogy task, exceeding best published syntactic word-analogy scores by a 58% error margin (Pennington et al., 2014). Furthermore, the model includes several parallel training methods, most notably allowing a skip-gram network with 160 billion parameters to be trained overnight on 3 multi-core CPUs, 14x larger than the previous largest neural network (Coates et al., 2013). |
Year | Venue | DocType |
|---|---|---|
2015 | International Conference on Machine Learning | Journal |
Volume | Citations | PageRank |
abs/1506.02338 | 6 | 0.50 |
References | Authors | |
20 | 3 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| andrew trask | 1 | 26 | 2.54 |
| David Gilmore | 2 | 8 | 0.88 |
| Matthew Russell | 3 | 6 | 0.50 |