Paper Info
Title
Second-order Co-occurrence Sensitivity of Skip-Gram with Negative Sampling.
Abstract
We simulate first- and second-order context overlap and show that Skip-Gram with Negative Sampling is similar to Singular Value Decomposition in capturing second-order co-occurrence information, while Pointwise Mutual Information is agnostic to it. We support the results with an empirical study finding that the models react differently when provided with additional second-order information. Our findings reveal a basic property of Skip-Gram with Negative Sampling and point towards an explanation of its success on a variety of tasks.
Year
DOI
Venue
2019
10.18653/v1/w19-4803
BLACKBOXNLP WORKSHOP ON ANALYZING AND INTERPRETING NEURAL NETWORKS FOR NLP AT ACL 2019
DocType
Volume
Citations 
Journal
abs/1906.02479
0
PageRank 
References 
Authors
0.34
0
3
Name
Order
Citations
PageRank
Dominik Schlechtweg127.85
Cennet Oguz200.34
Sabine Schulte im Walde344065.65