Name
Title | ||
|---|---|---|
Deep Learning-Based Language Identification in English-Hindi-Bengali Code-Mixed Social Media Corpora |
Abstract | ||
|---|---|---|
This article addresses language identification at the word level in Indian social media corpora taken from Facebook, Twitter and WhatsApp posts that exhibit code-mixing between English-Hindi, English-Bengali, as well as a blend of both language pairs. Code-mixing is a fusion of multiple languages previously mainly associated with spoken language, but which social media users also deploy when communicating in ways that tend to be rather casual. The coarse nature of code-mixed social media text makes language identification challenging. Here, the performance of deep learning on this task is compared to feature-based learning, with two Recursive Neural Network techniques, Long Short Term Memory (LSTM) and bidirectional LSTM, being contrasted to a Conditional Random Fields (CRF) classifier. The results show the deep learners outscoring the CRF, with the bidirectional LSTM demonstrating the best language identification performance. |
Year | DOI | Venue |
|---|---|---|
2019 | 10.1515/jisys-2017-0440 | JOURNAL OF INTELLIGENT SYSTEMS |
Keywords | Field | DocType |
Language identification,code-mixing,deep learning | Social media,Hindi,Computer science,Bengali,Language identification,Artificial intelligence,Deep learning,Linguistics,Code-mixing | Journal |
Volume | Issue | ISSN |
28 | SP3 | 0334-1860 |
Citations | PageRank | References |
0 | 0.34 | 0 |
Authors | ||
3 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Anupam Jamatia | 1 | 12 | 3.28 |
| Amitava Das | 2 | 198 | 42.49 |
| Björn Gambäck | 3 | 155 | 36.86 |