Name
Abstract | ||
|---|---|---|
Analyzing 5,665 crowd ratings on 1,133 social media comments, we find that individuals tend to agree on the extremes of a hate rating scale more than in the middle when evaluating the hatefulness of online comments. The agreement is higher for less hateful comments and lowest on moderately hateful comments. The results have implications for researchers developing machine learning models for online hate processing, as the extreme classes are likely to require fewer annotations for reaching statistical stability. Our findings suggest that the models developed in this domain should consider the distributions of hate ratings rather than average hate scores.
|
Year | DOI | Venue |
|---|---|---|
2019 | 10.1145/3295750.3298954 | conference on human information interaction and retrieval |
Keywords | Field | DocType |
Online hate, toxicity, ratings, interpretation, crowdsourcing | Stability (probability),Social media,Information retrieval,Computer science,Crowdsourcing,Cognitive psychology,Rating scale,Statistical analysis | Conference |
ISBN | Citations | PageRank |
978-1-4503-6025-8 | 0 | 0.34 |
References | Authors | |
13 | 5 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Joni Salminen | 1 | 44 | 24.54 |
| Hind Almerekhi | 2 | 17 | 3.81 |
| Ahmed Mohamed Kamel | 3 | 0 | 0.34 |
| Soon-Gyo Jung | 4 | 57 | 21.83 |
| Bernard J. Jansen | 5 | 4753 | 394.06 |