Paper Info
Title
Trust, but verify: predicting contribution quality for knowledge base construction and curation
Abstract
The largest publicly available knowledge repositories, such as Wikipedia and Freebase, owe their existence and growth to volunteer contributors around the globe. While the majority of contributions are correct, errors can still creep in, due to editors' carelessness, misunderstanding of the schema, malice, or even lack of accepted ground truth. If left undetected, inaccuracies often degrade the experience of users and the performance of applications that rely on these knowledge repositories. We present a new method, CQUAL, for automatically predicting the quality of contributions submitted to a knowledge base. Significantly expanding upon previous work, our method holistically exploits a variety of signals, including the user's domains of expertise as reflected in her prior contribution history, and the historical accuracy rates of different types of facts. In a large-scale human evaluation, our method exhibits precision of 91% at 80% recall. Our model verifies whether a contribution is correct immediately after it is submitted, significantly alleviating the need for post-submission human reviewing.
Year
DOI
Venue
2014
10.1145/2556195.2556227
WSDM
Keywords
Field
DocType
large-scale human evaluation,contribution quality,knowledge base construction,knowledge repository,knowledge base,method holistically,different type,historical accuracy rate,accepted ground truth,available knowledge repository,prior contribution history,new method,crowdsourcing
Data science,Data mining,Globe,Information retrieval,Computer science,Crowdsourcing,Carelessness,Exploit,Knowledge base,Recall,Malice,Schema (psychology)
Conference
Citations 
PageRank 
References 
7
0.57
32
Authors
4
Name
Order
Citations
PageRank
Chun How Tan1502.53
Eugene Agichtein24549269.70
Panagiotis G. Ipeirotis34528270.32
Evgeniy Gabrilovich44573224.48