Paper Info
Title
Context-aware result inference in crowdsourcing.
Abstract
Many result inference methods have been proposed to address the quality-control problem in crowdsourcing. However, existing methods are ineffective for context-sensitive tasks (CSTs), e.g., handwriting recognition, translation, speech transcription, where context correlation within a task cannot be ignored for two reasons. Firstly, it is ineffective to crowdsource a whole CST (e.g., recognizing handwritten texts) and use task-level inference methods to infer the answer, because it is rather hard to correctly complete a whole complicated task. Secondly, although a CST is composed of a set of atomic subtasks (e.g., recognizing a handwritten word), it is unsuitable to split it into multiple subtasks and adopt a subtask-level inference algorithm to infer the result, because this will lose the context correlation (e.g., phrases) among subtasks and increase the difficulty to complete a task. Thus it calls for a new approach to handling CSTs. In this work, we study the result inference problem for CSTs and propose a context-aware inference algorithm. We design an inference algorithm by incorporating the context information. Furthermore, we introduce an iterative method to improve the quality. The results of experiments on real-world CSTs demonstrated the superiority of our approach compared with the state-of-the-art methods.
Year
DOI
Venue
2018
10.1016/j.ins.2018.05.050
Information Sciences
Keywords
Field
DocType
Crowdsourcing,Human computation,Quality control,Context-sensitive tasks
Speech transcription,Inference,Crowdsourcing,Iterative method,Handwriting recognition,Crowdsource,Correlation,Artificial intelligence,Machine learning,Mathematics
Journal
Volume
ISSN
Citations 
460
0020-0255
2
PageRank 
References 
Authors
0.35
34
5
Name
Order
Citations
PageRank
Yili Fang1394.41
Hailong Sun268064.83
Guoliang Li33077154.70
Richong Zhang423239.67
Jinpeng Huai51187130.18