Title
Instance-optimality in the noisy value-and comparison-model: accept, accept, strong accept: which papers get in?
Abstract
Motivated by crowdsourced computation, peergrading, and recommendation systems, Braverman, Mao and Weinberg [7] studied the query and round complexity of fundamental problems such as finding the maximum (max), finding all elements above a certain value (threshold-v) or computing the top-k elements (Top-k) in a noisy environment. For example, consider the task of selecting papers for a conference. This task is challenging due to the crowdsourcing nature of peer reviews: the results of reviews are noisy and it is necessary to parallelize the review process as much as possible. We study the noisy value model and the noisy comparison model: In the noisy value model, a reviewer is asked to evaluate a single element: "What is the value of paper i?" (e.g., accept). In the noisy comparison model (introduced in the seminal work of Feige, Peleg, Raghavan and Upfal [17]) a reviewer is asked to do a pairwise comparison: "Is paper i better than paper j?" In this paper, we introduce new lower bound techniques for these classic problems. In comparison to previous work, our lower bounds are much more fine-grained: they focus on the interplay between round and query complexity and the dependency on the output size. In the setting of conference papers, this translates into a trade-off between number of reviews per paper and the number of review rounds necessary in order find the best 100 papers for the conference. We complement these results with simple algorithms which show that our lower bounds are almost tight. We then go beyond the worst-case and address the question of the importance of knowledge of the instance by providing, for a large range of parameters, instance-optimal algorithms with respect to the query complexity. We complement these results by showing that for some family of instances, no instance-optimal algorithm can exist.
Year
DOI
Venue
2018
10.5555/3381089.3381220
arXiv: Data Structures and Algorithms
Field
DocType
Volume
Recommender system,Round complexity,Pairwise comparison,Discrete mathematics,Combinatorics,Crowdsourcing,Upper and lower bounds,Mathematics,Computation
Journal
abs/1806.08182
Citations 
PageRank 
References 
0
0.34
15
Authors
3
Name
Order
Citations
PageRank
Vincent Cohen-Addad18525.47
Frederik Mallmann-Trenn23413.05
Claire Mathieu345225.78