Paper Info
Title
Reviewer Bias In Single-Versus Double-Blind Peer Review
Abstract
Peer review may be "single-blind," in which reviewers are aware of the names and affiliations of paper authors, or "double-blind," in which this information is hidden. Noting that computer science research often appears first or exclusively in peer-reviewed conferences rather than journals, we study these two reviewing models in the context of the 10th Association for Computing Machinery International Conference on Web Search and Data Mining, a highly selective venue (15.6% acceptance rate) in which expert committee members review full-length submissions for acceptance. We present a controlled experiment in which four committee members review each paper. Two of these four reviewers are drawn from a pool of committee members with access to author information; the other two are drawn from a disjoint pool without such access. This information asymmetry persists through the process of bidding for papers, reviewing papers, and entering scores. Reviewers in the single-blind condition typically bid for 22% fewer papers and preferentially bid for papers from top universities and companies. Once papers are allocated to reviewers, single-blind reviewers are significantly more likely than their double-blind counterparts to recommend for acceptance papers from famous authors, top universities, and top companies. The estimated odds multipliers are tangible, at 1.63, 1.58, and 2.10, respectively.
Year
DOI
Venue
2017
10.1073/pnas.1707323114
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA
Keywords
Field
DocType
peer review, double-blind, scientific method
Information asymmetry,Public relations,Computer science,Acceptance rate,Controlled experiment,Odds,Bidding
Journal
Volume
Issue
ISSN
114
48
0027-8424
Citations 
PageRank 
References 
13
1.63
9
Authors
3
Name
Order
Citations
PageRank
Andrew Tomkins193881401.23
Min Zhang21658134.93
William D Heavlin3131.97