Paper Info
Title
On the effectiveness of the scientific peer-review system: a case study of the Journal of High Energy Physics
Abstract
The importance and the need for the peer-review system is highly debated in the academic community, and recently there has been a growing consensus to completely get rid of it. This is one of the steps in the publication pipeline that usually requires the publishing house to invest a significant portion of their budget in order to ensure quality editing and reviewing of the submissions received. Therefore, a very pertinent question is if at all such investments are worth making. To answer this question, in this paper, we perform a rigorous measurement study on a massive dataset (29k papers with 70k distinct review reports) to unfold the detailed characteristics of the peer-review process considering the three most important entities of this process—(i) the paper (ii) the authors and (iii) the referees and thereby identify different factors related to these three entities which can be leveraged to predict the long-term impact of a submitted paper. These features when plugged into a regression model achieve a high \(R^2\) of 0.85 and RMSE of 0.39. Analysis of feature importance indicates that reviewer- and author-related features are most indicative of long-term impact of a paper. We believe that our framework could definitely be utilized in assisting editors to decide the fate of a paper.
Year
DOI
Venue
2020
10.1007/s00799-018-0247-9
International Journal on Digital Libraries
Keywords
DocType
Volume
Peer-review system, Citation, Publication
Journal
21
Issue
ISSN
Citations 
2
1432-1300
0
PageRank 
References 
Authors
0.34
12
5
Name
Order
Citations
PageRank
Sandipan Sikdar1504.86
Paras Tehria200.34
Matteo Marsili314917.65
Niloy Ganguly41306121.03
Animesh Mukherjee539262.78