Title
Reproducibility Of Experiments In Recommender Systems Evaluation
Abstract
Recommender systems evaluation is usually based on predictive accuracy metrics with better scores meaning recommendations of higher quality. However, the comparison of results is becoming increasingly difficult, since there are different recommendation frameworks and different settings in the design and implementation of the experiments. Furthermore, there might be minor differences on algorithm implementation among the different frameworks. In this paper, we compare well known recommendation algorithms, using the same dataset, metrics and overall settings, the results of which point to result differences across frameworks with the exact same settings. Hence, we propose the use of standards that should be followed as guidelines to ensure the replication of experiments and the reproducibility of the results.
Year
DOI
Venue
2018
10.1007/978-3-319-92007-8_34
ARTIFICIAL INTELLIGENCE APPLICATIONS AND INNOVATIONS, AIAI 2018
Keywords
Field
DocType
Recommender systems, Evaluation, Reproducibility, Replication
Recommender system,Reproducibility,Computer science,Artificial intelligence,Machine learning
Conference
Volume
ISSN
Citations 
519
1868-4238
1
PageRank 
References 
Authors
0.36
14
4
Name
Order
Citations
PageRank
Nikolaos Polatidis16411.42
Stelios Kapetanakis2159.79
Elias Pimenidis37920.59
Konstantinos Kosmidis410.36