Title
Evaluating the performance and neutrality/bias of search engines
Abstract
Different search engines provide different outputs for the same keyword. This may be due to different definitions of relevance, to different ranking aggregation methods, and/or to different knowledge/anticipation of users' preferences, but rankings are also suspected to be biased towards own content, which may prejudicial to other content providers. In this paper, we make some initial steps toward a rigorous comparison and analysis of search engines, by proposing a definition for a consensual relevance of a page with respect to a keyword, from a set of search engines. More specifically, we look at the results of several search engines for a sample of keywords, and define for each keyword the visibility of a page based on its ranking over all search engines. This allows to define a score of the search engine for a keyword, and then its average score over all keywords. Based on the pages visibility, we can also define the consensus search engine as the one showing the most visible results for each keyword, and discuss how biased results toward specific pages can be highlighted and quantified to provide answers to the search neutrality debate. We have implemented this model and present an analysis of the results.
Year
DOI
Venue
2019
10.1145/3306309.3306325
Proceedings of the 12th EAI International Conference on Performance Evaluation Methodologies and Tools
Keywords
Field
DocType
Search engines, consensus, search bias, search neutrality
Visibility,Search engine,Information retrieval,Ranking,Search neutrality,Computer science,Neutrality,Distributed computing
Conference
ISBN
Citations 
PageRank 
978-1-4503-6596-3
0
0.34
References 
Authors
0
3
Name
Order
Citations
PageRank
Ahmed Kamoun100.34
Patrick Maillé228243.33
Bruno Tuffin378987.60