Title
Consistent Position Bias Estimation without Online Interventions for Learning-to-Rank.
Abstract
Presentation bias is one of the key challenges when learning from implicit feedback in search engines, as it confounds the relevance signal with uninformative signals due to position in the ranking, saliency, and other presentation factors. While it was recently shown how counterfactual learning-to-rank (LTR) approaches cite{Joachims/etal/17a} can provably overcome presentation bias if observation propensities are known, it remains to show how to accurately estimate these propensities. In this paper, we propose the first method for producing consistent propensity estimates without manual relevance judgments, disruptive interventions, or restrictive relevance modeling assumptions. We merely require that we have implicit feedback data from multiple different ranking functions. Furthermore, we argue that our estimation technique applies to an extended class of Contextual Position-Based Propensity Models, where propensities not only depend on position but also on observable features of the query and document. Initial simulation studies confirm that the approach is scalable, accurate, and robust.
Year
Venue
Field
2018
arXiv: Learning
Learning to rank,Psychological intervention,Observable,Search engine,Ranking,Salience (neuroscience),Counterfactual thinking,Artificial intelligence,Machine learning,Mathematics,Scalability
DocType
Volume
Citations 
Journal
abs/1806.03555
0
PageRank 
References 
Authors
0.34
0
3
Name
Order
Citations
PageRank
Aman Agarwal1603.91
Ivan Zaitsev210.70
Thorsten Joachims3173871254.06