Name
Abstract | ||
|---|---|---|
Existing unbiased learning-to-rank models use counterfactual inference, notably Inverse Propensity Scoring (IPS), to learn a ranking function from biased click data. They handle the click incompleteness bias, but usually assume that the clicks are noise-free, i.e., a clicked document is always assumed to be relevant. In this paper, we relax this unrealistic assumption and study click noise explicitly in the unbiased learning-to-rank setting. Specifically, we model the noise as the position-dependent trust bias and propose a noise-aware Position-Based Model, named TrustPBM, to better capture user click behavior. We propose an Expectation-Maximization algorithm to estimate both examination and trust bias from click data in TrustPBM. Furthermore, we show that it is difficult to use a pure IPS method to incorporate click noise and thus propose a novel method that combines a Bayes rule application with IPS for unbiased learning-to-rank. We evaluate our proposed methods on three personal search data sets and demonstrate that our proposed model can significantly outperform the existing unbiased learning-to-rank methods.
|
Year | DOI | Venue |
|---|---|---|
2019 | 10.1145/3308558.3313697 | WWW '19: The Web Conference on The World Wide Web Conference WWW 2019 |
Keywords | Field | DocType |
Unbiased learning-to-rank, click noise, inverse propensity scoring, trust bias | Learning to rank,Data mining,Inverse,Data set,Ranking,Computer science,Inference,Counterfactual thinking,Artificial intelligence,Machine learning,Bayes' theorem | Conference |
ISBN | Citations | PageRank |
978-1-4503-6674-8 | 7 | 0.41 |
References | Authors | |
0 | 5 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Aman Agarwal | 1 | 60 | 3.91 |
| Xuanhui Wang | 2 | 1394 | 68.85 |
| Cheng Li | 3 | 28 | 3.07 |
| Michael Bendersky | 4 | 986 | 48.69 |
| Marc A. Najork | 5 | 2538 | 278.16 |