Title
Training Deep Ranking Model With Weak Relevance Labels
Abstract
Deep neural networks have already achieved great success in a number of fields, for example, computer vision, natural language processing, speech recognition, and etc. However, such advances have not been observed in information retrieval (IR) tasks yet, such as ad-hoc retrieval. A potential explanation is that in a particular IR task, training a document ranker usually needs large amounts of relevance labels which describe the relationship between queries and documents. However, this kind of relevance judgments are usually very expensive to obtain. In this paper, we propose to train deep ranking models with weak relevance labels generated by click model based on real users' click behavior. We investigate the effectiveness of different weak relevance labels trained based on several major click models, such as DBN, RCM, PSCM, TCM, and UBM. The experimental results indicate that the ranking models trained with weak relevance labels are able to utilize large scale of behavior data and they can get similar performance compared to the ranking model trained based on relevance labels from external assessors, which are supposed to be more accurate. This preliminary finding encourages us to develop deep ranking models with weak supervised data.
Year
DOI
Venue
2017
10.1007/978-3-319-68155-9_16
DATABASES THEORY AND APPLICATIONS, ADC 2017
Keywords
Field
DocType
Ranking model, Click model, Deep learning
Data mining,Ranking,Computer science,Artificial intelligence,Deep learning,Machine learning,Click model,Deep neural networks
Conference
Volume
ISSN
Citations 
10538
0302-9743
4
PageRank 
References 
Authors
0.38
19
6
Name
Order
Citations
PageRank
Cheng Luo19412.58
Yukun Zheng2314.02
Jiaxin Mao316426.30
Yiqun Liu41592136.51
Min Zhang51658134.93
Shaoping Ma61544126.00