Title
Pre-training Methods in Information Retrieval.
Abstract
The core of information retrieval (IR) is to identify relevant information from large-scale resources and return it as a ranked list to respond to user's information need. Recently, the resurgence of deep learning has greatly advanced this field and leads to a hot topic named NeuIR (i.e., neural information retrieval), especially the paradigm of pre-training methods (PTMs). Owing to sophisticated pre-training objectives and huge model size, pre-trained models can learn universal language representations from massive textual data, which are beneficial to the ranking task of IR. Since there have been a large number of works dedicating to the application of PTMs in IR, we believe it is the right time to summarize the current status, learn from existing methods, and gain some insights for future development. In this survey, we present an overview of PTMs applied in different components of IR system, including the retrieval component, the re-ranking component, and other components. In addition, we also introduce PTMs specifically designed for IR, and summarize available datasets as well as benchmark leaderboards. Moreover, we discuss some open challenges and envision some promising directions, with the hope of inspiring more works on these topics for future research.
Year
DOI
Venue
2022
10.1561/1500000100
Foundations and Trends in Information Retrieval
DocType
Volume
Issue
Journal
16
3
ISSN
Citations 
PageRank 
1554-0669
0
0.34
References 
Authors
0
9
Name
Order
Citations
PageRank
Yixing Fan120219.39
Xiaohui Xie2407.49
Yinqiong Cai301.35
Jia Chen4224.16
Xinyu Ma581.53
Xiangsheng Li6253.84
Ruqing Zhang71510.40
Jiafeng Guo81737102.17
Yiqun Liu91592136.51