Title
Learning to Account for Good Abandonment in Search Success Metrics
Abstract
Abandonment in web search has been widely used as a proxy to measure user satisfaction. Initially it was considered a signal of dissatisfaction, however with search engines moving towards providing answer-like results, a new category of abandonment was introduced and referred to as Good Abandonment. Predicting good abandonment is a hard problem and it was the subject of several previous studies. All those studies have focused, though, on predicting good abandonment in offline settings using manually labeled data. Thus, it remained a challenge how to have an online metric that accounts for good abandonment. In this work we describe how a search success metric can be augmented to account for good abandonment sessions using a machine learned metric that depends on user’s viewport information. We use real user traffic from millions of users to evaluate the proposed metric in an A/B experiment. We show that taking good abandonment into consideration has a significant effect on the overall performance of the online metric.
Year
DOI
Venue
2016
10.1145/2983323.2983867
ACM International Conference on Information and Knowledge Management
Keywords
Field
DocType
good abandonment,search,metric
Proxy (climate),Data mining,Search engine,Information retrieval,Viewport,Computer science,Labeled data
Conference
ISBN
Citations 
PageRank 
978-1-4503-4073-1
1
0.35
References 
Authors
13
6
Name
Order
Citations
PageRank
Madian Khabsa123718.81
Aidan Crook210.35
Ahmed Hassan394357.64
Imed Zitouni411.02
Tasos Anastasakos541256.01
Kyle Williams620821.61