Paper Info
Title
An analysis of time-instability in web search results
Abstract
Due to the dynamic nature of web and the complex architectures of modern commercial search engines, top results in major search engines can change dramatically over time. Our experimental data shows that, for all three major search engines (Google, Bing and Yahoo!), approximately 90% of queries have their top 10 results altered within a period of ten days. Although this instability is expected in some situations such as in news-related queries, it is problematic in general because it can dramatically affect retrieval performance measurements and negatively affect users' perception of search quality (for instance, when users cannot re-find a previously found document). In this work we present the first large scale study on the degree and nature of these changes. We introduce several types of query instability, and several metrics to quantify it.We then present a quantitative analysis using 12,600 queries collected from a commercial web search engine over several weeks. Our analysis shows that the results from all major search engines have similar levels of instability, and that many of these changes are temporary. We also identified classes of queries with clearly different instability profiles - for instance, navigational queries are considerably more stable than non-navigational, while longer queries are significantly less stable than shorter ones.
Year
DOI
Venue
2011
10.1007/978-3-642-20161-5_46
ECIR
Keywords
Field
DocType
web search result,modern commercial search engine,major search engine,quantitative analysis,search quality,complex architecture,commercial web search engine,top result,dynamic nature,query instability,different instability profile,web search engine,search engine,negative affect
Web search engine,Data mining,Web search query,Metasearch engine,Organic search,Information retrieval,Semantic search,Computer science,Search engine indexing,Web query classification,Search analytics
Conference
Volume
ISSN
Citations 
6611
0302-9743
4
PageRank 
References 
Authors
0.47
5
2
Name
Order
Citations
PageRank
Jinyoung Kim11085.56
Vitor R. Carvalho267236.38