Paper Info
Title
Automatic evaluation of world wide web search services
Abstract
Users of the World-Wide Web are not only confronted by an immense overabundance of information, but also by a plethora of tools for searching for the web pages that suit their information needs. Web search engines differ widely in interface, features, coverage of the web, ranking methods, delivery of advertising, and more. In this paper, we present a method for comparing search engines automatically based on how they rank known item search results. Because the engines perform their search on overlapping (but different) subsets of the web collected at different points in time, evaluation of search engines poses significant challenges to the traditional information retrieval methodology. Our method uses known item searching; comparing the relative ranks of the items in the search engines' rankings. Our approach automatically constructs known item queries using query log analysis and automatically constructs the result via analysis of editor comments from the ODP (Open Directory Project). Additionally, we present our comparison on five (Lycos, Netscape, Fast, Google, HotBot) well-known search services and find that some services perform known item searches better than others, but the majority are statistically equivalent.
Year
DOI
Venue
2002
10.1145/564376.564474
SIGIR
Keywords
Field
DocType
well-known search service,traditional information retrieval methodology,web page,different point,automatic evaluation,information need,ranking method,query log analysis,item search result,search engine,web search engine,world wide web search,mean reciprocal rank,world wide web,web pages,information retrieval
Web search engine,Web search query,Data mining,World Wide Web,Metasearch engine,Web page,Information retrieval,Semantic search,Computer science,Web query classification,Web crawler,Online search
Conference
ISBN
Citations 
PageRank 
1-58113-561-0
29
2.45
References 
Authors
4
2
Name
Order
Citations
PageRank
Abdur Chowdhury12013160.59
Ian Soboroff21907218.39