Name
Title | ||
|---|---|---|
Using manually-built web directories for automatic evaluation of known-item retrieval |
Abstract | ||
|---|---|---|
Information retrieval system evaluation is complicated by the need for manually assessed relevance judgments. Large manually-built directories on the web open the door to new evaluation procedures. By assuming that web pages are the known relevant items for queries that exactly match their title, we use the ODP (Open Directory Project) and Looksmart directories for system evaluation. We test our approach with a sample from a log of ten million web queries and show that such an evaluation is unbiased in terms of the directory used, stable with respect to the query set selected, and correlated with a reasonably large manual evaluation. |
Year | DOI | Venue |
|---|---|---|
2003 | 10.1145/860435.860507 | SIGIR |
Keywords | Field | DocType |
known-item retrieval,looksmart directory,large manually-built directory,web page,million web query,information retrieval system evaluation,automatic evaluation,open directory project,new evaluation procedure,system evaluation,manually-built web directory,large manual evaluation,relevance judgment,information retrieval system,web pages | Web search engine,IR evaluation,Information retrieval,Web page,Directory,Computer science,System evaluation,Database | Conference |
ISBN | Citations | PageRank |
1-58113-646-3 | 8 | 0.73 |
References | Authors | |
5 | 5 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Steven M. Beitzel | 1 | 696 | 46.72 |
| Eric C. Jensen | 2 | 696 | 46.72 |
| Abdur Chowdhury | 3 | 2013 | 160.59 |
| David Grossman | 4 | 525 | 34.73 |
| Ophir Frieder | 5 | 3300 | 419.55 |