Name
Abstract | ||
|---|---|---|
We consider the issue of evaluating information retrieval systems on the basis of a limited number of topics. In contrast to statistically-based work on sample sizes, we hypothesize that some topics or topic sets are better than others at predicting true system effectiveness, and that with the right choice of topics, accurate predictions can be obtained from small topics sets. Using a variety of effectiveness metrics and measures of goodness of prediction, a study of a set of TREC and NTCIR results confirms this hypothesis, and provides evidence that the value of a topic set for this purpose does generalize. |
Year | DOI | Venue |
|---|---|---|
2009 | 10.1145/1629096.1629099 | ACM Trans. Inf. Syst. |
Keywords | Field | DocType |
evaluation experiments,topic set,retrieval evaluation,small topics set,effectiveness metrics,true system effectiveness,information retrieval system,right choice,search effectiveness,sample size,test corpora,accurate prediction,topic selection,good topic,topic set reduction,ntcir result,limited number | Data mining,Information retrieval,Computer science,Sample size determination | Journal |
Volume | Issue | ISSN |
27 | 4 | 1046-8188 |
Citations | PageRank | References |
32 | 1.30 | 14 |
Authors | ||
3 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| John Guiver | 1 | 482 | 21.48 |
| Stefano Mizzaro | 2 | 862 | 85.52 |
| STEPHEN ROBERTSON | 3 | 6204 | 669.07 |