Name
Abstract | ||
|---|---|---|
The main problem with the state of the art in the semantic search domain is the lack of comprehensive evaluations. There exist only a few efforts to evaluate semantic search tools and to compare the results with other evaluations of their kind. In this paper, we present a systematic approach for testing and benchmarking semantic search tools that was developed within the SEALS project. Unlike other semantic web evaluations our methodology tests search tools both automatically and interactively with a human user in the loop. This allows us to test not only functional performance measures, such as precision and recall, but also usability issues, such as ease of use and comprehensibility of the query language. The paper describes the evaluation goals and assumptions; the criteria and metrics; the type of experiments we will conduct as well as the datasets required to conduct the evaluation in the context of the SEALS initiative. To our knowledge it is the first effort to present a comprehensive evaluation methodology for Semantic Web search tools. |
Year | DOI | Venue |
|---|---|---|
2010 | 10.1145/1863879.1863889 | Proceedings of the 3rd International Semantic Search Workshop |
Keywords | DocType | Citations |
seals initiative,semantic query,comprehensive evaluation,semantic search tool,bench-marking,evaluation,usability,campaign design,semantic web search tool,methodology tests search tool,semantic search domain,evaluation goal,comprehensive evaluation methodology,semantic web evaluation,seals project,performance measure,semantic search,query language,ease of use,semantic web,bench marking | Conference | 8 |
PageRank | References | Authors |
0.63 | 12 | 5 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Stuart N. Wrigley | 1 | 181 | 20.56 |
| Dorothee Reinhard | 2 | 15 | 1.08 |
| Khadija Elbedweihy | 3 | 43 | 4.03 |
| Abraham Bernstein | 4 | 2116 | 173.91 |
| Fabio Ciravegna | 5 | 1635 | 140.18 |