Title
An Empirical Performance Evaluation of Relational Keyword Search Techniques
Abstract
Extending the keyword search paradigm to relational data has been an active area of research within the database and IR community during the past decade. Many approaches have been proposed, but despite numerous publications, there remains a severe lack of standardization for the evaluation of proposed search techniques. Lack of standardization has resulted in contradictory results from different evaluations, and the numerous discrepancies muddle what advantages are proffered by different approaches. In this paper, we present the most extensive empirical performance evaluation of relational keyword search techniques to appear to date in the literature. Our results indicate that many existing search techniques do not provide acceptable performance for realistic retrieval tasks. In particular, memory consumption precludes many search techniques from scaling beyond small data sets with tens of thousands of vertices. We also explore the relationship between execution time and factors varied in previous evaluations; our analysis indicates that most of these factors have relatively little impact on performance. In summary, our work confirms previous claims regarding the unacceptable performance of these search techniques and underscores the need for standardization in evaluations--standardization exemplified by the IR community.
Year
DOI
Venue
2014
10.1109/TKDE.2012.228
Knowledge and Data Engineering, IEEE Transactions  
Keywords
Field
DocType
information retrieval,relational databases,IR community,empirical performance evaluation,information retrieval,keyword search paradigm,memory consumption,relational data,relational keyword search techniques,standardization,Keyword search,empirical evaluation,information retrieval,relational database
Data mining,Small data,Relational database,Information retrieval,Computer science,Search-oriented architecture,Encyclopedia,Standardization,Benchmark (computing),The Internet,Electronic publishing
Journal
Volume
Issue
ISSN
26
1
1041-4347
Citations 
PageRank 
References 
29
0.97
28
Authors
2
Name
Order
Citations
PageRank
Joel Coffman1324.44
Alfred C. Weaver247355.79