Paper Info
Title
Propeller: A Scalable Real-Time File-Search Service in Distributed Systems
Abstract
File-search service is a valuable facility to accelerate many analytics applications, because it can drastically reduce the scale of the input data. The main challenge facing the design of large-scale and accurate file-search services is how to support real-time indexing in an efficient and scalable way. To address this challenge, we propose a distributed file-search service, called Propeller, which utilizes a special file-access pattern, called access-causality, to partition file-indices in order to expose substantial access locality and parallelism to accelerate the file-indexing process. The extensive evaluations of Propeller show that it is real-time in file-indexing operations, accurate in file-search results, and scalable in large datasets. It achieves significantly better file-indexing and file-search performance (up to 250x) than a centralized solution (MySQL) and much higher accuracy and substantially lower query latency (up to 22x than a state-of-the-art desktop search engine (Spotlight).
Year
DOI
Venue
2014
10.1109/ICDCS.2014.46
ICDCS
Keywords
Field
DocType
file search, data management, metadata,scalable real-time file-search service,file search,real-time indexing,file-indexing process,propeller,file organisation,distributed file-search service,metadata,file-access pattern,access-causality,data management,distributed processing
Distributed File System,SSH File Transfer Protocol,Global Namespace,Self-certifying File System,Computer science,Device file,Search engine indexing,Versioning file system,File system fragmentation,Distributed computing
Conference
ISSN
Citations 
PageRank 
1063-6927
7
0.48
References 
Authors
25
4
Name
Order
Citations
PageRank
Lei Xu170.48
Hong Jiang22137157.96
Lei Tian385339.45
Ziling Huang4233.76