Name
Title | ||
|---|---|---|
An Efficient Ring-Based Metadata Management Policy for Large-Scale Distributed File Systems |
Abstract | ||
|---|---|---|
The growing size of modern file system is expected to reach EB-scale. Therefore, an efficient and scalable metadata service is critical to system performance. Distributed metadata management schemes, which use multiple metadata servers (MDS's) to store metadata, provide a highly effective approach to alleviate the workload of a single server. However, it is difficult to maintain good metadata locality and load balancing among MDS's at the same time. In this paper, we propose a novel hashing scheme called AngleCut to partition metadata namespace tree and serve large-scale distributed storage systems. AngleCut first uses a locality preserving hashing (LPH) function to project the namespace tree into linear keyspace, i.e., multiple Chord-like rings. Then we design a history-based allocation strategy to adjust the workload of MDS's dynamically. Besides, we propose a two-layer metadata cache mechanism, including server-side cache and client-side cache to provide the two stage access acceleration. Last but not least, we introduce a distributed metadata processing 2PC Protocol Based on Message Queue (2PC-MQ) to ensure data consistency. In general, our scheme preserves good metadata locality as well as maintains a high load balancing between MDS's. The theoretical proof and extensive experiments on Amazon EC2 demonstrate the superiority of AngleCut over previous literature. |
Year | DOI | Venue |
|---|---|---|
2019 | 10.1109/tpds.2019.2901883 | IEEE Transactions on Parallel and Distributed Systems |
Keywords | Field | DocType |
Metadata,Servers,Load management,Distributed databases,Resource management,Protocols,File systems | Metadata,File system,Load balancing (computing),Cache,Computer science,Server,Distributed data store,Distributed database,Metadata management,Distributed computing | Journal |
Volume | Issue | ISSN |
30 | 9 | 1045-9219 |
Citations | PageRank | References |
2 | 0.36 | 0 |
Authors | ||
5 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Yuanning Gao | 1 | 4 | 2.73 |
| Xiaofeng Gao | 2 | 713 | 98.58 |
| Xiaochun Yang | 3 | 18 | 2.31 |
| Jiaxi Liu | 4 | 6 | 1.59 |
| guihai chen | 5 | 3537 | 317.28 |