Paper Info
Title
Experiences building network-coding-based distributed storage systems
Abstract
Large-scale distributed storage systems are prone to node failures. To provide fault tolerance, data is often encoded to maintain data redundancy over multiple storage nodes. If a node fails, it can be repaired by downloading data from surviving nodes and regenerating the lost data in a new node. Network coding has recently been proposed (e.g., see [2]) to generate data redundancy. It is shown that network coding can minimize the amount of data being transferred for repair, while maintaining the same fault tolerance as in conventional erasure coding schemes. Its idea is to have storage nodes first encode their stored data and then send the encoded data for regeneration. On the other hand, the topic of network coding in storage systems is mostly investigated in theoretical studies. Its performance in real deployment remains an open issue. This motivates us to study the practicality of deploying network coding in real-world distributed storage systems. We highlight two of our implementation projects of network-coding-based storage systems at the Chinese University of Hong Kong, namely NCCloud and CORE. Both of them target different storage applications, while building on network coding to enable high availability and efficient recovery of storage systems.
Year
DOI
Venue
2014
10.1109/ITA.2014.6804234
Information Theory and Applications Workshop
Keywords
Field
DocType
distributed databases,fault tolerant computing,network coding,CORE,Chinese University of Hong Kong,NCCloud,data redundancy,erasure coding schemes,fault tolerance,large-scale distributed storage systems,multiple storage nodes,network-coding-based distributed storage systems,node failures,real-world distributed storage systems
Linear network coding,Computer science,Distributed data store,Upload,Data redundancy,Fault tolerance,Distributed database,Erasure code,High availability,Distributed computing
Conference
Citations 
PageRank 
References 
0
0.34
8
Authors
1
Name
Order
Citations
PageRank
patrick lee115710.91