Paper Info
Title
Alexandria: A Proof-of-Concept Implementation and Evaluation of Generalised Data Deduplication
Abstract
The amount of data generated worldwide is expected to grow from 33 to 175 ZB by 2025 in part driven by the growth of Internet of Things (IoT) and cyber-physical systems (CPS). To cope with this enormous amount of data, new cloud storage techniques must be developed. Generalised Data Deduplication (GDD) is a new paradigm for reducing the cost of storage by systematically identifying near identical data chunks, storing their common component once, and a compact representation of the deviation to the original chunk for each chunk. This paper presents a system architecture for GDD and a proof-of-concept implementation. We evaluated the compression gain of Generalised Data Deduplication using three data sets of varying size and content and compared to the performance of the EXT4 and ZFS file systems, where the latter employs classic deduplication. We show that Generalised Data Deduplication provide up to 16.75% compression gain compared to both EXT4 and ZFS with data sets with less than 5 GB of data.
Year
DOI
Venue
2019
10.1109/GCWkshps45667.2019.9024368
2019 IEEE Globecom Workshops (GC Wkshps)
Keywords
DocType
ISSN
identical data chunks,proof-of-concept implementation,generalised data deduplication,data sets,data generated worldwide,Internet of Things,IoT,cyber-physical systems,CPS,GDD,EXT4,ZFS file systems
Conference
2166-0069
ISBN
Citations 
PageRank 
978-1-7281-0961-9
0
0.34
References 
Authors
3
6
Name
Order
Citations
PageRank
Lars Nielsen111.71
Rasmus Vestergaard213.40
Niloofar Yazdani301.69
Prasad Talasila400.68
Daniel E. Lucani523642.29
Marton Sipos633.09