Paper Info
Title
Content-Aware Data Distribution Over Cluster Nodes
Abstract
Proper data items distribution may seriously improve the performance of data processing in distributed environment. However, typical datastorage systems as well as distributed computational frameworks do not pay special attention to that aspect. In this paper author introduces two custom data items addressing methods for distributed datastorage on the example of Scalable Distributed Two-Layer Datastore. The basic idea of those methods is to preserve that data items stored on the same cluster node are similar to each other following concepts of data clustering. Still, most of the data clustering mechanisms have serious problem with data scalability which is a severe limitation in Big Data applications. The proposed methods allow to efficiently distribute data set over a set of buckets. As it was shown by the experimental results, all proposed methods generate good results efficiently in comparison to traditional clustering techniques like k-means, agglomerative and birch clustering. Distributed environment experiments shown that proper data distribution can seriously improve the effectiveness of Big Data processing.
Year
DOI
Venue
2021
10.3233/IDA-205360
INTELLIGENT DATA ANALYSIS
Keywords
DocType
Volume
SD2DS, data clustering, Big Data
Journal
25
Issue
ISSN
Citations 
4
1088-467X
0
PageRank 
References 
Authors
0.34
0
1
Name
Order
Citations
PageRank
Adam Krechowicz132.85