Name
Abstract | ||
|---|---|---|
We present a multicore-enabled smart storage for clusters in general and MapReduce clusters in particular. The goal of this research is to improve performance of data-intensive parallel applications on clusters by offloading data processing to multicore processors in storage nodes. Compared with traditional storage devices, next-generation disks will have computing capability to reduce computational load of host processors or CPUs. With the advance of processor and memory technologies, smart storage systems are promising devices to perform complex on-disk operations. The proposed smart storage system can avoid moving a huge amount of data back and forth between storage nodes and computing nodes in a cluster. To enhance the performance of data-intensive applications, we have designed a smart storage system called Multicore-enabled Smart Storage (McSD), in which a multicore processor is integrated in storage nodes. We have implemented a programming framework for data-intensive applications running on a computing system coupled with McSD. The programming framework aims at balancing load between computing nodes and multicore-enabled smart storage nodes. To fully utilize multicore processors in smart storage nodes, we have implemented the MapReduce model for McSDs to handle parallel computing on a cluster. A prototype of McSD has been implemented in a cluster connected by Gigabit Ethernet. Experimental results show that McSD can significantly reduce the execution times of three real-world applications - word count, string matching, and matrix multiplication. We demonstrate that the integration of multicore-enabled smart storage with MapReduce clusters is a promising approach to improving overall performance of data-intensive applications on clusters. |
Year | DOI | Venue |
|---|---|---|
2012 | 10.1109/CLUSTER.2012.70 | CLUSTER |
Keywords | Field | DocType |
matrix multiplication,string matching,multicore-enabled smart storage,general clusters,parallel programming,programming framework,mapreduce clusters,multicore-enabled smart storage node,word count,host processors,storage allocation,disc drives,storage node,multicore processor,complex on-disk operations,resource allocation,execution time reduction,mcsd,multiprocessing systems,smart storage system,data-intensive application,smart storage node,multicore-enabled smart storage nodes,computational load reduction,gigabit ethernet,mapreduce cluster,proposed smart storage system,computing nodes,memory technologies,cpu,parallel computing,data-intensive parallel applications performance improve,load balancing,traditional storage device,offloading data processing,multicore processing,programming,prototypes | String searching algorithm,Converged storage,Computer science,Computer data storage,Parallel computing,Real-time computing,Resource allocation,Gigabit Ethernet,Matrix multiplication,Multi-core processor,Software framework | Conference |
ISSN | ISBN | Citations |
1552-5244 | 978-1-4673-2422-9 | 0 |
PageRank | References | Authors |
0.34 | 9 | 8 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Zhiyang Ding | 1 | 147 | 7.77 |
| Xunfei Jiang | 2 | 56 | 9.40 |
| shu yin | 3 | 307 | 22.05 |
| Xiao Qin | 4 | 1836 | 125.69 |
| Kai-Hsiung Chang | 5 | 22 | 8.13 |
| Xiaojun Ruan | 6 | 390 | 25.87 |
| mohammed i alghamdi | 7 | 57 | 10.85 |
| Meikang Qiu | 8 | 3722 | 246.98 |