Paper Info
Title
The Integration of Shared Storages with the CephFS and Rados Gateway for Big Data Accessing
Abstract
In recent years, high availability shared storage will become a popular information technology industry development orientation. Currently, information technology industries emphasize to reduce high risk data requirements and improve read and write performance of data storage. Therefore, the main purpose of this work is to improve read and write performance with the best way on Ceph Storage Cluster. In this system, the data is stored on Hadoop Distributed File System (HDFS), and the data stored in-memory virtual distributed store system that mentioned as Alluxio automatically. Then, the data would be processed through Hadoop Map Reduce method and the output would be inserted into Hadoop Distributed File System and Alluxio environment. The first experiment is to use S3 as application program interface that will connect to RADOS Gateway stored data into Object Storage Daemon (OSD). The second experiment is based on the first out experiment would be through Ceph File System (CephFS) connected to Object Storage Daemon directly. The data is saved in Ceph environment more secure than in Alluxio as in-memory storage system because OSD can be used for data backup based on object storage level. We can use S3 browser (GUI) to maintain data like grant access, maintain folders maintenance, create user accounts, move data location etc. The last one, we used Inkscope monitors all system. If there is any problem, system will give warning or error responds to users automatically.
Year
DOI
Venue
2018
10.1109/COMPSAC.2018.10209
2018 IEEE 42nd Annual Computer Software and Applications Conference (COMPSAC)
Keywords
Field
DocType
Big data,Share Storage,Ceph Storage System
Distributed File System,Object storage,File system,Computer data storage,Computer science,Server,Real-time computing,Big data,Daemon,Operating system,Backup
Conference
Volume
ISSN
ISBN
02
0730-3157
978-1-5386-2667-2
Citations 
PageRank 
References 
0
0.34
6
Authors
3
Name
Order
Citations
PageRank
Jia-Yow Weng100.34
Chao-Tung Yang21196139.50
Chih-Hung Chang323344.07