Name
Title | ||
|---|---|---|
A Versatile Event-Driven Data Model in HBase Database for Multi-source Data of Power Grid |
Abstract | ||
|---|---|---|
As a column-oriented, distributed and high fault-tolerant database, HBase is gradually adopted by manufactory industry due to its huge capacity, fast random read and bulk processing performance. Power grid usually generates huge amount of data from multi-sources with hundreds of complicated data types. However, JOIN operation performance is low in HBase since it was not designed to serve relational queries as classic relational database. How to store the data to ensure a sufficient performance of JOIN operation and random read is the critical problem that must to be solved in HBase applications. In this paper, we proposed an event-driven HBase data model to resolve this problem. In our data model, each record of data is defined as a unique event that happened in power grid. Any type of data from any data sources can be distinguished in database. Therefore our data model can store multi-source data generated by power grid devices. In addition, JOIN operation is integrated in our data model, which improves the performance of reading data from multiple data sources. This is realized by designing a specific RowKey in tables. We also proposed a schema including a novel Virtual ColumnFamily, which resolves the compatibility problem of storing data from multi-sources. Virtual ColumnFamily is realized by designing specific qualifiers. To verify the effectiveness of our data model, we conducted empirical experiments on a Hadoop platform to compare our optimized schema and original schema. Experimental results showed that our data model ensures that our optimized schema performs better than the original schema based on original data model. |
Year | DOI | Venue |
|---|---|---|
2016 | 10.1109/SmartCloud.2016.28 | 2016 IEEE International Conference on Smart Cloud (SmartCloud) |
Keywords | Field | DocType |
HBase,Data model,Event-driven,JOIN operation,Virtual ColumnFamily | Data warehouse,Data modeling,Data mining,Relational database,Database model,Semi-structured model,Computer science,Database design,Data model,Elasticity (data store),Database | Conference |
ISBN | Citations | PageRank |
978-1-5090-5264-6 | 1 | 0.43 |
References | Authors | |
0 | 8 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Bin Liu | 1 | 2 | 0.81 |
| Yongxin Zhu | 2 | 466 | 58.07 |
| chang wang | 3 | 33 | 12.55 |
| Yufeng Chen | 4 | 6 | 1.30 |
| Tian Huang | 5 | 53 | 7.40 |
| WeiWei Shi | 6 | 51 | 12.26 |
| Mengjun Li | 7 | 1 | 0.43 |
| Yishu Mao | 8 | 6 | 3.28 |