Paper Info
Title
A Framework for Real-Time Information Derivation from Big Sensor Data
Abstract
In data-intensive real-time applications, e.g., transportation management and location-based services, the amount of sensor data is exploding. In these applications, it is desirable to extract value-added information, e.g., fast driving routes, from sensor data streams in real-time rather than overloading users with massive raw data. However, achieving the objective is challenging due to the data volume and complex data analysis tasks with stringent timing constraints. Most existing big data management systems, e.g., Hadoop, are not directly applicable to real-time sensor data analytics, since they are timing agnostic and focus on batch processing of previously stored data that are potentially outdated and subject to I/O overheads. To address the problem, we design a new real-time big data management framework, which supports a non-preemptive periodic task model for continuous in-memory sensor data analysis and a schedulability test based on the EDF (Earliest Deadline First) algorithm to derive information from current sensor data in real-time by extending the map-reduce model originated in functional programming. As a proof-of-concept case study, a prototype system is implemented. In the performance evaluation, it is empirically shown that all deadlines can be met for the tested sensor data analysis benchmarks.
Year
DOI
Venue
2015
10.1109/HPCC-CSS-ICESS.2015.46
HPCC/CSS/ICESS
Field
DocType
ISSN
Data warehouse,Data modeling,Data stream mining,Data analysis,Real-time data,Computer science,Raw data,Real-time computing,Data virtualization,Big data,Distributed computing
Conference
2576-3504
Citations 
PageRank 
References 
0
0.34
10
Authors
2
Name
Order
Citations
PageRank
Liehuo Chen100.34
Kyoung-Don Kang256337.51