Paper Info
Title
Valmar: High-bandwidth real-time streaming data management
Abstract
In applications ranging from radio telescopes to Internet traffic monitoring, our ability to generate data has outpaced our ability to effectively capture, mine, and manage it. These ultra-high-bandwidth data streams typically contain little useful information and most of the data can be safely discarded. Periodically, however, an event of interest is observed and a large segment of the data must be preserved, including data preceding detection of the event. Doing so requires guaranteed data capture at source rates, line speed filtering to detect events and data points of interest, and TiVo-like ability to save past data once an event has been detected. We present Valmar, a system for guaranteed capture, indexing, and storage of ultra-high-bandwidth data streams. Our results show that Valmar performs at nearly full disk bandwidth, up to several orders of magnitude faster than flat file and database systems, works well with both small and large data elements, and allows concurrent read and search access without compromising data capture guarantees.
Year
DOI
Venue
2012
10.1109/MSST.2012.6232387
Mass Storage Systems and Technologies
Keywords
Field
DocType
storage management,Internet traffic monitoring,TiVo-like ability,Valmar,data preceding detection,line speed filtering,radio telescope,real-time streaming data management,ultra-high-bandwidth data stream capturing,ultra-high-bandwidth data stream indexing,ultra-high-bandwidth data stream storage
Data point,Data stream mining,Computer science,Search engine indexing,Flat file database,Real-time computing,Ranging,Bandwidth (signal processing),Automatic identification and data capture,Internet traffic
Conference
ISSN
ISBN
Citations 
2160-195X E-ISBN : 978-1-4673-1746-7
978-1-4673-1746-7
0
PageRank 
References 
Authors
0.34
10
4
Name
Order
Citations
PageRank
David O. Bigelow100.34
Scott Brandt2644.67
John Bent3959.02
Hsing-bung Chen400.34