Title
A scalable heterogeneous solution for massive data collection and database loading
Abstract
Massive collection of data at high rates is critical for many industries. Typically, a massive stream of records is gathered from the business information network at a very high rate. Because of the complexity of the collection process, the classical database solution falls short. The high volume and rate of records involved requires a heterogeneous pipeline comprised of two major parts: a system that carries out massive collection and then uploads the information to a database, and a subsequent data analysis and management system consisting of an Extract Transform and Load component. We developed a massive collection and loading system, based on a highly scalable heterogeneous architecture solution. The solution has been applied successfully for Telco revenue assurance, and can be applied to other industrial areas. The solution was successful in scaling up a Telco client system to handle streams of records ten times larger than was previously possible.
Year
DOI
Venue
2006
10.1007/978-3-540-73950-0_5
BIRTE
Keywords
Field
DocType
high rate,database loading,scalable heterogeneous architecture solution,telco revenue assurance,management system,collection process,massive collection,massive data collection,massive stream,classical database solution,high volume,telco client system,scalable heterogeneous solution,data collection,data analysis
Java messaging service,Data mining,Data collection,Business information,Computer science,Upload,Revenue assurance,Management system,Database,Scalability
Conference
Volume
ISSN
Citations 
4365
0302-9743
1
PageRank 
References 
Authors
0.41
1
5
Name
Order
Citations
PageRank
Uri Shani1246.94
Aviad Sela2375.65
Alex Akilov310.41
Inna Skarbovski410.41
David Berk510.41