Name
Title | ||
|---|---|---|
Big Data and Analysis of Data Transfers for International Research Networks Using NetSage |
Abstract | ||
|---|---|---|
Modern science is increasingly data-driven and collaborative in nature. Many scientific disciplines, including genomics, high-energy physics, astronomy, and atmospheric science, produce petabytes of data that must be shared with collaborators all over the world. The National Science Foundation-supported International Research Network Connection (IRNC) links have been essential to enabling this collaboration, but as data sharing has increased, so has the amount of information being collected to understand network performance. New capabilities to measure and analyze the performance of international wide-area networks are essential to ensure end-users are able to take full advantage of such infrastructure for their big data applications. NetSage is a project to develop a unified, open, privacy-aware network measurement, and visualization service to address the needs of monitoring today's high-speed international research networks. NetSage collects data on both backbone links and exchange points, which can be as much as 1Tb per month. This puts a significant strain on hardware, not only in terms storage needs to hold multi-year historical data, but also in terms of processor and memory needs to analyze the data to understand network behaviors. This paper addresses the basic NetSage architecture, its current data collection and archiving approach, and details the constraints of dealing with this big data problem of handling vast amounts of monitoring data, while providing useful, extensible visualization to end users. |
Year | DOI | Venue |
|---|---|---|
2017 | 10.1109/BigDataCongress.2017.51 | 2017 IEEE International Congress on Big Data (BigData Congress) |
Keywords | Field | DocType |
IRNC Measurement,Infrastructure,International,Tools,Visualization,Analytics | Data science,Data mining,End user,Data analysis,Computer science,Analytics,Data collection,Data visualization,World Wide Web,Data sharing,Big data,Database,Network performance | Conference |
ISSN | ISBN | Citations |
2379-7703 | 978-1-5386-1997-1 | 0 |
PageRank | References | Authors |
0.34 | 5 | 7 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Alberto González | 1 | 39 | 5.78 |
| Jason Leigh | 2 | 909 | 111.85 |
| Sean Peisert | 3 | 246 | 31.44 |
| Brian Tierney | 4 | 611 | 70.38 |
| Edward Balas | 5 | 5 | 1.20 |
| Predrag Radulovic | 6 | 0 | 0.34 |
| Jennifer M. Schopf | 7 | 751 | 45.49 |