Name
Abstract | ||
|---|---|---|
Today, our picture of the Universe radically differs from that of just over a decade ago. We now know that the Universe is not only expanding as Hubble discovered in 1929, but that the rate of expansion is accelerating, propelled by mysterious new physics dubbed "Dark Energy." This revolutionary discovery was made by comparing the brightness of nearby Type Ia supernovae (which exploded in the past billion years) to that of much more distant ones (from up to seven billion years ago). The reliability of this comparison hinges upon a very detailed understanding of the physics of the nearby events. As part of its effort to further this understanding, the Nearby Supernova Factory (SNfactory) relies upon a complex pipeline of serial processes that execute various image processing algorithms in parallel on ~10TBs of data. This pipeline has traditionally been run on a local cluster. Cloud computing offers many features that make it an attractive alternative. The ability to completely control the software environment in a Cloud is appealing when dealing with a community developed science pipeline with many unique library and platform requirements. In this context we study the feasibility of porting the SNfactory pipeline to the Amazon Web Services environment. Specifically we: describe the tool set we developed to manage a virtual cluster on Amazon EC2, explore the various design options available for application data placement, and offer detailed performance results and lessons learned from each of the above design options. |
Year | DOI | Venue |
|---|---|---|
2010 | 10.1145/1851476.1851538 | HPDC |
Keywords | Field | DocType |
science pipeline,billion year,amazon web services environment,detailed understanding,application data placement,design option,amazon ec2,snfactory pipeline,complex pipeline,performance study,detailed performance result,dark energy,image processing,distributed systems,distributed system,new physics,cloud computing,high performance computing,web service,community development | Supernova,World Wide Web,Supercomputer,Dark energy,Computer science,Real-time computing,Software,Porting,Universe,Amazon web services,Distributed computing,Cloud computing | Conference |
Citations | PageRank | References |
30 | 1.33 | 4 |
Authors | ||
4 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Keith R. Jackson | 1 | 481 | 35.62 |
| lavanya ramakrishnan | 2 | 710 | 56.18 |
| Karl J. Runge | 3 | 44 | 2.75 |
| Rollin C. Thomas | 4 | 70 | 6.11 |