Name
Abstract | ||
|---|---|---|
We describe NDStore, a scalable multi-hierarchical data storage deployment for spatial analysis of neuroscience data on the AWS cloud. The system design is inspired by the requirement to maintain high I/O throughput for workloads that build neural connectivity maps of the brain from peta-scale imaging data using computer vision algorithms. We store all our data on the AWS object store S3 to limit our deployment costs. S3 serves as our base-tier of storage. Redis, an in-memory key-value engine, is used as our caching tier. The data is dynamically moved between the different storage tiers based on user access. All programming interfaces to this system are RESTful web-services. We include a performance evaluation that shows that our production system provides good performance for a variety of workloads by combining the assets of multiple cloud services. |
Year | DOI | Venue |
|---|---|---|
2018 | 10.1109/eScience.2018.00037 | 2018 IEEE 14th International Conference on e-Science (e-Science) |
Keywords | Field | DocType |
Spatial Data,Big Data,Cloud Computing,Object Storage,Neuroscience | Data visualization,Software deployment,Hierarchical storage management,Computer data storage,Computer science,Systems design,Throughput,Distributed computing,Scalability,Cloud computing | Conference |
ISSN | ISBN | Citations |
2325-372X | 978-1-5386-9157-1 | 0 |
PageRank | References | Authors |
0.34 | 0 | 7 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| kunal lillaney | 1 | 9 | 2.18 |
| Dean Kleissas | 2 | 14 | 2.71 |
| Alexander Eusman | 3 | 0 | 0.34 |
| Eric Perlman | 4 | 54 | 5.00 |
| William Gray Roncal | 5 | 38 | 8.25 |
| Joshua T. Vogelstein | 6 | 273 | 31.99 |
| Randal Burns | 7 | 1955 | 115.15 |