Name
Abstract | ||
|---|---|---|
Benchmarking is critical when evaluating performance, but is especially difficult for file and storage systems. Complex interactions between I/O devices, caches, kernel daemons, and other OS components result in behavior that is rather difficult to analyze. Moreover, systems have different features and optimizations, so no single benchmark is always suitable. The large variety of workloads that these systems experience in the real world also adds to this difficulty. In this article we survey 415 file system and storage benchmarks from 106 recent papers. We found that most popular benchmarks are flawed and many research papers do not provide a clear indication of true performance. We provide guidelines that we hope will improve future performance evaluations. To show how some widely used benchmarks can conceal or overemphasize overheads, we conducted a set of experiments. As a specific example, slowing down read operations on ext2 by a factor of 32 resulted in only a 2--5% wall-clock slowdown in a popular compile benchmark. Finally, we discuss future work to improve file system and storage benchmarking. |
Year | DOI | Venue |
|---|---|---|
2008 | 10.1145/1367829.1367831 | TOS |
Keywords | Field | DocType |
file system,single benchmark,storage benchmarking,storage system,additional key words and phrases: benchmarks,storage benchmarks,systems experience,future work,popular benchmarks,future performance evaluation,true performance,file systems,storage systems acm reference format:,year study,measurement,design | Kernel (linear algebra),File system,Computer science,Compiler,Operating system,Database,Benchmarking,Overhead (business) | Journal |
Volume | Issue | Citations |
4 | 2 | 105 |
PageRank | References | Authors |
4.96 | 127 | 4 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Avishay Traeger | 1 | 281 | 16.07 |
| Erez Zadok | 2 | 1461 | 105.28 |
| Nikolai Joukov | 3 | 320 | 25.34 |
| Charles P. Wright | 4 | 587 | 36.67 |