Name
Abstract | ||
|---|---|---|
When training complex neural networks, memory usage can be an important bottleneck. The question of when to rematerialize, i.e., to recompute intermediate values rather than retaining them in memory, becomes critical to achieving the best time and space efficiency. In this work we consider the rematerialization problem and devise efficient algorithms that use structural characterizations of computation graphs-treewidth and pathwidth-to obtain provably efficient rematerialization schedules. Our experiments demonstrate the performance of these algorithms on many common deep learning models. |
Year | Venue | Keywords |
|---|---|---|
2019 | ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 32 (NIPS 2019) | the question |
Field | DocType | Volume |
Bottleneck,Computer science,Theoretical computer science,Schedule,Artificial intelligence,Deep learning,Artificial neural network,Machine learning,Rematerialization,Computation | Conference | 32 |
ISSN | Citations | PageRank |
1049-5258 | 0 | 0.34 |
References | Authors | |
0 | 5 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Ravi Kumar | 1 | 13932 | 1642.48 |
| Manish Purohit | 2 | 46 | 10.84 |
| Svitkina, Zoya | 3 | 0 | 0.68 |
| Erik Vee | 4 | 747 | 43.61 |
| Joshua R. Wang | 5 | 69 | 5.83 |