Name
Title | ||
|---|---|---|
Omni-Sparsity DNN: Fast Sparsity Optimization for On-Device Streaming E2E ASR Via Supernet. |
Abstract | ||
|---|---|---|
From wearables to powerful smart devices, modern automatic speech recognition (ASR) models run on a variety of edge devices with different computational budgets. To navigate the Pareto front of model accuracy vs model size, researchers are trapped in a dilemma of optimizing model accuracy by training and fine-tuning models for each individual edge device while keeping the training GPU-hours tractable. In this paper, we propose Omni-sparsity DNN, where a single neural network can be pruned to generate optimized model for a large range of model sizes. We develop training strategies for Omni-sparsity DNN that allows it to find models along the Pareto front of word-error-rate (WER) vs model size while keeping the training GPU-hours to no more than that of training one singular model. We demonstrate the Omni-sparsity DNN with streaming E2E ASR models. Our results show great saving on training time and resources with similar or better accuracy on LibriSpeech compared to individually pruned sparse models: 2%-6.6% better WER on Test-other. |
Year | DOI | Venue |
|---|---|---|
2022 | 10.1109/ICASSP43922.2022.9746469 | IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) |
DocType | Citations | PageRank |
Conference | 0 | 0.34 |
References | Authors | |
0 | 9 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Haichuan Yang | 1 | 0 | 1.01 |
| Yuan Shangguan | 2 | 1 | 2.04 |
| Dilin Wang | 3 | 0 | 0.68 |
| Meng Li | 4 | 132 | 17.74 |
| Pierce Chuang | 5 | 0 | 0.34 |
| Xiaohui Zhang | 6 | 0 | 1.01 |
| Ganesh Venkatesh | 7 | 274 | 17.97 |
| Ozlem Kalinli | 8 | 1 | 3.39 |
| Vikas Chandra | 9 | 691 | 59.76 |