Name
Abstract | ||
|---|---|---|
Recently, the performance of speech recognition system based on neural network has been greatly improved. Arguably, this huge improvement can be mainly attributed to deeper and wider layers. These systems are more difficult to be deployed on the embedded devices due to their large size and high computational complexity. To address these issues, we propose a method to compress deep feed-forward neural network (DNN) based acoustic model. In detail, a state-of-the-art acoustic model is trained as the baseline model. In this step, layer normalization is applied to accelerating the model convergence and improving the generalization performance. Knowledge distillation and pruning are then conducted to compress the model. Our final model can achieve 14:59x parameters reduction, 5x storage size reduction and comparable performance compared with the baseline model. |
Year | DOI | Venue |
|---|---|---|
2018 | 10.1109/ICPR.2018.8545028 | 2018 24TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION (ICPR) |
Keywords | Field | DocType |
speech recognition, layer normalization, knowledge distillation, pruning, model compression | Convergence (routing),Normalization (statistics),Pattern recognition,Computer science,Recurrent neural network,Distillation,Artificial intelligence,Artificial neural network,Computer engineering,Acoustic model,Computational complexity theory,Fold (higher-order function) | Conference |
ISSN | Citations | PageRank |
1051-4651 | 0 | 0.34 |
References | Authors | |
0 | 5 | |