An Overview of Deep Neural Network Model Compression

Abstract:
In recent years, with the rapid development of machine learning, deep neural networks have achieved great success in the fields of computer vision and natural language processing. However, accompany with the remarkable performance, came the huge amount of parameters, the high cost of storage and computing. It is a great challenge for the embedded devices, such as autonomous vehicles, robotics and so on. Therefore, model compression and model pruning has become a research hotspot. This paper summarizes the achievements and progress in model compression from the aspects of model pruning, quantization, and lightweight network design. The future research directions in the field of model compression and acceleration are also prospected.