[AIRG] Model Compression -- Binarized Neural Network

[Bryce XU <xxu373@xxxxxxxx>]

Hi, everyone!

I am a visiting Junior undergraduate from University of Electronic Science and Technology of China.

Next Wednesday ( 7th Nov. ), I would like to share you guys with something about Model Compression.

I will first talk about the significance of model compression in artificial intelligence and possible solutions. Then, I will show you how to binarize the model in order to compress and speed it up. Afterwards, I will talk a little bit about my experiments and some problems we may encounter in binarizing the model. Last, some recent work will be covered.

The paper you need to read is https://arxiv.org/abs/1602.02830.

I will also give you some optional papers if you are interested in this topic.

http://papers.nips.cc/paper/5647-binaryconnect-training-deep-neural-networks-with-b
https://link.springer.com/chapter/10.1007/978-3-319-46493-0_32

https://arxiv.org/abs/1612.01064

https://arxiv.org/abs/1806.07550

Best,

Xianda (Bryce) Xu

[1602.02830] Binarized Neural Networks: Training Deep ...

arxiv.org

Abstract: We introduce a method to train Binarized Neural Networks (BNNs) - neural networks with binary weights and activations at run-time. At training-time the binary weights and activations are used for computing the parameters gradients. During the forward pass, BNNs drastically reduce memory size and accesses, and replace most arithmetic operations with bit-wise operations, which is ...