This repository contains my evaluations of Merge and Run in Binarized Residual Neural Work
Copyright: Xianda Xu [email protected]
Accuracy | Full-Precise (1) | Binarized (1) | Binarized (2) | Netscope |
---|---|---|---|---|
ResNet-18 | 93.28% | 90.50% | Network | |
MR-ResNet-20 | 92.15% | 87.77% | 90.48% | Network |
MR-ResNet-32 | 93.39% | 90.46% | 92.58% | Network |
Style 1 does downsampling by using concatenation and convolusion (kernel_size=1, stride=2, padding=0)
Style 2 does downsampling by using concatenation and average pooling (kernel_size=2, stride=2)
-
Keep full-precision on the first convolutional layer and the last linear layer.
-
In binarized convolutional layers, all weights are binarized and scaled in propagation (https://arxiv.org/abs/1603.05279). But here, the scale factors are not learnt but all set to 1.
-
BatchNorms in binarized blocks have no affine weights and bias parameters.
-
Since activations are not binarized, ReLU is used instead of HardTanh (https://arxiv.org/pdf/1602.02830).
Paper: (https://arxiv.org/abs/1512.03385)
Netscope: Network
Full-Precise Accuracy on Cifar-10: 93.28% with 80 epoches
Binarized Accuracy on Cifar-10: 90.50% with 80 epoches
Paper: (https://arxiv.org/abs/1611.07718)
Netscope: Network
Full-Precise Accuracy on Cifar-10: 92.15% with 80 epoches
Binarized Accuracy on Cifar-10: 87.77% with 80 epoches
- Downsampling is done by firstly concatenating left-branch and right-branch and secondly using a convolusion (kernel-size:1, stride:2, padding:0)
Binarized Accuracy on Cifar-10: 90.48% with 80 epoches
- Downsampling is done by firstly concatenating left-branch and right-branch and secondly using a average pooling (kernel_size:2, stride:2)
Paper: (https://arxiv.org/abs/1611.07718)
Netscope: Network
Full-Precise Accuracy on Cifar-10: 93.39% with 80 epoches
Binarized Accuracy on Cifar-10: 90.46% with 80 epoches
- Downsampling is done by firstly concatenating left-branch and right-branch and secondly using a convolusion (kernel-size:1, stride:2, padding:0)
Binarized Accuracy on Cifar-10: 92.58% with 80 epoches
- Downsampling is done by firstly concatenating left-branch and right-branch and secondly using a average pooling (kernel_size:2, stride:2)