This is the PyTorch implementation of the paper SSN: Learning Sparse Switchable Normalization via SparsestMax, CVPR 2019.
By Wenqi Shao, Tianjian Meng, Jingyu Li, Ruimao Zhang, Yudian Li, Xiaogang Wang, Ping Luo.
Sparse Switchable Normalization is able to learn only one normalization operation for each normalization layer in a deep neural network in an end-to-end manner.
Comparisons of top-1 accuracies on the validation set of ImageNet, by using ResNet50 trained with SSN, SN, BN, and GN in different batch size settings. The bracket (·, ·) denotes (#GPUs,#samples per GPU).
| Normalizer | (8,32) | (8,16) | (8,8) | (8,4) | (8,2) |
|---|---|---|---|---|---|
| BN | 76.4 | 76.3 | 75.5 | 72.7 | 65.3 |
| GN | 75.9 | 75.8 | 76.0 | 75.8 | 75.9 |
| SN | 76.9 | 76.7 | 76.7 | 75.9 | 75.6 |
| SSN | 77.2 | 77.0 | 76.8 | 76.1 | 75.9 |
- Install PyTorch
- Clone the repo:
git clone https://github.com/switchablenorms/Sparse_SwitchNorm.git
- python packages
- pytorch>=0.4.0
- torchvision>=0.2.1
- tensorboardX
- pyyaml
- Download the ImageNet dataset and put them into the
{repo_root}/data/imagenet.
./train.sh
Number of GPUs and configuration file to use can be modified in train.sh
Download the pretrained models from Model Zoo and put them into the {repo_root}/model_zoo
./test.sh
Or you can specify the checkpoint path by modifying test.sh
--checkpoint_path model_zoo/ssn_8x2_75.848.pth \
We provide models pretrained with SSN on ImageNet. The configuration of SSN is denoted as (#GPUs, #images per GPU).
| Model | Top-1* | Top-5* | Download | MD5 |
|---|---|---|---|---|
| ResNet50v1+SSN (8,32) | 77.25% | 93.29% | [Google Drive] [Baidu Pan (pin:ckd2)] | 5c9fb111577b040e62461db51ffce69b |
| ResNet50v1+SSN (8,16) | 76.98% | 93.29% | [Google Drive] [Baidu Pan (pin:995u)] | d7085632529a9b1945a28a4d3bf4cacb |
| ResNet50v1+SSN (8,8) | 76.75% | 93.33% | [Google Drive] [Baidu Pan (pin:k9ay)] | af85a39920644421fc48e216aba6ff0e |
| ResNet50v1+SSN (8,4) | 76.07% | 96.71% | [Google Drive] [Baidu Pan (pin:jpvu)] | 949d4bd54a7c63a3371b941eb4d3ea69 |
| ResNet50v1+SSN (8,2) | 75.85% | 92.73% | [Google Drive] [Baidu Pan (pin:pdyx)] | dadb0f1a0c49c31aedd6cb83d4996a03 |
*single-crop validation accuracy on ImageNet (a 224x224 center crop from resized image with shorter side=256)
In evaluation, download the above models and put them into the {repo_root}/model_zoo.
If you find this work helpful in your project or use our model zoo, please consider citing:
@article{shao2019ssn,
title={SSN: Learning Sparse Switchable Normalization via SparsestMax},
author={Shao, Wenqi and Meng, Tianjian and Li, Jingyu and Zhang, Ruimao and Li, Yudian and Wang, Xiaogang and Luo, Ping},
journal={arXiv preprint arXiv:1903.03793},
year={2019}
}