Transfer-based attacks

💡 Requirements

All Python libraries that tranfer-based attacks in BlackboxBench depend on are listed in requirements.txt. You can run the following script to configurate necessary environment:

pip install -r requirements.txt

🤩 Quick start❕

1️⃣ Load pretrained models

Before user run the main file main_attack.py, they need to load model with .pth file.

📍 If a standard pretrained model is desired

Here is an example of how to load ResNet-50 pretrained on ImageNet. Users need to put pretrained model file resnet50-19c8e357.pth into 'surrogate_model/NIPS2017/pretrained/' and change the file path in the according model framework file surrogate_model/imagenet_models/resnet.py:

def resnet50(pretrained=False, **kwargs):
    if pretrained:
        state_dict_dir = './surrogate_model/NIPS2017/pretrained/resnet50-19c8e357.pth'
    else:
        state_dict_dir = None

    return _resnet('resnet50', Bottleneck, [3, 4, 6, 3], state_dict_dir, progress=True,**kwargs)

(🔗 Download links of pretrained weights can be found in Supplementary Sec. II of our paper. )

📍 If an user-customized model is desired

Here is an example of how to load an user-customized model <MODEL_ARCH> pretrained on <DATASET>. Users need to put model file ***.pth into 'surrogate_model/<DATASET>/<MODEL_ARCH>/'

Valid <DATASET> includes:

['CIFAR10', 'NIPS2017']

Valid <MODEL_ARCH> includes:

IMAGENET_MODEL_NAMES = ['resnet18', 'resnet34', 'resnet50', 'resnet152', 'vgg11_bn', 'vgg19', 'vgg19_bn', 'inception_v3', 'densenet121', 'mobilenet_v2', 'mobilenet_v3', 'senet154', 'resnext101', 'wrn50', 'wrn101', 'pnasnet', 'mnasnet', 'convnext_b', 'convnext_l', 'convnext_t', 'swin_b', 'swin_t', 'swin_l', 'vit_b_16', 'vit_b_32', 'vit_l_16', 'adv_convnext_b', 'adv_resnet50', 'adv_swin_b', 'adv_wrn50']
CIFAR10_MODEL_NAMES = ['densenet', 'pyramidnet272', 'resnext', 'vgg19_bn', 'wrn', 'gdas', 'adv_wrn_28_10', 'resnet50', 'inception_v3']

2️⃣ Configurate the hyperparameters of attacks

Users can modify the configuration file (***.json) to run different attack methods with ${l_\infty, l_2} \times {targeted, untargeted}$ setting. Here is an example json file for $l_\infty$, untargeted I-FGSM with ResNet-50 as the surrogate model onNIPS2017dataset, evaluated on three target models VGG19_bn, ResNet-152, Inception-V3.

{
  "source_model_path": ["NIPS2017/pretrained/resnet50"], #Path to all the model files of the ensembled surrogate models. Support path to a single model file or path containing many models.
  "target_model_path": ["NIPS2017/pretrained/vgg19_bn",
                        "NIPS2017/pretrained/resnet152",
                        "NIPS2017/pretrained/inception_v3"], #Path to all the target models.Only support path to a single model file.
  "n_iter": 100, #Number of iterations.
  "shuffle": true, #Random order of models vs sequential order of (ensembled) surrogate models.
  "batch_size": 200, #Batch size. Try a lower value if out of memory.
  "norm_type": "inf", #Type of L-norm.
  "epsilon": 0.03, #Max L-norm of the perturbation.
  "norm_step": 0.00392157, #Max norm at each step.
  "seed": 0, #Set random seed.
  "n_ensemble": 1, #Number of samples to ensemble for each iteration(Default: 1).
  "targeted": false, #Achieve targeted attack or not.
  "save_dir": "./save", #Path to save adversarial images.

  "input_transformation": "", #Input transformation compatible with each attack.
  "loss_function": "cross_entropy", #Loss function compatible with each attack.
  "grad_calculation": "general", #Define a gradient calculator compatible with each attack.
  "backpropagation": "nonlinear", #Linear backpropagation vs noninear backpropagation
  "update_dir_calculation": "sgd" #Update direction calculator compatible with each attack.
}

📍 If ensemble attacks is desired, list all ensembles models in source_model_path like this

"source_model_path": ["NIPS2017/pretrained/resnet50",
                      "NIPS2017/pretrained/wrn101",
                      "NIPS2017/pretrained/pnasnet",
                      "NIPS2017/pretrained/mnasnet",]

3️⃣ Run attacks

After modifying the attacks config files as desired, include config files of the considered attacks in main_attack.py as follows (running config/NIPS2017/untargeted/l_inf/I-FGSM.json as an example):

python -u main_attack.py --json-path ./config/NIPS2017/untargeted/l_inf/I-FGSM.json

To fully reproduce the evalutions in BlackboxBench, please run the following .sh files

NIPS2017	Untargetd	Targeted
$l_\infty$	main_NIPS2017_UT_INF.sh	main_NIPS2017_T_INF.sh
$l_2$	main_NIPS2017_UT_2.sh	main_NIPS2017_T_2.sh

💡 Refined models

Transfer-based black-box attacks from Model Perspective refine the basic surrogate model to improve the transferability. If users wish to avoid refineing models on their own, we provide our pretrained checkpoints for LGV, SWA, Bayesian attack on CIFAR10 and NIPS2017:

CIFAR10	ResNet-50	VGG19-bn	Inception-V3	DenseNet-BC
LGV	resnet50	vgg19_bn	inception_v3	densenet
SWA	resnet50	vgg19_bn	inception_v3	densenet
Bayesian attack	resnet50	vgg19_bn	inception_v3	densenet

NIPS2017	ResNet-50	VGG19-bn	Inception-V3	DenseNet-121	ViT-B/16
LGV	resnet50	vgg19_bn	inception_v3	densenet121	vit_b_16
SWA	resnet50	vgg19_bn	inception_v3	densenet121	vit_b_16
Bayesian attack	resnet50	vgg19_bn	inception_v3	densenet121	vit_b_16