Towards Adversarial Robustness of Bayesian Neural Network through Hierarchical Variational Inference [paper]
Baseline of this code is the official repository for this paper. We just replace the BNN regularizer from ELBO with enhanced Bayesian regularizer based on hierarchical-ELBO.
Citation
If you find this work helpful, please cite it as:
@misc{
lee2021towards,
title={Towards Adversarial Robustness of Bayesian Neural Network through Hierarchical Variational Inference},
author={Byung-Kwan Lee and Youngjoon Yu and Yong Man Ro},
year={2021},
url={https://openreview.net/forum?id=Cue2ZEBf12}
}
Hierarchical-Bayeisan-Defense
Dataset
- CIFAR10
- STL10
- CIFAR100
- Tiny-ImageNet
Network
- VGG16 (for CIFAR-10/CIFAR-100/Tiny-ImageNet)
- Aaron (for STL10)
- WideResNet (for CIFAR-10/100)
Attack (by torchattack)
- PGD attack
- EOT-PGD attack
Defense methods
adv: Adversarial trainingadv_vi: Adversarial training with Bayesian neural networkadv_hvi: Adversarial training with Enhanced Bayesian neural network based on hierarchical-ELBO
How to Train
1. Adversarial training
Run train_adv.sh
lr=0.01
steps=10
max_norm=0.03
data=tiny # or `cifar10`, `stl10`, `cifar100`
root=./datasets
model=vgg # vgg for `cifar10` `stl10` `cifar100`, aaron for `stl10`, wide for `cifar10` or `cifar100`
model_out=./checkpoint/${data}_${model}_${max_norm}_adv
echo "Loading: " ${model_out}
CUDA_VISIBLE_DEVICES=0 python ./main_adv.py \
--lr ${lr} \
--step ${steps} \
--max_norm ${max_norm} \
--data ${data} \
--model ${model} \
--root ${root} \
--model_out ${model_out}.pth \
2. Adversarial training with BNN
Run train_adv_vi.sh
lr=0.01
steps=10
max_norm=0.03
sigma_0=0.1
init_s=0.1
data=tiny # or `cifar10`, `stl10`, `cifar100`
root=./datasets
model=vgg # vgg for `cifar10` `stl10` `cifar100`, aaron for `stl10`, wide for `cifar10` or `cifar100`
model_out=./checkpoint/${data}_${model}_${max_norm}_adv_vi
echo "Loading: " ${model_out}
CUDA_VISIBLE_DEVICES=0 python3 ./main_adv_vi.py \
--lr ${lr} \
--step ${steps} \
--max_norm ${max_norm} \
--sigma_0 ${sigma_0} \
--init_s ${init_s} \
--data ${data} \
--model ${model} \
--root ${root} \
--model_out ${model_out}.pth \
3. Adversarial training with enhanced Bayesian regularizer based on hierarchical-ELBO
Run train_adv_hvi.sh
lr=0.01
steps=10
max_norm=0.03
sigma_0=0.1
init_s=0.1
data=tiny # or `cifar10`, `stl10`, `cifar100`
root=./datasets
model=vgg # vgg for `cifar10` `stl10` `cifar100`, aaron for `stl10`, wide for `cifar10` or `cifar100`
model_out=./checkpoint/${data}_${model}_${max_norm}_adv_hvi
echo "Loading: " ${model_out}
CUDA_VISIBLE_DEVICES=0 python3 ./main_adv_hvi.py \
--lr ${lr} \
--step ${steps} \
--max_norm ${max_norm} \
--sigma_0 ${sigma_0} \
--init_s ${init_s} \
--data ${data} \
--model ${model} \
--root ${root} \
--model_out ${model_out}.pth \
How to Test
Testing adversarial robustness
Run acc_under_attack.sh
model=vgg # vgg for `cifar10` `stl10` `cifar100`, aaron for `stl10`, wide for `cifar10` or `cifar100`
defense=adv_hvi # or `adv_vi`, `adv`
data=tiny-imagenet # or `cifar10`, `stl10`, `cifar100`
root=./datasets
n_ensemble=50
step=10
max_norm=0.03
echo "Loading" ./checkpoint/${data}_${model}_${max_norm}_${defense}.pth
CUDA_VISIBLE_DEVICES=0 python3 acc_under_attack.py \
--model $model \
--defense $defense \
--data $data \
--root $root \
--n_ensemble $n_ensemble \
--step $step \
--max_norm $max_norm
How to check the learning parameters and KL divergence
Run check_parameters.sh
model=vgg # vgg for `cifar10` `stl10` `cifar100`, aaron for `stl10`, wide for `cifar10` or `cifar100`
defense=adv_hvi # or `adv_vi`
data=tiny-imagenet # or `cifar10`, `stl10`, `cifar100`
max_norm=0.03
echo "Loading" ./checkpoint/${data}_${model}_${max_norm}_${defense}.pth
CUDA_VISIBLE_DEVICES=0 python3 check_parameters.py \
--model $model \
--defense $defense \
--data $data \
--max_norm $max_norm \
How to check uncertainty by predictive entropy
Run uncertainty.sh
model=vgg # vgg for `cifar10` `stl10` `cifar100`, aaron for `stl10`, wide for `cifar10` or `cifar100`
defense=adv_hvi # or `adv_vi`
data=tiny-imagenet # or `cifar10`, `stl10`, `cifar100`
root=./datasets
n_ensemble=50
step=10
max_norm=0.03
echo "Loading" ./checkpoint/${data}_${model}_${max_norm}_${defense}.pth
CUDA_VISIBLE_DEVICES=0 python3 uncertainty.py \
--model $model \
--defense $defense \
--data $data \
--root $root \
--n_ensemble $n_ensemble \
--step $step \
--max_norm $max_norm
125 Dec 25, 2022
14 Dec 13, 2022
100 Dec 19, 2022
18 Nov 21, 2022
140 Sep 26, 2022
4.4k Jan 01, 2023
55 Jan 04, 2023
1.2k Jan 07, 2023
3.8k Jan 01, 2023
9 Aug 12, 2022
1 Feb 13, 2022
29 Nov 29, 2022
15 Feb 10, 2022
57 Nov 21, 2022
77 Nov 22, 2022
36 Nov 23, 2022
17 Jan 04, 2023
12 Dec 28, 2022
87 Dec 26, 2022
18 Dec 21, 2022