About normalize in evaluate.py

[2480667859]

In evaluate.py, if normalize=False, then the experiment result of AT is

• Nature Acc: 0.8173
• Attack_type: [fgsm] done, acc: 0.5700
• Attack_type: [pgd] done, acc: 0.5231
• Attack_type: [mifgsm] done, acc: 0.5529

If normalize=False, then the result of the experiment AT is

• Nature Acc: 0.8173
• Attack_type: [fgsm] done, acc: 0.3573
• Attack_type: [pgd] done, acc: 0.3524
• Attack_type: [mifgsm] done, acc: 0.3527

This is why？

[UniSerj]

In evaluate.py, if the norm is set to False, we will move the data normalization (transforms.Normalize in dataloader) to the model side. If the norm is set to True, data normalization follows the standard processing (transforms.Normalize). Note that in evaluate.py, we are using the torchattack lib to evaluate various attacks, and the attacking code assumes that the input data X is unnormalized to guarantee the perturbation size follows the standard, such as 8/255. The robust accuracy can be inaccurate if you normalize the data in the dataloader and feed normalized data to the attacking code since the perturbation size has been changed. Thus, to evaluate the models that are trained with data normalization, we move the data normalization to the model side so that they can be evaluated via torchattack.