[NeurIPS2021] Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks

Overview

Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks

Code for NeurIPS 2021 Paper "Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks" by Hanxun Huang, Yisen Wang, Sarah Monazam Erfani, Quanquan Gu, James Bailey, Xingjun Ma


Robust Configurations for WideResNet (WRN-34-R)

def RobustWideResNet34(num_classes=10):
    # WRN-34-R configurations
    return RobustWideResNet(
        num_classes=num_classes, channel_configs=[16, 320, 640, 512],
        depth_configs=[5, 5, 5], stride_config=[1, 2, 2], stem_stride=1,
        drop_rate_config=[0.0, 0.0, 0.0], zero_init_residual=False,
        block_types=['basic_block', 'basic_block', 'basic_block'],
        activations=['ReLU', 'ReLU', 'ReLU'], is_imagenet=False,
        use_init=True)

Reproduce results from the paper

  • Pretrained Weights for WRN-34-R used in Table 2 available on Google Drive
  • All hyperparameters/settings for each model/method used in Table 2 are stored in configs/*.yaml files.

Evaluations of the robustness of WRN-34-R

WRN-34-R trained with TRADES

Replace PGD with other attacks ['CW', 'GAMA', 'AA'].

python main.py --config_path configs/config-WRN-34-R
               --exp_name /path/to/experiments/folders
               --version WRN-34-R-trades
               --load_best_model --attack PGD --data_parallel
WRN-34-R trained with TRADES and additional 500k data

Replace PGD with other attacks ['CW', 'GAMA', 'AA'].

python main.py --config_path configs/config-WRN-34-R
               --exp_name /path/to/experiments/folders
               --version WRN-34-R-trades-500k
               --load_best_model --attack PGD --data_parallel

Train WRN-34-R with 500k additional data from scratch

python main.py --config_path configs/config-WRN-34-R
               --exp_name /path/to/experiments/folders
               --version WRN-34-R-trades-500k
               --train --data_parallel

CIFAR-10 - Linf AutoAttack Leaderboard using additional 500k data

  • Note: This is not maintained, please find up-to-date leaderboard is available in RobustBench.
# paper model architecture clean report. AA
1 (Gowal et al., 2020) available WRN-70-16 91.10 65.87 65.88
2 Ours‡ + EMA available WRN-34-R 91.23 62.54 62.54
3 Ours available WRN-34-R 90.56 61.56 61.56
4 (Wu et al., 2020a) available WRN-34-15 87.67 60.65 60.65
5 (Wu et al., 2020b) available WRN-28-10 88.25 60.04 60.04
6 (Carmon et al., 2019) available WRN-28-10 89.69 62.5 59.53
7 (Sehwag et al., 2020) available WRN-28-10 88.98 - 57.14
8 (Wang et al., 2020) available WRN-28-10 87.50 65.04 56.29

Citation

@inproceedings{huang2021exploring,
    title={Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks},
    author={Hanxun Huang and Yisen Wang and Sarah Monazam Erfani and Quanquan Gu and James Bailey and Xingjun Ma},
    booktitle={NeurIPS},
    year={2021}
}

Part of the code is based on the following repo:

Owner
Hanxun Huang
Hanxun Huang
LLVM-based compiler for LightGBM gradient-boosted trees. Speeds up prediction by ≥10x.

LLVM-based compiler for LightGBM gradient-boosted trees. Speeds up prediction by ≥10x.

Simon Boehm 183 Jan 02, 2023
Applicator Kit for Modo allow you to apply Apple ARKit Face Tracking data from your iPhone or iPad to your characters in Modo.

Applicator Kit for Modo Applicator Kit for Modo allow you to apply Apple ARKit Face Tracking data from your iPhone or iPad with a TrueDepth camera to

Andrew Buttigieg 3 Aug 24, 2021
Build Low Code Automated Tensorflow, What-IF explainable models in just 3 lines of code.

Build Low Code Automated Tensorflow explainable models in just 3 lines of code.

Hasan Rafiq 170 Dec 26, 2022
Implementation of SETR model, Original paper: Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers.

SETR - Pytorch Since the original paper (Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers.) has no official

zhaohu xing 112 Dec 16, 2022
Occlusion robust 3D face reconstruction model in CFR-GAN (WACV 2022)

Occlusion Robust 3D face Reconstruction Yeong-Joon Ju, Gun-Hee Lee, Jung-Ho Hong, and Seong-Whan Lee Code for Occlusion Robust 3D Face Reconstruction

Yeongjoon 31 Dec 19, 2022
Our solution for SSN Invente 2021's Hackathon

Our solution for SSN Invente 2021's Hackathon. To help maitain godowns in a pristine and safe condition using raspberry pi.

1 Jan 12, 2022
DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.

DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.

Microsoft 8.4k Jan 01, 2023
PyTorch implementation for "Mining Latent Structures with Contrastive Modality Fusion for Multimedia Recommendation"

MIRCO PyTorch implementation for paper: Latent Structures Mining with Contrastive Modality Fusion for Multimedia Recommendation Dependencies Python 3.

Big Data and Multi-modal Computing Group, CRIPAC 9 Dec 08, 2022
Official code repository for the EMNLP 2021 paper

Integrating Visuospatial, Linguistic and Commonsense Structure into Story Visualization PyTorch code for the EMNLP 2021 paper "Integrating Visuospatia

Adyasha Maharana 23 Dec 19, 2022
A model which classifies reviews as positive or negative.

SentiMent Analysis In this project I built a model to classify movie reviews fromn the IMDB dataset of 50K reviews. WordtoVec : Neural networks only w

Rishabh Bali 2 Feb 09, 2022
Pytorch implementation of RED-SDS (NeurIPS 2021).

Recurrent Explicit Duration Switching Dynamical Systems (RED-SDS) This repository contains a reference implementation of RED-SDS, a non-linear state s

Abdul Fatir 10 Dec 02, 2022
Realtime segmentation with ENet, the fast and accurate segmentation net.

Enet This is a realtime segmentation net with almost 22 fps on GTX1080 ti, and the model size is very small with only 28M. This repo contains the infe

JinTian 14 Aug 30, 2022
Inteligência artificial criada para realizar interação social com idosos.

IA SONIA 4.0 A SONIA foi inspirada no assistente mais famoso do mundo e muito bem conhecido JARVIS. Todo mundo algum dia ja sonhou em ter o seu própri

Vinícius Azevedo 2 Oct 21, 2021
A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis

A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis Figure: Shape-Accurate 3D-Aware Image Synthesis. A Shading-Guid

Xingang Pan 115 Dec 18, 2022
nanodet_plus,yolov5_v6.0

OAK_Detection OAK设备上适配nanodet_plus,yolov5_v6.0 Environment pytorch = 1.7.0

炼丹去了 1 Feb 18, 2022
HNN: Human (Hollywood) Neural Network

HNN: Human (Hollywood) Neural Network Learn the top 1000 actors on IMDB with your very own low cost, highly parallel, CUDAless biological neural netwo

Madhava Jay 0 Dec 21, 2021
CS550 Machine Learning course project on CNN Detection.

CNN Detection (CS550 Machine Learning Project) Team Members (Tensor) : Yadava Kishore Chodipilli (11940310) Thashmitha BS (11941250) This is a work do

yaadava_kishore 2 Jan 30, 2022
A Python package for generating concise, high-quality summaries of a probability distribution

GoodPoints A Python package for generating concise, high-quality summaries of a probability distribution GoodPoints is a collection of tools for compr

Microsoft 28 Oct 10, 2022
Multi-Scale Vision Longformer: A New Vision Transformer for High-Resolution Image Encoding

Vision Longformer This project provides the source code for the vision longformer paper. Multi-Scale Vision Longformer: A New Vision Transformer for H

Microsoft 209 Dec 30, 2022
A neuroanatomy-based augmented reality experience powered by computer vision. Features 3D visuals of the Atlas Brain Map slices.

Brain Augmented Reality (AR) A neuroanatomy-based augmented reality experience powered by computer vision that features 3D visuals of the Atlas Brain

Yasmeen Brain 10 Oct 06, 2022