Code for the paper: Adversarial Training Against Location-Optimized Adversarial Patches. ECCV-W 2020.

Overview

Adversarial Training Against Location-Optimized Adversarial Patches

arXiv | Paper | Code | Video | Slides

Code for the paper:

Sukrut Rao, David Stutz, Bernt Schiele. (2020) Adversarial Training Against Location-Optimized Adversarial Patches. In: Bartoli A., Fusiello A. (eds) Computer Vision – ECCV 2020 Workshops. ECCV 2020. Lecture Notes in Computer Science, vol 12539. Springer, Cham. https://doi.org/10.1007/978-3-030-68238-5_32

Setup

Requirements

  • Python 3.7 or above
  • PyTorch
  • scipy
  • h5py
  • scikit-image
  • scikit-learn

Optional requirements

To use script to convert data to HDF5 format

  • torchvision
  • Pillow
  • pandas

To use Tensorboard logging

  • tensorboard

With the exception of Python and PyTorch, all requirements can be installed directly using pip:

$ pip install -r requirements.txt

Setting the paths

In common/paths.py, set the following variables:

  • BASE_DATA: base path for datasets.
  • BASE_EXPERIMENTS: base path for trained models and perturbations after attacks.
  • BASE_LOGS: base path for tensorboard logs (if used).

Data

Data needs to be provided in the HDF5 format. To use a dataset, use the following steps:

  • In common/paths.py, set BASE_DATA to the base path where data will be stored.
  • For each dataset, create a directory named <dataset-name> in BASE_DATA
  • Place the following files in this directory:
    • train_images.h5: Training images
    • train_labels.h5: Training labels
    • test_images.h5: Test images
    • test_labels.h5: Test labels

A script create_dataset_h5.py has been provided to convert data in a comma-separated CSV file consisting of full paths to images and their corresponding labels to a HDF5 file. To use this script, first set BASE_DATA in common/paths.py. If the files containing training and test data paths and labels are train.csv and test.csv respectively, use:

$ python scripts/create_dataset_h5.py --train_csv /path/to/train.csv --test_csv /path/to/test.csv --dataset dataset_name

where dataset_name is the name for the dataset.

Training and evaluating a model

Training

To train a model, use:

$ python scripts/train.py [options]

A list of available options and their descriptions can be found by using:

$ python scripts/train.py -h

Evaluation

To evaluate a trained model, use:

$ python scripts/evaluate.py [options]

A list of available options and their descriptions can be found by using:

$ python scripts/evaluate.py -h

Using models and attacks from the paper

The following provides the arguments to use with the training and evaluation scripts to train the models and run the attacks described in the paper. The commands below assume that the dataset is named cifar10 and has 10 classes.

Models

Normal

$ python scripts/train.py --cuda --dataset cifar10 --n_classes 10 --cuda --mode normal --log_dir logs --snapshot_frequency 5 --models_dir models --use_tensorboard --use_flip

Occlusion

$ python scripts/train.py --cuda --dataset cifar10 --n_classes 10 --mask_dims 8 8 --mode adversarial --location random --exclude_box 11 11 10 10 --epsilon 0.1 --signed_grad --max_iterations 1 --log_dir logs --snapshot_frequency 5 --models_dir models --use_tensorboard --use_flip

AT-Fixed

$ python scripts/train.py --cuda --dataset cifar10 --n_classes 10 --mask_pos 3 3 --mask_dims 8 8 --mode adversarial --location fixed --exclude_box 11 11 10 10 --epsilon 0.1 --signed_grad --max_iterations 25 --log_dir logs --snapshot_frequency 5 --models_dir models --use_tensorboard --use_flip

AT-Rand

$ python scripts/train.py --cuda --dataset cifar10 --n_classes 10 --mask_dims 8 8 --mode adversarial --location random --exclude_box 11 11 10 10 --epsilon 0.1 --signed_grad --max_iterations 25 --log_dir logs --snapshot_frequency 5 --models_dir models --use_tensorboard --use_flip

AT-RandLO

$ python scripts/train.py --cuda --dataset cifar10 --n_classes 10 --mask_dims 8 8 --mode adversarial --location random --exclude_box 11 11 10 10 --epsilon 0.1 --signed_grad --max_iterations 25 --optimize_location --opt_type random --stride 2 --log_dir logs --snapshot_frequency 5 --models_dir models --use_tensorboard --use_flip

AT-FullLO

$ python scripts/train.py --cuda --dataset cifar10 --n_classes 10 --mask_dims 8 8 --mode adversarial --location random --exclude_box 11 11 10 10 --epsilon 0.1 --signed_grad --max_iterations 25 --optimize_location --opt_type full --stride 2 --log_dir logs --snapshot_frequency 5 --models_dir models --use_tensorboard --use_flip

Attacks

The arguments used here correspond to using 100 iterations and 30 attempts. These can be changed by appropriately setting --iterations and --attempts respectively.

AP-Fixed

$ python scripts/evaluate.py --cuda --dataset cifar10 --n_classes 10 --mask_pos 3 3 --mask_dims 8 8 --mode adversarial --log_dir logs --models_dir models --saved_model_file model_complete_200 --attempts 30 --location fixed --epsilon 0.05 --iterations 100 --signed_grad --perturbations_file perturbations --use_tensorboard

AP-Rand

$ python scripts/evaluate.py --cuda --dataset cifar10 --n_classes 10 --mask_dims 8 8 --mode adversarial --log_dir logs --models_dir models --saved_model_file model_complete_200 --attempts 30 --location random --epsilon 0.05 --iterations 100 --exclude_box 11 11 10 10 --signed_grad --perturbations_file perturbations --use_tensorboard

AP-RandLO

$ python scripts/evaluate.py --cuda --dataset cifar10 --n_classes 10 --mask_dims 8 8 --mode adversarial --log_dir logs --models_dir models --saved_model_file model_complete_200 --attempts 30 --location random --epsilon 0.05 --iterations 100 --exclude_box 11 11 10 10 --optimize_location --opt_type random --stride 2 --signed_grad --perturbations_file perturbations --use_tensorboard

AP-FullLO

$ python scripts/evaluate.py --cuda --dataset cifar10 --n_classes 10 --mask_dims 8 8 --mode adversarial --log_dir logs --models_dir models --saved_model_file model_complete_200 --attempts 30 --location random --epsilon 0.05 --iterations 100 --exclude_box 11 11 10 10 --optimize_location --opt_type full --stride 2 --signed_grad --perturbations_file perturbations --use_tensorboard

Citation

Please cite the paper as follows:

@InProceedings{Rao2020Adversarial,
author = {Sukrut Rao and David Stutz and Bernt Schiele},
title = {Adversarial Training Against Location-Optimized Adversarial Patches},
booktitle = {Computer Vision -- ECCV 2020 Workshops},
year = {2020},
editor = {Adrien Bartoli and Andrea Fusiello},
publisher = {Springer International Publishing},
address = {Cham},
pages = {429--448},
isbn = {978-3-030-68238-5}
}

Acknowledgement

This repository uses code from davidstutz/confidence-calibrated-adversarial-training.

License

Copyright (c) 2020 Sukrut Rao, David Stutz, Max-Planck-Gesellschaft

Please read carefully the following terms and conditions and any accompanying documentation before you download and/or use this software and associated documentation files (the "Software").

The authors hereby grant you a non-exclusive, non-transferable, free of charge right to copy, modify, merge, publish, distribute, and sublicense the Software for the sole purpose of performing non-commercial scientific research, non-commercial education, or non-commercial artistic projects.

Any other use, in particular any use for commercial purposes, is prohibited. This includes, without limitation, incorporation in a commercial product, use in a commercial service, or production of other artefacts for commercial purposes.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

You understand and agree that the authors are under no obligation to provide either maintenance services, update services, notices of latent defects, or corrections of defects with regard to the Software. The authors nevertheless reserve the right to update, modify, or discontinue the Software at any time.

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. You agree to cite the corresponding papers (see above) in documents and papers that report on research using the Software.

TensorFlow-based implementation of "ICNet for Real-Time Semantic Segmentation on High-Resolution Images".

ICNet_tensorflow This repo provides a TensorFlow-based implementation of paper "ICNet for Real-Time Semantic Segmentation on High-Resolution Images,"

HsuanKung Yang 406 Nov 27, 2022
RuDOLPH: One Hyper-Modal Transformer can be creative as DALL-E and smart as CLIP

[Paper] [Хабр] [Model Card] [Colab] [Kaggle] RuDOLPH 🦌 🎄 ☃️ One Hyper-Modal Transformer can be creative as DALL-E and smart as CLIP Russian Diffusio

AI Forever 232 Jan 04, 2023
Public Implementation of ChIRo from "Learning 3D Representations of Molecular Chirality with Invariance to Bond Rotations"

Learning 3D Representations of Molecular Chirality with Invariance to Bond Rotations This directory contains the model architectures and experimental

35 Dec 05, 2022
Unofficial pytorch implementation of paper "One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing"

One-Shot Free-View Neural Talking Head Synthesis Unofficial pytorch implementation of paper "One-Shot Free-View Neural Talking-Head Synthesis for Vide

ZLH 406 Dec 23, 2022
High-Resolution Image Synthesis with Latent Diffusion Models

Latent Diffusion Models Requirements A suitable conda environment named ldm can be created and activated with: conda env create -f environment.yaml co

CompVis Heidelberg 5.6k Jan 04, 2023
Align and Prompt: Video-and-Language Pre-training with Entity Prompts

ALPRO Align and Prompt: Video-and-Language Pre-training with Entity Prompts [Paper] Dongxu Li, Junnan Li, Hongdong Li, Juan Carlos Niebles, Steven C.H

Salesforce 127 Dec 21, 2022
MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation

MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation This repo is the official implementation of "MHFormer: Multi-Hypothesis Transforme

Vegetabird 281 Jan 07, 2023
Out-of-boundary View Synthesis towards Full-frame Video Stabilization

Out-of-boundary View Synthesis towards Full-frame Video Stabilization Introduction | Update | Results Demo | Introduction This repository contains the

25 Oct 10, 2022
MoViNets PyTorch implementation: Mobile Video Networks for Efficient Video Recognition;

MoViNet-pytorch Pytorch unofficial implementation of MoViNets: Mobile Video Networks for Efficient Video Recognition. Authors: Dan Kondratyuk, Liangzh

189 Dec 20, 2022
Code for all the Advent of Code'21 challenges mostly written in python

Advent of Code 21 Code for all the Advent of Code'21 challenges mostly written in python. They are not necessarily the best or fastest solutions but j

4 May 26, 2022
The project of phase's key role in complex and real NN

Phase-in-NN This is the code for our project at Princeton (co-authors: Yuqi Nie, Hui Yuan). The paper title is: "Neural Network is heterogeneous: Phas

YuqiNie-lab 1 Nov 04, 2021
SoK: Vehicle Orientation Representations for Deep Rotation Estimation

SoK: Vehicle Orientation Representations for Deep Rotation Estimation Raymond H. Tu, Siyuan Peng, Valdimir Leung, Richard Gao, Jerry Lan This is the o

FIRE Capital One Machine Learning of the University of Maryland 12 Oct 07, 2022
Monify: an Expense tracker Program implemented in a Graphical User Interface that allows users to keep track of their expenses

💳 MONIFY (EXPENSE TRACKER PRO) 💳 Description Monify is an Expense tracker Program implemented in a Graphical User Interface allows users to add inco

Moyosore Weke 1 Dec 14, 2021
BisQue is a web-based platform designed to provide researchers with organizational and quantitative analysis tools for 5D image data. Users can extend BisQue by implementing containerized ML workflows.

Overview BisQue is a web-based platform specifically designed to provide researchers with organizational and quantitative analysis tools for up to 5D

Vision Research Lab @ UCSB 26 Nov 29, 2022
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising

Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising

Kai Zhang 1.2k Dec 29, 2022
Official codebase for ICLR oral paper Unsupervised Vision-Language Grammar Induction with Shared Structure Modeling

CLIORA This is the official codebase for ICLR oral paper: Unsupervised Vision-Language Grammar Induction with Shared Structure Modeling. We introduce

Bo Wan 32 Dec 23, 2022
📝 Wrapper library for text generation / language models at char and word level with RNN in TensorFlow

tensorlm Generate Shakespeare poems with 4 lines of code. Installation tensorlm is written in / for Python 3.4+ and TensorFlow 1.1+ pip3 install tenso

Kilian Batzner 63 May 22, 2021
The Curious Layperson: Fine-Grained Image Recognition without Expert Labels (BMVC 2021)

The Curious Layperson: Fine-Grained Image Recognition without Expert Labels Subhabrata Choudhury, Iro Laina, Christian Rupprecht, Andrea Vedaldi Code

Subhabrata Choudhury 18 Dec 27, 2022
Pytorch implementation of "MOSNet: Deep Learning based Objective Assessment for Voice Conversion"

MOSNet pytorch implementation of "MOSNet: Deep Learning based Objective Assessment for Voice Conversion" https://arxiv.org/abs/1904.08352 Dependency L

9 Nov 18, 2022
Pytorch implementation of OCNet series and SegFix.

openseg.pytorch News 2021/09/14 MMSegmentation has supported our ISANet and refer to ISANet for more details. 2021/08/13 We have released the implemen

openseg-group 1.1k Dec 23, 2022