Implementation of "Adversarial purification with Score-based generative models", ICML 2021

Related tags

Text Data & NLPadp
Overview

Adversarial Purification with Score-based Generative Models

by Jongmin Yoon, Sung Ju Hwang, Juho Lee

This repository includes the official PyTorch implementation of our paper:

Adversarial Purification with Score-based Generative Models

Jongmin Yoon, Sung Ju Hwang, Juho Lee

the 38th International Conference for Machine Learning (ICML 2021)

ArXiv: https://arxiv.org/abs/2106.06041

What does our work do?

We propose a method that gives adversarial robustness to a neural network model against (stochastic) adversarial attacks by using an Energy-based Model (EBM) trained with Denoising Score Matching (DSM), which is called Adversarial denosing purification (ADP).

Running Codes

Dependency

Run the following command to install some necessary python packages to run our code.

pip install -r requirements.txt

Running code

To run the experiments with adp.py or adp_decision.py, enter the following command.

python main.py --config <config-file>

For example, we provide the example configuration file configs/cifar10_bpda_eot_sigma025_eot15.yml in the repository.

Attack and defense

For adversarial attacks, the classifier PGD attack and BPDA+EOT attack are implemented in attacks/clf_pgd.py and attacks/bpda_strong.py, respectively. At the configuration file, setting the attack.attack_method into clf_pgd or bpda_strong will run these attacks, respectively. For defense, we implemented the main ADP algorithm and ADP after detecting adversarial examples (Appendix F.) in purification/adp.py and purification/adp_decision.py, respectively.

Main components

File name Explanation
main.py Execute the main code, with initializing configurations and loggers.
runners/empirical.py Attacks and purifies the image to show empirical adversarial robustness.
attacks/bpda_strong.py Code for BPDA+EOT attack.
purification/adp.py Code for adversarial purification.
ncsnv2/* Code for training the EBM, i.e., NCSNv2 (paper, code).
networks/* Code for used classifier network architectures.
utils/* Utility files.

Notes

  • For the configuration files, we use the pixel ranges [0, 255] for the perturbation scale attack.ptb and the one-step attack scale attack.alpha. And the main experiments are performed within the pixel range [0, 1] after being rescaled during execution.
  • For training the EBM and classifier models, we primarily used the pre-existing methods such as NCSNv2 and WideResNet classifier. Here is the repository we used for training the WideResNet classifier. Nevertheless, other classifiers, such as the pre-trained adversarially robust classifier implemented in here can be used.

Reference

If you find our work useful for your research, please consider citing this.

@inproceedings{
yoon2021advpur,
title={Adversarial Purification with Score-based Generative Models},
author={Jongmin Yoon and Sung Ju Hwang and Juho Lee},
booktitle={Proceedings of The 38th International Conference on Machine Learning (ICML 2021)},
year={2021},
}

Contact

For further details, please contact [email protected].

License

MIT

UniSpeech - Large Scale Self-Supervised Learning for Speech

UniSpeech The family of UniSpeech: WavLM (arXiv): WavLM: Large-Scale Self-Supervised Pre-training for Full Stack Speech Processing UniSpeech (ICML 202

Microsoft 281 Dec 15, 2022
Blackstone is a spaCy model and library for processing long-form, unstructured legal text

Blackstone Blackstone is a spaCy model and library for processing long-form, unstructured legal text. Blackstone is an experimental research project f

ICLR&D 579 Jan 08, 2023
An ActivityWatch watcher to pose questions to the user and record her answers.

aw-watcher-ask An ActivityWatch watcher to pose questions to the user and record her answers. This watcher uses Zenity to present dialog boxes to the

Bernardo Chrispim Baron 33 Dec 03, 2022
My implementation of Safaricom Machine Learning Codility test. The code has bugs, logical I guess I made errors and any correction will be appreciated.

Safaricom_Codility Machine Learning 2022 The test entails two questions. Question 1 was on Machine Learning. Question 2 was on SQL I ran out of time.

Lawrence M. 1 Mar 03, 2022
Protein Language Model

ProteinLM We pretrain protein language model based on Megatron-LM framework, and then evaluate the pretrained model results on TAPE (Tasks Assessing P

THUDM 77 Dec 27, 2022
Ptorch NLU, a Chinese text classification and sequence annotation toolkit, supports multi class and multi label classification tasks of Chinese long text and short text, and supports sequence annotation tasks such as Chinese named entity recognition, part of speech tagging and word segmentation.

Pytorch-NLU,一个中文文本分类、序列标注工具包,支持中文长文本、短文本的多类、多标签分类任务,支持中文命名实体识别、词性标注、分词等序列标注任务。 Ptorch NLU, a Chinese text classification and sequence annotation toolkit, supports multi class and multi label classifi

186 Dec 24, 2022
Header-only C++ HNSW implementation with python bindings

Hnswlib - fast approximate nearest neighbor search Header-only C++ HNSW implementation with python bindings. NEWS: version 0.6 Thanks to (@dyashuni) h

2.3k Jan 05, 2023
Code for using and evaluating SpanBERT.

SpanBERT This repository contains code and models for the paper: SpanBERT: Improving Pre-training by Representing and Predicting Spans. If you prefer

Meta Research 798 Dec 30, 2022
Implementaion of our ACL 2022 paper Bridging the Data Gap between Training and Inference for Unsupervised Neural Machine Translation

Bridging the Data Gap between Training and Inference for Unsupervised Neural Machine Translation This is the implementaion of our paper: Bridging the

hezw.tkcw 20 Dec 12, 2022
Simple telegram bot to convert files into direct download link.you can use telegram as a file server 🪁

TGCLOUD 🪁 Simple telegram bot to convert files into direct download link.you can use telegram as a file server 🪁 Features Easy to Deploy Heroku Supp

Mr.Acid dev 6 Oct 18, 2022
Huggingface Transformers + Adapters = ❤️

adapter-transformers A friendly fork of HuggingFace's Transformers, adding Adapters to PyTorch language models adapter-transformers is an extension of

AdapterHub 1.2k Jan 09, 2023
In this project, we compared Spanish BERT and Multilingual BERT in the Sentiment Analysis task.

Applying BERT Fine Tuning to Sentiment Classification on Amazon Reviews Abstract Sentiment analysis has made great progress in recent years, due to th

Alexander Leonardo Lique Lamas 5 Jan 03, 2022
Comprehensive-E2E-TTS - PyTorch Implementation

A Non-Autoregressive End-to-End Text-to-Speech (text-to-wav), supporting a family of SOTA unsupervised duration modelings. This project grows with the research community, aiming to achieve the ultima

Keon Lee 114 Nov 13, 2022
NLP library designed for reproducible experimentation management

Welcome to the Transfer NLP library, a framework built on top of PyTorch to promote reproducible experimentation and Transfer Learning in NLP You can

Feedly 290 Dec 20, 2022
My Implementation for the paper EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks using Tensorflow

Easy Data Augmentation Implementation This repository contains my Implementation for the paper EDA: Easy Data Augmentation Techniques for Boosting Per

Aflah 9 Oct 31, 2022
Implementation of "Adversarial purification with Score-based generative models", ICML 2021

Adversarial Purification with Score-based Generative Models by Jongmin Yoon, Sung Ju Hwang, Juho Lee This repository includes the official PyTorch imp

15 Dec 15, 2022
Python generation script for BitBirds

BitBirds generation script Intro This is published under MIT license, which means you can do whatever you want with it - entirely at your own risk. Pl

286 Dec 06, 2022
Pytorch-Named-Entity-Recognition-with-BERT

BERT NER Use google BERT to do CoNLL-2003 NER ! Train model using Python and Inference using C++ ALBERT-TF2.0 BERT-NER-TENSORFLOW-2.0 BERT-SQuAD Requi

Kamal Raj 1.1k Dec 25, 2022
Auto_code_complete is a auto word-completetion program which allows you to customize it on your needs

auto_code_complete is a auto word-completetion program which allows you to customize it on your needs. the model for this program is one of the deep-learning NLP(Natural Language Process) model struc

RUO 2 Feb 22, 2022
Training open neural machine translation models

Train Opus-MT models This package includes scripts for training NMT models using MarianNMT and OPUS data for OPUS-MT. More details are given in the Ma

Language Technology at the University of Helsinki 167 Jan 03, 2023