SeqAttack: a framework for adversarial attacks on token classification models

Overview

SeqAttack: a framework for adversarial attacks on token classification models

SeqAttack is a framework for conducting adversarial attacks against Named Entity Recognition (NER) models and for data augmentation. This library is heavily based on the popular TextAttack framework, and can similarly be used for:

  • Understanding models by running adversarial attacks against them and observing their shortcomings
  • Develop new attack strategies
  • Guided data augmentation, generating additional training samples that can be used to fix a model's shortcomings

The SeqAttack paper is available here.

Setup

Run pip install -r requirements.txt and you're good to go! If you want to run experiments on a fresh virtual machine, check out scripts/gcp.sh which installs all system dependencies for running the code.

The code was tested with python 3.7, if you're using a different version your mileage may vary.

Usage

The main features of the framework are available via the command line interface, wrapped by cli.py. The following subsections describe the usage of the various commands.

Attack

Attacks are executed via the python cli.py attack subcommand. Attack commands are split in two parts:

  • General setup: options common to all adversarial attacks (e.g. model, dataset...)
  • Attack specific setup: options specific to a particular attack strategy

Thus, a typical attack command might look like the following:

python cli.py attack [general-options] attack-recipe [recipe-options]

For example, if we want to attack dslim/bert-base-NER, a NER model trained on CoNLL2003 using deepwordbug as the attack strategy we might run:

python cli.py attack                                            \
       --model-name dslim/bert-base-NER                         \
       --output-path output-dataset.json                        \
       --cache                                                  \
       --dataset-config configs/conll2003-config.json           \
       deepwordbug

The dataset configuration file, configs/conll2003-config.json defines:

  • The dataset path or name (in the latter case it will be downloaded from HuggingFace)
  • The split (e.g. train, test). Only for HuggingFace datasets
  • The human-readable names (a mapping between numerical labels and textual labels), given as a list
  • A labels map, used to remap the dataset's ground truth to align it with the model output as needed. This field can be null if no remapping is needed

In the example above, labels_map is used to align the dataset labels to the output from dslim/bert-base-NER. The dataset labels are the following:

O (0), B-PER (1), I-PER (2), B-ORG (3), I-ORG (4) B-LOC (5), I-LOC (6) B-MISC (7), I-MISC (8)

while the model labels are:

O (0), B-MISC (1), I-MISC (2), B-PER (3), I-PER (4) B-ORG (5), I-ORG (6) B-LOC (7), I-LOC (8)

Thus a remapping is needed and labels_map takes care of it.


The available attack strategies are the following:

Attack Strategy Transformation Constraints Paper
BAE word swap USE sentence cosine similarity https://arxiv.org/abs/2004.01970
BERT-Attack word swap USE sentence cosine similarity, Maximum words perturbed https://arxiv.org/abs/2004.09984
CLARE word swap and insertion USE sentence cosine similarity https://arxiv.org/abs/2009.07502
DeepWordBug character insertion, deletion, swap (ab --> ba) and substitution Levenshtein edit distance https://arxiv.org/abs/1801.04354
Morpheus inflection word swap https://www.aclweb.org/anthology/2020.acl-main.263.pdf
SCPN paraphrasing https://www.aclweb.org/anthology/N18-1170
TextFooler word swap USE sentence cosine similarity, POS match, word-embedding distance https://arxiv.org/abs/1907.11932

The table above is based on this table. In addition to the constraints shown above the attack strategies are also forbidden from modifying and inserting named entities by default.

Evaluation

To evaluate a model against a standard dataset run:

python cli.py evaluate                  \
       --model dslim/bert-base-NER      \
       --dataset conll2003              \
       --split test                     \
       --mode strict                    \

To evaluate the effectivenes of an attack run the following command:

python cli.py evaluate                                  \
       --model dslim/bert-base-NER                      \
       --attacked-dataset experiments/deepwordbug.json  \
       --mode strict                                    \

The above command will compute and display the metrics for the original predictions and their adversarial counterparts.

The evaluation is based on seqeval

Dataset selection

Given a dataset, our victim model may be able to predict some dataset samples perfectly, but it may produce significant errors on others. To evaluate an attack's effectiveness we may want to select samples with a small initial misprediction score. This can be done via the following command:

python cli.py pick-samples                              \
       --model dslim/bert-base-NER                      \
       --dataset-config configs/conll2003-config.json   \
        --max-samples 256                               \
       --max-initial-score 0.5                          \ # The maximum initial misprediction score
       --output-filename cherry-picked.json             \
       --goal-function untargeted

Tests

Tests can be run with pytest

Adversarial examples visualization

The output datasets can be visualized with SeqAttack-Visualization

Owner
Walter
Software Developer from 🇮🇹 based in 🇳🇱
Walter
Multi-Person Extreme Motion Prediction

Multi-Person Extreme Motion Prediction Implementation for paper Wen Guo, Xiaoyu Bie, Xavier Alameda-Pineda, Francesc Moreno-Noguer, Multi-Person Extre

GUO-W 38 Nov 15, 2022
Official PyTorch Implementation of HELP: Hardware-adaptive Efficient Latency Prediction for NAS via Meta-Learning (NeurIPS 2021 Spotlight)

[NeurIPS 2021 Spotlight] HELP: Hardware-adaptive Efficient Latency Prediction for NAS via Meta-Learning [Paper] This is Official PyTorch implementatio

42 Nov 01, 2022
PERIN is Permutation-Invariant Semantic Parser developed for MRP 2020

PERIN: Permutation-invariant Semantic Parsing David Samuel & Milan Straka Charles University Faculty of Mathematics and Physics Institute of Formal an

ÚFAL 40 Jan 04, 2023
Data for "Driving the Herd: Search Engines as Content Influencers" paper

herding_data Data for "Driving the Herd: Search Engines as Content Influencers" paper Dataset description The collection contains 2250 documents, 30 i

0 Aug 17, 2021
Yoga - Yoga asana classifier for python

Yoga Asana Classifier Description Hi welcome to my new deep learning project "Yo

Programminghut 35 Dec 12, 2022
Pairwise Learning for Neural Link Prediction for OGB (PLNLP-OGB)

Pairwise Learning for Neural Link Prediction for OGB (PLNLP-OGB) This repository provides evaluation codes of PLNLP for OGB link property prediction t

Zhitao WANG 31 Oct 10, 2022
DeepMReye: magnetic resonance-based eye tracking using deep neural networks

DeepMReye: magnetic resonance-based eye tracking using deep neural networks

73 Dec 21, 2022
This is the repo for Uncertainty Quantification 360 Toolkit.

UQ360 The Uncertainty Quantification 360 (UQ360) toolkit is an open-source Python package that provides a diverse set of algorithms to quantify uncert

International Business Machines 207 Dec 30, 2022
HyperDict - Self linked dictionary in Python

Hyper Dictionary Advanced python dictionary(hash-table), which can link it-self

8 Feb 06, 2022
A FAIR dataset of TCV experimental results for validating edge/divertor turbulence models.

TCV-X21 validation for divertor turbulence simulations Quick links Intro Welcome to TCV-X21. We're glad you've found us! This repository is designed t

0 Dec 18, 2021
Code and Experiments for ACL-IJCNLP 2021 Paper Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answering.

Code and Experiments for ACL-IJCNLP 2021 Paper Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answering.

Sidd Karamcheti 50 Nov 16, 2022
This repository contains all source code, pre-trained models related to the paper "An Empirical Study on GANs with Margin Cosine Loss and Relativistic Discriminator"

An Empirical Study on GANs with Margin Cosine Loss and Relativistic Discriminator This is a Pytorch implementation for the paper "An Empirical Study o

Cuong Nguyen 3 Nov 15, 2021
TAug :: Time Series Data Augmentation using Deep Generative Models

TAug :: Time Series Data Augmentation using Deep Generative Models Note!!! The package is under development so be careful for using in production! Fea

35 Dec 06, 2022
Code for our paper "Sematic Representation for Dialogue Modeling" in ACL2021

AMR-Dialogue An implementation for paper "Semantic Representation for Dialogue Modeling". You may find our paper here. Requirements python 3.6 pytorch

xfbai 45 Dec 26, 2022
BirdCLEF 2021 - Birdcall Identification 4th place solution

BirdCLEF 2021 - Birdcall Identification 4th place solution My solution detail kaggle discussion Inference Notebook (best submission) Environment Use K

tattaka 42 Jan 02, 2023
Annotate with anyone, anywhere.

h h is the web app that serves most of the https://hypothes.is/ website, including the web annotations API at https://hypothes.is/api/. The Hypothesis

Hypothesis 2.6k Jan 08, 2023
Twins: Revisiting the Design of Spatial Attention in Vision Transformers

Twins: Revisiting the Design of Spatial Attention in Vision Transformers Very recently, a variety of vision transformer architectures for dense predic

482 Dec 18, 2022
FAIR's research platform for object detection research, implementing popular algorithms like Mask R-CNN and RetinaNet.

Detectron is deprecated. Please see detectron2, a ground-up rewrite of Detectron in PyTorch. Detectron Detectron is Facebook AI Research's software sy

Facebook Research 25.5k Jan 07, 2023
A curated list of awesome game datasets, and tools to artificial intelligence in games

🎮 Awesome Game Datasets In computer science, Artificial Intelligence (AI) is intelligence demonstrated by machines. Its definition, AI research as th

Leonardo Mauro 454 Jan 03, 2023
PyTorch implementation of MICCAI 2018 paper "Liver Lesion Detection from Weakly-labeled Multi-phase CT Volumes with a Grouped Single Shot MultiBox Detector"

Grouped SSD (GSSD) for liver lesion detection from multi-phase CT Note: the MICCAI 2018 paper only covers the multi-phase lesion detection part of thi

Sang-gil Lee 36 Oct 12, 2022