Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning

Overview

structshot

Code and data for paper "Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning", Yi Yang and Arzoo Katiyar, in EMNLP 2020.

Data

Due to license reason, we are only able to release the full CoNLL 2003 and WNUT 2017 dataset. We also release the support sets that we sampled from the CoNLL/WNUT/I2B2 dev sets to enable the reproducing of our evaluation results.

CoNLL 2003

The CoNLL 2003 NER train/dev/test datasets are data/train.txt, data/dev.txt, and data/test.txt respectively. The labels are available in data/labels.txt.

WNUT 2017

The WNUT 2017 NER dev/test datasets are data/dev-wnut.txt and data/test-wnut.txt respectively. The labels are available in data/labels-wnut.txt.

Support sets for CoNLL 2003, WNUT 2017, and I2B2 2014

The one-shot and five-shot support sets used in the paper are available in data/support-* folders.

Usage

Due to data license limitation, we will show how to do five-shot transfer learning from the CoNLL 2003 dataset to the WNUT 2017 dataset, instead of transfering from the OntoNotes 5 dataset, as presented in our paper.

The first step is to install the package and cd into the structshot directory:

pip install -e .
cd structshot

Pretrain BERT-NER model

The marjority of the code is copied from the HuggingFace transformers repo, which is used to pretrain a BERT-NER model:

# Pretrain a conventional BERT-NER model on CoNLL 2003 
bash run_pl.sh

In our paper, we actually merged B- and I- tags together for pretraining as well.

Few-shot NER with NNShot

Given the pretrained model located at output-model/checkpointepoch=2.ckpt, we now can perform five-shot NER transfer on the WNUT test set:

# Five-shot NER with NNShot
bash run_pred.sh output-model/checkpointepoch=2.ckpt NNShot

We use the IO tagging scheme rather than the BIO tagging scheme due to its simplicity and better performance. I obtained 22.8 F1 score.

Few-shot NER with StructShot

Given the same pretrained model, simply run:

# Five-shot NER with StructShot
bash run_pred.sh output-model/checkpointepoch=2.ckpt StructShot

I obtained 29.5 F1 score. You can tune the parameter tau in the run_pred.sh script based on dev set performance.

Notes

There are a few differences between this implementation and the one reported in the paper due to data license reason etc.:

  • This implementation pretrains the BERT-NER model with the BIO tagging scheme, while in our paper we uses the IO tagging scheme.
  • This implementation performs five-shot transfer learning from CoNLL 2003 to WNUT 2017, while in our paper we perform five-shot transfer learning from OntoNotes 5 to CoNLL'03/WNUT'17/I2B2'14.

If you can access OntoNotes 5 and I2B2'14, reproducing the results of the paper should be trivial.

Owner
ASAPP Research
AI for Enterprise
ASAPP Research
Code for the paper "Next Generation Reservoir Computing"

Next Generation Reservoir Computing This is the code for the results and figures in our paper "Next Generation Reservoir Computing". They are written

OSU QuantInfo Lab 105 Dec 20, 2022
for a paper about leveraging discourse markers for training new models

TSLM-DISCOURSE-MARKERS Scope This repository contains: (1) Code to extract discourse markers from wikipedia (TSA). (1) Code to extract significant dis

International Business Machines 6 Nov 02, 2022
Instance Semantic Segmentation List

Instance Semantic Segmentation List This repository contains lists of state-or-art instance semantic segmentation works. Papers and resources are list

bighead 87 Mar 06, 2022
A Game-Theoretic Perspective on Risk-Sensitive Reinforcement Learning

Officile code repository for "A Game-Theoretic Perspective on Risk-Sensitive Reinforcement Learning"

Mathieu Godbout 1 Nov 19, 2021
An implementation of shampoo

shampoo.pytorch An implementation of shampoo, proposed in Shampoo : Preconditioned Stochastic Tensor Optimization by Vineet Gupta, Tomer Koren and Yor

Ryuichiro Hataya 69 Sep 10, 2022
Code for reproducing experiments in "Improved Training of Wasserstein GANs"

Improved Training of Wasserstein GANs Code for reproducing experiments in "Improved Training of Wasserstein GANs". Prerequisites Python, NumPy, Tensor

Ishaan Gulrajani 2.2k Jan 01, 2023
On-device speech-to-intent engine powered by deep learning

Rhino Made in Vancouver, Canada by Picovoice Rhino is Picovoice's Speech-to-Intent engine. It directly infers intent from spoken commands within a giv

Picovoice 510 Dec 30, 2022
A collection of scripts I developed for personal and working projects.

A collection of scripts I developed for personal and working projects Table of contents Introduction Repository diagram structure List of scripts pyth

Gianluca Bianco 109 Dec 26, 2022
A diff tool for language models

LMdiff Qualitative comparison of large language models. Demo & Paper: http://lmdiff.net LMdiff is a MIT-IBM Watson AI Lab collaboration between: Hendr

Hendrik Strobelt 27 Dec 29, 2022
This is the code of using DQN to play Sekiro .

Update for using DQN to play sekiro 2021.2.2(English Version) This is the code of using DQN to play Sekiro . I am very glad to tell that I have writen

144 Dec 25, 2022
For the paper entitled ''A Case Study and Qualitative Analysis of Simple Cross-Lingual Opinion Mining''

Summary This is the source code for the paper "A Case Study and Qualitative Analysis of Simple Cross-Lingual Opinion Mining", which was accepted as fu

1 Nov 10, 2021
This tool converts a Nondeterministic Finite Automata (NFA) into a Deterministic Finite Automata (DFA)

This tool converts a Nondeterministic Finite Automata (NFA) into a Deterministic Finite Automata (DFA)

Quinn Herden 1 Feb 04, 2022
TensorFlow implementation of Elastic Weight Consolidation

Elastic weight consolidation Introduction A TensorFlow implementation of elastic weight consolidation as presented in Overcoming catastrophic forgetti

James Stokes 67 Oct 11, 2022
BigDetection: A Large-scale Benchmark for Improved Object Detector Pre-training

BigDetection: A Large-scale Benchmark for Improved Object Detector Pre-training By Likun Cai, Zhi Zhang, Yi Zhu, Li Zhang, Mu Li, Xiangyang Xue. This

290 Dec 29, 2022
Self-Supervised Pre-Training for Transformer-Based Person Re-Identification

Self-Supervised Pre-Training for Transformer-Based Person Re-Identification [pdf] The official repository for Self-Supervised Pre-Training for Transfo

Hao Luo 116 Jan 04, 2023
Quantify the difference between two arbitrary curves in space

similaritymeasures Quantify the difference between two arbitrary curves Curves in this case are: discretized by inidviudal data points ordered from a

Charles Jekel 175 Jan 08, 2023
ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training

ActNN : Activation Compressed Training This is the official project repository for ActNN: Reducing Training Memory Footprint via 2-Bit Activation Comp

UC Berkeley RISE 178 Jan 05, 2023
TextBPN Adaptive Boundary Proposal Network for Arbitrary Shape Text Detection

TextBPN Adaptive Boundary Proposal Network for Arbitrary Shape Text Detection; Accepted by ICCV2021. Note: The complete code (including training and t

S.X.Zhang 84 Dec 13, 2022
Implementation of Graph Convolutional Networks in TensorFlow

Graph Convolutional Networks This is a TensorFlow implementation of Graph Convolutional Networks for the task of (semi-supervised) classification of n

Thomas Kipf 6.6k Dec 30, 2022
Code for reproducible experiments presented in KSD Aggregated Goodness-of-fit Test.

Code for KSDAgg: a KSD aggregated goodness-of-fit test This GitHub repository contains the code for the reproducible experiments presented in our pape

Antonin Schrab 5 Dec 15, 2022