[ACL-IJCNLP 2021] "EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets"

Last update: May 11, 2022

Related tags

Overview

EarlyBERT

This is the official implementation for the paper in ACL-IJCNLP 2021 "EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets" by Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Zhangyang Wang and Jingjing Liu.

The code base consists of the finetuning and pretraining parts.

Citation

If you use this code base for your research project, please include this work as reference

@inproceedings{chen2021earlybert,
  title={EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets},
  author={Chen, Xiaohan and Cheng, Yu and Wang, Shuohang and Gan, Zhe and Wang,
  Zhangyang and Liu, Jingjing},
  booktitle={Proceedings of the Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing},
  year={2021}
}

Owner

VITA

Visual Informatics Group @ University of Texas at Austin

GitHub Repository

Simple ray intersection library similar to coldet - succedeed by libacc

Ray Intersection This project offers a header only acceleration structure library including implementations for a BVH- and KD-Tree. Applications may i

29 Jun 23, 2022

Code for CVPR2021 paper "Robust Reflection Removal with Reflection-free Flash-only Cues"

Robust Reflection Removal with Reflection-free Flash-only Cues (RFC) Paper | To be released: Project Page | Video | Data Tensorflow implementation for

162 Jan 05, 2023

Machine Learning Privacy Meter: A tool to quantify the privacy risks of machine learning models with respect to inference attacks, notably membership inference attacks

ML Privacy Meter Machine learning is playing a central role in automated decision making in a wide range of organization and service providers. The da

357 Jan 06, 2023

Code release for "MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound"

merlot_reserve Code release for "MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound" MERLOT Reserve (in submission) is a mo

92 Dec 11, 2022

Example how to deploy deep learning model with aiohttp.

aiohttp-demos Demos for aiohttp project. Contents Imagetagger Deep Learning Image Classifier URL shortener Toxic Comments Classifier Moderator Slack B

661 Jan 04, 2023

BT-Unet: A-Self-supervised-learning-framework-for-biomedical-image-segmentation-using-Barlow-Twins

BT-Unet: A-Self-supervised-learning-framework-for-biomedical-image-segmentation-using-Barlow-Twins Deep learning has brought most profound contributio

12 Dec 04, 2022

Annealed Flow Transport Monte Carlo

Annealed Flow Transport Monte Carlo Open source implementation accompanying ICML 2021 paper by Michael Arbel*, Alexander G. D. G. Matthews* and Arnaud

30 Nov 21, 2022

[ICML 2020] Prediction-Guided Multi-Objective Reinforcement Learning for Continuous Robot Control

PG-MORL This repository contains the implementation for the paper Prediction-Guided Multi-Objective Reinforcement Learning for Continuous Robot Contro

65 Jan 07, 2023

Here is the implementation of our paper S2VC: A Framework for Any-to-Any Voice Conversion with Self-Supervised Pretrained Representations.

S2VC Here is the implementation of our paper S2VC: A Framework for Any-to-Any Voice Conversion with Self-Supervised Pretrained Representations. In thi

81 Dec 15, 2022

🍀 Pytorch implementation of various Attention Mechanisms, MLP, Re-parameter, Convolution, which is helpful to further understand papers.⭐⭐⭐

7.7k Jan 05, 2023

The code for paper Efficiently Solve the Max-cut Problem via a Quantum Qubit Rotation Algorithm

Quantum Qubit Rotation Algorithm Single qubit rotation gates $$ U(\Theta)=\bigotimes_{i=1}^n R_x (\phi_i) $$ QQRA for the max-cut problem This code wa

0 Oct 18, 2021

[ACL-IJCNLP 2021] "EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets"

Related tags

Overview

EarlyBERT

Citation

Owner

VITA

Simple ray intersection library similar to coldet - succedeed by libacc

Code for CVPR2021 paper "Robust Reflection Removal with Reflection-free Flash-only Cues"

Machine Learning Privacy Meter: A tool to quantify the privacy risks of machine learning models with respect to inference attacks, notably membership inference attacks

Code release for "MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound"

Example how to deploy deep learning model with aiohttp.

BT-Unet: A-Self-supervised-learning-framework-for-biomedical-image-segmentation-using-Barlow-Twins

Annealed Flow Transport Monte Carlo

[ICML 2020] Prediction-Guided Multi-Objective Reinforcement Learning for Continuous Robot Control

Here is the implementation of our paper S2VC: A Framework for Any-to-Any Voice Conversion with Self-Supervised Pretrained Representations.

Pytorch cuda extension of grid_sample1d

dualFace: Two-Stage Drawing Guidance for Freehand Portrait Sketching (CVMJ)

Technical Analysis Indicators - Pandas TA is an easy to use Python 3 Pandas Extension with 130+ Indicators

PyTorch Implementation of Region Similarity Representation Learning (ReSim)

VIsually-Pivoted Audio and(N) Text

Script utilizando OpenCV e modelo Machine Learning para detectar o uso de máscaras.

Simple Text-Generator with OpenAI gpt-2 Pytorch Implementation

Automated Evidence Collection for Fake News Detection

Algorithmic trading using machine learning.

🍀 Pytorch implementation of various Attention Mechanisms, MLP, Re-parameter, Convolution, which is helpful to further understand papers.⭐⭐⭐

The code for paper Efficiently Solve the Max-cut Problem via a Quantum Qubit Rotation Algorithm