transformers-into-vaes

Code for Finetuning Pretrained Transformers into Variational Autoencoders (our submission to NLP Insights Workshop 2021).

Gathering data used in the paper:

1. Download all data (penn, snli, yahoo, yelp) from this repository.

2. Change data path in base_models.py accordingly.

Running experiments:

1. Install dependencies.

pip install -r requirements.txt

2. Run phase 1 (encoder only training):

./run_encoder_training snli

3. Run phase 2 (full training):

./run_training snli <path_to_checkpoint_from_phase_1>

Calculating metrics:

python evaluate_all.py -d snli -bs 256 -c <path_to_config_file> -ckpt <path_to_checkpoint_file>