Implementation of the paper 'Sentence Bottleneck Autoencoders from Transformer Language Models'

Overview

Introduction

This repository contains the code for the paper Sentence Bottleneck Autoencoders from Transformer Language Models by Ivan Montero, Nikolaos Pappas, and Noah A. Smith published at EMNLP 2021.

This paper proposes an approach to learning sentence representations by applying an autoencoder on top of pretrained masked LMs.

Further documentation and code cleanup are under the works!

Installation

Coming soon!

This code was tested under Ubuntu 18.04, Python 3.7, and PyTorch 1.6.

To install, run the setup file.

bash setup.sh

We use modified versions of the following repositories:

transformers==3.3.1
fairseq==0.9.0

Example Usage

Coming soon!

Modified Files

Coming soon!

Owner
Ivan Montero
UW Undergrad | NLP Research Previously: @Apple, @Google, @Facebook
Ivan Montero
Boto is a Python package that provides interfaces to Amazon Web Services.

Boto is a Python package that provides interfaces to Amazon Web Services.

the boto project 6.5k Jan 01, 2023
Telegram tools

Telegram-Tools Telegram tools. Explanation English | 中文 Features Export group memebrs Add users to the group Send message to users Setup API Open http

4 Apr 02, 2022
Console XMPP client in python

poezio Homepage: https://poez.io Forge Page: https://lab.louiz.org/poezio/poezio Poezio is a console Jabber/XMPP client. The initial goal was to provi

48 Dec 19, 2022
Typed interactions with the GitHub API v3

PyGitHub PyGitHub is a Python library to access the GitHub API v3 and Github Enterprise API v3. This library enables you to manage GitHub resources su

5.7k Jan 06, 2023
Telegram bot for logistic - Telegram bot for logistic

Демонстрационный телеграм-бот для нужд транспортной компании Цель проекта Реализ

M1chigun 1 Feb 05, 2022
You can connect with Sanila Ranatunga using this bot😉😉

Sanila-Ranatunga-s-Assistant-Bot You can connect with Sanila Ranatunga using this bot 😉 😉 Reach me on Telegram Sanila's Assistant Bot What is Telegr

Sanila Ranatunga 5 Feb 01, 2022
MemeBot - A discord bot that tracks how good people's memes are

MemeBot A discord Meme "Karma" Tracking bot Dependancies Make sure you have pymongo installed and a mongodb cluster setup with two collections. pip in

Uday Sharma 3 Aug 10, 2022
Gets instagram public username and returns usefull informations like profilepic(b64), video_urls etc.

InstaSucker Gets instagram public username and returns usefull informations like profilepic(b64), video_urls etc. Information this project contains a

Armin Amiri 5 Apr 30, 2022
Discord ToolBox is a discord bot developed by DJD320 created for the purpose of having some convenient tools in the form of a single bot.

Discord ToolBox Discord ToolBox is a discord bot developed by DJD320 created for the purpose of having some convenient tools in the form of a single b

3 Aug 07, 2021
Python bindings to the Syncthing REST interface.

python-syncthing Python bindings to the Syncthing REST interface. Python API Documentation Syncthing Syncthing REST Documentation Syncthing Forums $ p

Blake VandeMerwe 64 Aug 13, 2022
A discord token grabber made in Python 3

Discord Token Grabber A Discord token grabber written in Python 3. This version of the grabber only supports Windows. Features Transfers via Discord w

Mega145 4 Aug 04, 2022
Tiktok-bot - A tiktok bot with python

Install the requirements pip install selenium pip install pyfiglet==0.7.5 How ca

Ukis 5 Aug 23, 2022
Cloudshell-sandbox-reporter - Helper modules and classes for writing to Cloudshell sandbox console

Cloudshell Sandbox Reporter This project provides utility classes for formatting

QualiLab 2 Sep 07, 2022
Modified Version of mega.py package for Pyrogram Bots

Pyro Mega.py Python library for the Mega.co.nz API, currently supporting: login uploading downloading deleting searching sharing renaming moving files

I'm Not A Bot #Left_TG 10 Aug 03, 2022
Dynamic Twitter banner, to show off your spotify status. Banner updated every 5 minutes.

Spotify Twitter Banner Dynamic Twitter banner, to show off your spotify status. Banner updated every 5 minutes. Installation and Usage Install the dep

Sunrit Jana 23 Jan 05, 2023
A Chip-8 emulator written using Python's default libraries

Chippure A Chip-8 emulator written using Python's default libraries. Instructions: Simply launch the .py file and type the name of the Chip8 ROM you w

5 Sep 27, 2022
A pyrogram simple bot for Educational purpose.

A pyrogram simple bot for Educational purpose. To Learn More check at @PyrogramBot or on Documentation Mandatory variables API_ID - Get It From my.tel

SpamShield 10 Dec 06, 2022
This is RequestTrackerBot and it used for tracking request made by user in a group

This is a Request Tracker Bot repo, It is for those who upload content like movies, anime, etc. It can be used for tracking request of content that your members asked for.

Abhijeet 27 Dec 29, 2022
a simple floating window for watch cryptocurrency price

floating-monitor with cryptocurrency 浮動視窗虛擬貨幣價格監控 a floating monitor window to show price of cryptocurrency. use binance api to get price 半透明的浮動視窗讓你方便

Lin_Yi_Shen 1 Oct 22, 2021
Roaster - this gui app + program bundle roasts.

Roaster - this gui app + program bundle roasts.

Harsh ADV) 1 Jan 04, 2022