Utils to quickly evaluate many 🤗 models on the GLUE tasks

Overview

Utils to quickly evaluate many 🤗 models on the GLUE tasks

Setup

wget https://raw.githubusercontent.com/huggingface/transformers/master/examples/pytorch/text-classification/run_glue.py
conda env create -f environment.yaml
conda activate glue-utils

Evaluate multiple models on every glue task:

python runner.py \
    path/to/model_1/ \
    path/to/model_2/ \
    path/to/model_3/ \
    --gpu_idx 1

This will store results in path/to/model_N/glue/[task_name]/, for each model, for each task.

To quickly gather all results, run:

python gather_scores.py \
    path/to/model_1/glue/ \
    path/to/model_2/glue/ \
    path/to/model_3/glue/
Owner
Przemyslaw K. Joniak
Grad student at The University of Tokyo
Przemyslaw K. Joniak
Force you (or your user) annotate Python function type hints.

Must-typing Force you (or your user) annotate function type hints. Notice: It's more like a joke, use it carefully. If you call must_typing in your mo

Konge 13 Feb 19, 2022
Usando Multi Player Perceptron e Regressão Logistica para classificação de SPAM

Relatório dos procedimentos executados e resultados obtidos. Objetivos Treinar um modelo para classificação de SPAM usando o dataset train_data. Class

André Mediote 1 Feb 02, 2022
Devil - Very Semple Auto Filter V1 Bot

Devil Very Semple Auto Filter V1 Bot

2 Jun 27, 2022
This is a simple python script for checking A/L Examination results of srilankan students

AL-Result-Checker This is a simple python script for checking A/L Examination results of srilankan students INSTALLATION [Termux] [Linux] : apt-get up

Razor Kenway 8 Oct 24, 2022
A simple tool made in Python language

Simple tool Uma simples ferramenta feita 100% em linguagem Python 💻 Requisitos: Python3 instalado em seu dispositivo Clonagem e acesso 📳 git clone h

josh washington 4 Dec 07, 2021
A simple API to upload notes or files to KBFS

This API can be used to upload either secure notes or files to a secure KeybaseFS folder.

Dakota Brown 1 Oct 08, 2021
Structured, dependable legos for Starknet development.

cairomate • Structured, dependable legos for starknet development. Directory Structure contracts ├─ defi │ ├─ ChainlinkPriceOracle — "Simple price or

andreas 127 Nov 23, 2022
The official FOSSCOMM 2021 CTF by [email protected]

FOSSCOMM 2021 CTF Table of Contents General Info FAQ General Info Purpose: This CTF is a collaboration between the FOSSCOMM conference and the Machina 2 Nov 14, 2021

Tutorial on Tempo, Beat and Downbeat estimation

Tempo, Beat and Downbeat Estimation By Matthew E. P. Davies, Sebastian Böck and Magdalena Fuentes Resources and Jupyter Book for the ISMIR 2021 tutori

49 Nov 06, 2022
Url-check-migration-python - A python script using Apica API's to migrate URL checks between environments

url-check-migration-python A python script using Apica API's to migrate URL chec

Angelo Aquino 1 Feb 16, 2022
TickerRain is an open-source web app that stores and analysis Reddit posts in a transparent and semi-interactive manner.

TickerRain is an open-source web app that stores and analysis Reddit posts in a transparent and semi-interactive manner

GonVas 180 Oct 08, 2022
Tools for teachers and students using nng (Natural Number Game)

nngtools Usage Place your nngsave.json to the directory in which you want to extract the level files. Place nngmap.json on the same directory. Run nng

Thanos Tsouanas 1 Dec 12, 2021
This is some simple code to scrape vistbook's system to get an overview of the different cabins availability.

DNT_cabin_availability_system This is some simple code to scrape visbook's system to get an overview of the different cabins availability. The system

Andreas Lorentzen 1 Sep 25, 2022
RFDesign - Protein hallucination and inpainting with RoseTTAFold

RFDesign: Protein hallucination and inpainting with RoseTTAFold Jue Wang (juewan

139 Jan 06, 2023
Our product DrLeaf which not only makes the work easier but also reduces the effort and expenditure of the farmer to identify the disease and its treatment methods.

Our product DrLeaf which not only makes the work easier but also reduces the effort and expenditure of the farmer to identify the disease and its treatment methods. We have to upload the image of an

Aniruddha Jana 2 Feb 02, 2022
Python Programming Bootcamp

python-bootcamp Python Programming Bootcamp Begin: 27th August 2021 End: 8th September 2021 Registration deadline: 22nd August 2021 Fees: No course or

Rohitash Chandra 11 Oct 19, 2022
Python pyside2 kütüphanesi ile oluşturduğum drone için yer kontrol istasyonu yazılımı.

Ground Control Station (Yer Kontrol İstasyonu) Teknofest yarışmasında yerlilik kısmında Yer Kontrol İstasyonu yazılımı seçeneği bulunuyordu. Bu yüzden

Emirhan Bülbül 4 May 14, 2022
Python - Aprendendo Python na ByLearn

PYTHON Identação Escopo Pai Escopo filho Escopo neto Variaveis

Italo Rafael 3 May 31, 2022
Swim between bookmarks in the Windows terminal

Marlin Swim between bookmarks in the terminal! Marlin is an easy to use bookmark manager for the terminal. Choose a folder, bookmark it and swim there

wilfredinni 7 Nov 03, 2022
A command line interface tool converting starknet warp transpiled outputs into readable cairo contracts.

warp-to-cairo warp-to-cairo is a simple tool converting starknet warp outputs (NethermindEth/warp) outputs into readable cairo contracts. The warp out

Michael K 5 Jun 10, 2022