Utils to quickly evaluate many 🤗 models on the GLUE tasks

Overview

Utils to quickly evaluate many 🤗 models on the GLUE tasks

Setup

wget https://raw.githubusercontent.com/huggingface/transformers/master/examples/pytorch/text-classification/run_glue.py
conda env create -f environment.yaml
conda activate glue-utils

Evaluate multiple models on every glue task:

python runner.py \
    path/to/model_1/ \
    path/to/model_2/ \
    path/to/model_3/ \
    --gpu_idx 1

This will store results in path/to/model_N/glue/[task_name]/, for each model, for each task.

To quickly gather all results, run:

python gather_scores.py \
    path/to/model_1/glue/ \
    path/to/model_2/glue/ \
    path/to/model_3/glue/
Owner
Przemyslaw K. Joniak
Grad student at The University of Tokyo
Przemyslaw K. Joniak
Python library for datamining glitch information from Gen 1 Pokémon GameBoy ROMs

g1utils This is a Python library for datamining information about various glitches (glitch Pokémon, glitch maps, etc.) from Gen 1 Pokémon ROMs. TODO A

1 Jan 13, 2022
python scripts - mostly automation scripts

python python scripts - mostly automation scripts You can set your environment in various ways bash #!/bin/bash python - locally on remote host #!/bi

Enyi 1 Jan 05, 2022
Easily map device and application controls to a midi controller

pymidicontroller Introduction Easily map device and application controls to a midi controller

Tane Barriball 24 May 16, 2022
Pre-crisis Risk Management for Personal Finance

Антикризисный риск-менеджмент личных финансов Риск-менеджмент личных финансов условиях санкций и/или финансового кризиса: делаем сегодня все, чтобы за

Dmitry Petukhov 593 Jan 09, 2023
A tool for removing PUPs using signatures

Unwanted program removal tool A tool for removing PUPs using signatures What is the unwanted program removal tool? The unwanted program removal tool i

4 Sep 20, 2022
Blender pluggin (python script) that adds a randomly generated tree with random branches and bend orientations

Blender pluggin (python script) that adds a randomly generated tree with random branches and bend orientations

Travis Gruber 2 Dec 24, 2021
Advanced Keylogger in Python

Advanced Keylogger in Python Important Disclaimer: The author will not be held r

Suvanth Erranki 1 Feb 07, 2022
Learning a Little about Containerlab

Learning a Little about Containerlab Hello all. This is the respository based on this blog post. Getting Started Feel free to use this example. You wi

10 Oct 16, 2022
A module comment generator for python

Module Comment Generator The comment style is as a tribute to the comment from the RA . The comment generator can parse the ast tree from the python s

飘尘 1 Oct 21, 2021
Python library for ODE integration via Taylor's method and LLVM

heyoka.py Modern Taylor's method via just-in-time compilation Explore the docs » Report bug · Request feature · Discuss The heyókȟa [...] is a kind of

Francesco Biscani 45 Dec 21, 2022
Demo of using DataLoader to prevent out of memory

Demo of using DataLoader to prevent out of memory

3 Jun 25, 2022
A comprensive software collection for nmea manipulation

nmeatoolkit A comprensive software collection for nmea manipulation; it includes a library and a collections of command line tools. Library pipes: con

Davide Gessa 1 Sep 14, 2022
Online learning platform

🛠 Status: In Development Teached is currently in development. So we encourage you to use it and give us your feedback, but there are things that have

Mohamed Nesredin 2 Feb 07, 2021
Registro Online (100% Python-Mysql)

Registro elettronico scritto in python, utilizzando database Mysql e Collegando Registro elettronico scritto in PHP

Sergiy Grimoldi 1 Dec 20, 2021
Automatização completa do site https://blaze.com

PyBlaze Pyblaze possibilita o acesso a api do site blaze utilizando python, retornando os últimos resultados de crashs e doubles. Agora também é possí

Cleiton Leonel 24 Dec 30, 2022
ERPNext Easy Letterhead

ERPNext Easy Letterhead Intro Quality letterheads are a problem for non-technical users. So we've built (really hacked together) a slightly easier sol

Bantoo 3 Jan 02, 2023
A python script that changes your desktop background based on current weather and time of the day.

Desktop background wallpaper, based on current weather and time A python script that changes your computer's desktop background based on current weath

Maj Gaberšček 1 Nov 16, 2021
Script for resizing MTD partitions on a QNAP device in order to be available to upgrade from buster to bullseye

QNAP partitions resize for kirkwood devices. As explained by Marin Michlmayr, Debian bullseye support on kirkwood QNAP devices was dropped due to [mai

Arnaud Mouiche 26 Jan 05, 2023
An end-to-end Python-based Infrastructure as Code framework for network automation and orchestration.

Nectl An end-to-end Python-based Infrastructure as Code framework for network automation and orchestration. Features Data modelling and validation. Da

Adam Kirchberger 15 Oct 14, 2022
A simple IDA Pro plugin to show all HexRays decompiler comments written by user

XRaysComments A simple IDA Pro plugin to show all HexRays decompiler comments written by user Installation Copy the file xray_comments.py to the plugi

Nox 20 Dec 27, 2022