Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)

Last update: Dec 27, 2022

Related tags

Overview

T-Zero

This repository serves primarily as codebase and instructions for training, evaluation and inference of T0.

T0 is the model developed in Multitask Prompted Training Enables Zero-Shot Task Generalization. In this paper, we demonstrate that massive multitask prompted fine-tuning is extremely effective to obtain task zero-shot generalization. T0 outperforms or matches GPT-3 while being 16x smaller.

While the codebase in this repository mainly reproduces and replicates the training and evaluation of T0, it will be useful for future research on massively multitask fine-tuning.

Training: reproducing (or replicating) the massively multitask fine-tuning
Evaluation: reproducing the main results reported in the paper
Inference: running inference with T0

Citation

If you find this resource useful, please cite the paper introducing T0:

@misc{sanh2021multitask,
      title={Multitask Prompted Training Enables Zero-Shot Task Generalization},
      author={Victor Sanh and Albert Webson and Colin Raffel and Stephen H. Bach and Lintang Sutawika and Zaid Alyafeai and Antoine Chaffin and Arnaud Stiegler and Teven Le Scao and Arun Raja and Manan Dey and M Saiful Bari and Canwen Xu and Urmish Thakker and Shanya Sharma Sharma and Eliza Szczechla and Taewoon Kim and Gunjan Chhablani and Nihal Nayak and Debajyoti Datta and Jonathan Chang and Mike Tian-Jian Jiang and Han Wang and Matteo Manica and Sheng Shen and Zheng Xin Yong and Harshit Pandey and Rachel Bawden and Thomas Wang and Trishala Neeraj and Jos Rozen and Abheesht Sharma and Andrea Santilli and Thibault Fevry and Jason Alan Fries and Ryan Teehan and Stella Biderman and Leo Gao and Tali Bers and Thomas Wolf and Alexander M. Rush},
      year={2021},
      eprint={2110.08207},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)

Related tags

Overview

T-Zero

Contents

Citation

Owner

BigScience Workshop

A modular active learning framework for Python

High performance, easy-to-use, and scalable machine learning (ML) package, including linear model (LR), factorization machines (FM), and field-aware factorization machines (FFM) for Python and CLI interface.

Deep Learning Interviews book: Hundreds of fully solved job interview questions from a wide range of key topics in AI.

We utilize deep reinforcement learning to obtain favorable trajectories for visual-inertial system calibration.

U-Time: A Fully Convolutional Network for Time Series Segmentation

A simple and useful implementation of LPIPS.

Open source hardware and software platform to build a small scale self driving car.

Deep Learning Datasets Maker is a QGIS plugin to make datasets creation easier for raster and vector data.

This is the codebase for the ICLR 2021 paper Trajectory Prediction using Equivariant Continuous Convolution

New approach to benchmark VQA models

Camera calibration & 3D pose estimation tools for AcinoSet

A Simple Example for Imitation Learning with Dataset Aggregation (DAGGER) on Torcs Env

Official PyTorch implementation of "Synthesis of Screentone Patterns of Manga Characters"

HGCN: Harmonic Gated Compensation Network For Speech Enhancement

Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding

Python Multi-Agent Reinforcement Learning framework

GLANet - The code for Global and Local Alignment Networks for Unpaired Image-to-Image Translation arxiv

Frigate - NVR With Realtime Object Detection for IP Cameras

Image reconstruction done with untrained neural networks.

Microsoft Cognitive Toolkit (CNTK), an open source deep-learning toolkit