Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)

Last update: Dec 27, 2022

Related tags

Overview

T-Zero

This repository serves primarily as codebase and instructions for training, evaluation and inference of T0.

T0 is the model developed in Multitask Prompted Training Enables Zero-Shot Task Generalization. In this paper, we demonstrate that massive multitask prompted fine-tuning is extremely effective to obtain task zero-shot generalization. T0 outperforms or matches GPT-3 while being 16x smaller.

While the codebase in this repository mainly reproduces and replicates the training and evaluation of T0, it will be useful for future research on massively multitask fine-tuning.

Training: reproducing (or replicating) the massively multitask fine-tuning
Evaluation: reproducing the main results reported in the paper
Inference: running inference with T0

Citation

If you find this resource useful, please cite the paper introducing T0:

@misc{sanh2021multitask,
      title={Multitask Prompted Training Enables Zero-Shot Task Generalization},
      author={Victor Sanh and Albert Webson and Colin Raffel and Stephen H. Bach and Lintang Sutawika and Zaid Alyafeai and Antoine Chaffin and Arnaud Stiegler and Teven Le Scao and Arun Raja and Manan Dey and M Saiful Bari and Canwen Xu and Urmish Thakker and Shanya Sharma Sharma and Eliza Szczechla and Taewoon Kim and Gunjan Chhablani and Nihal Nayak and Debajyoti Datta and Jonathan Chang and Mike Tian-Jian Jiang and Han Wang and Matteo Manica and Sheng Shen and Zheng Xin Yong and Harshit Pandey and Rachel Bawden and Thomas Wang and Trishala Neeraj and Jos Rozen and Abheesht Sharma and Andrea Santilli and Thibault Fevry and Jason Alan Fries and Ryan Teehan and Stella Biderman and Leo Gao and Tali Bers and Thomas Wolf and Alexander M. Rush},
      year={2021},
      eprint={2110.08207},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)

Related tags

Overview

T-Zero

Contents

Citation

Owner

BigScience Workshop

Source code for the ACL-IJCNLP 2021 paper entitled "T-DNA: Taming Pre-trained Language Models with N-gram Representations for Low-Resource Domain Adaptation" by Shizhe Diao et al.

Transfer Reinforcement Learning for Differing Action Spaces via Q-Network Representations

A python code to convert Keras pre-trained weights to Pytorch version

The King is Naked: on the Notion of Robustness for Natural Language Processing

Astrostatistics class for the MSc degree in Astrophysics at the University of Milan-Bicocca (Italy)

Omnidirectional camera calibration in python

Pytorch version of SfmLearner from Tinghui Zhou et al.

Source code for our paper "Empathetic Response Generation with State Management"

This repository contains the official implementation code of the paper Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis, accepted at EMNLP 2021.

CenterNet:Objects as Points目标检测模型在Pytorch当中的实现

本项目是一个带有前端界面的垃圾分类项目，加载了训练好的模型参数，模型为efficientnetb4，暂时为40分类问题。

GAN-based 3D human pose estimation model for 3DV'17 paper

Fine-grained Post-training for Improving Retrieval-based Dialogue Systems - NAACL 2021

🔥 Cannlytics-powered artificial intelligence 🤖

给yolov5加个gui界面，使用pyqt5，yolov5是5.0版本

Multi-Task Deep Neural Networks for Natural Language Understanding

Mini-hmc-jax - A simple implementation of Hamiltonian Monte Carlo in JAX

Multi-Anchor Active Domain Adaptation for Semantic Segmentation (ICCV 2021 Oral)

Deep learning PyTorch library for time series forecasting, classification, and anomaly detection

Sketch-Based 3D Exploration with Stacked Generative Adversarial Networks