RITA is a family of autoregressive protein models, developed by LightOn in collaboration with the OATML group at Oxford and the Debora Marks Lab at Harvard.

Last update: Dec 22, 2022

Overview

RITA: a Study on Scaling Up Generative Protein Sequence Models

RITA is a family of autoregressive protein models, developed by a collaboration of Lighton, the OATML group at Oxford, and the Debbie Marks Lab at Harvard.

Model	#Params	d_model	layers	lm loss uniref-100
Small	85M	768	12	2.31
Medium	300M	1024	24	2.01
Large	680M	1536	24	1.82
XLarge	1.2B	2048	24	1.70

Results

For full results see our preprint: https://arxiv.org/abs/2205.05789

Usage

Instantiate a model like so:

from transformers import AutoModel, AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("lightonai/RITA_s, trust_remote_code=True")
tokenizer = AutoTokenizer.from_pretrained("lightonai/RITA_s")

for generation we support pipelines:

from transformers import pipeline
rita_gen = pipeline('text-generation', model=model, tokenizer=tokenizer)
sequences = rita_gen("MAB", max_length=20, do_sample=True, top_k=950, repetition_penalty=1.2, 
                     num_return_sequences=2, eos_token_id=2)
for seq in sequences:
    print(f"seq: {seq['generated_text'].replace(' ', '')}")

Or see example.py

How to cite

@article{hesslow2022rita,
  title={RITA: a Study on Scaling Up Generative Protein Sequence Models},
  author={Hesslow, Daniel and Zanichelli, Niccol{\'o} and Notin, Pascal and Poli, Iacopo and Marks, Debora},
  journal={arXiv preprint arXiv:2205.05789},
  year={2022}
}

RITA is a family of autoregressive protein models, developed by LightOn in collaboration with the OATML group at Oxford and the Debora Marks Lab at Harvard.

Related tags

Overview

RITA: a Study on Scaling Up Generative Protein Sequence Models

Results

Usage

How to cite

Owner

LightOn

A general and strong 3D object detection codebase that supports more methods, datasets and tools (debugging, recording and analysis).

A Simple Framwork for CV Pre-training Model (SOCO, VirTex, BEiT)

code for the ICLR'22 paper: On Robust Prefix-Tuning for Text Classification

Multitask Learning Strengthens Adversarial Robustness

🐤 Nix-TTS: An Incredibly Lightweight End-to-End Text-to-Speech Model via Non End-to-End Distillation

PyTorch implementation of MuseMorphose, a Transformer-based model for music style transfer.

Code for "Adversarial Attack Generation Empowered by Min-Max Optimization", NeurIPS 2021

Source code for 2021 ICCV paper "In-the-Wild Single Camera 3D Reconstruction Through Moving Water Surfaces"

BiSeNet based on pytorch

Python package for covariance matrices manipulation and Biosignal classification with application in Brain Computer interface

Code for AutoNL on ImageNet (CVPR2020)

NeuralDiff: Segmenting 3D objects that move in egocentric videos

Official implementation for "Low-light Image Enhancement via Breaking Down the Darkness"

Pgn2tex - Scripts to convert pgn files to latex document. Useful to build books or pdf from pgn studies

2021-AIAC-QQ-Browser-Hyperparameter-Optimization-Rank6

Compute execution plan: A DAG representation of work that you want to get done. Individual nodes of the DAG could be simple python or shell tasks or complex deeply nested parallel branches or embedded DAGs themselves.

Focal Loss for Dense Rotation Object Detection

A Python library that provides a simplified alternative to DBAPI 2

Addition of pseudotorsion caclulation eta, theta, eta', and theta' to barnaba package

Code for ICML 2021 paper: How could Neural Networks understand Programs?