RVT: Robust Vision Transformers

This repository contains PyTorch code for Robust Vision Transformers.

For details see Rethinking the Design Principles of Robust Vision Transformer by Xiaofeng Mao, Gege Qi, Yuefeng Chen, Yuan He and Hui Xue.

Usage

First, clone the repository locally:

git clone https://github.com/vtddggg/Robust-Vision-Transformer.git

Then, install PyTorch 1.7.0+ and torchvision 0.8.1+ and pytorch-image-models 0.3.2:

conda install -c pytorch pytorch torchvision
pip install timm==0.3.2

We use 4 nodes with 8 gpus to train RVT-Ti, RVT-S and RVT-B:

Training RVT-Ti

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_tiny --data-path /path/to/imagenet --output_dir output --dist-eval

Training RVT-S

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_small --data-path /path/to/imagenet --output_dir output --dist-eval

Training RVT-B

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_base --data-path /path/to/imagenet --output_dir output --batch-size 32 --dist-eval

If you want to train RVT-Ti*, RVT-S* or RVT-B*, simply add --use_mask and --use_patch_aug to enable positon-aware attention scaling and patch-wise augmentation.

This repository contains PyTorch code for Robust Vision Transformers.

Related tags

Overview

RVT: Robust Vision Transformers

Usage

Training RVT-Ti

Training RVT-S

Training RVT-B

Owner

Codebase to experiment with a hybrid Transformer that combines conditional sequence generation with regression

Linear image-to-image translation

Deep learning library featuring a higher-level API for TensorFlow.

This project generates news headlines using a Long Short-Term Memory (LSTM) neural network.

This repo is about to create the Streamlit application for given ML model.

Mercury: easily convert Python notebook to web app and share with others

Dieser Scanner findet Websites, die nicht direkt in Suchmaschinen auftauchen, aber trotzdem erreichbar sind.

Code release for NeRF (Neural Radiance Fields)

Pre-trained Deep Learning models and demos (high quality and extremely fast)

The code repository for "PyCIL: A Python Toolbox for Class-Incremental Learning" in PyTorch.

UCSD Oasis platform

Exact Pareto Optimal solutions for preference based Multi-Objective Optimization

Implementation of "Efficient Regional Memory Network for Video Object Segmentation" (Xie et al., CVPR 2021).

An adaptive hierarchical energy management strategy for hybrid electric vehicles

Pytorch implementation of the paper Time-series Generative Adversarial Networks

[NeurIPS 2021] Introspective Distillation for Robust Question Answering

MultiSiam: Self-supervised Multi-instance Siamese Representation Learning for Autonomous Driving

Based on the given clinical dataset, Predict whether the patient having Heart Disease or Not having Heart Disease

Library of various Few-Shot Learning frameworks for text classification

This project aims at providing a concise, easy-to-use, modifiable reference implementation for semantic segmentation models using PyTorch.