RVT: Robust Vision Transformers

This repository contains PyTorch code for Robust Vision Transformers.

For details see Rethinking the Design Principles of Robust Vision Transformer by Xiaofeng Mao, Gege Qi, Yuefeng Chen, Yuan He and Hui Xue.

Usage

First, clone the repository locally:

git clone https://github.com/vtddggg/Robust-Vision-Transformer.git

Then, install PyTorch 1.7.0+ and torchvision 0.8.1+ and pytorch-image-models 0.3.2:

conda install -c pytorch pytorch torchvision
pip install timm==0.3.2

We use 4 nodes with 8 gpus to train RVT-Ti, RVT-S and RVT-B:

Training RVT-Ti

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_tiny --data-path /path/to/imagenet --output_dir output --dist-eval

Training RVT-S

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_small --data-path /path/to/imagenet --output_dir output --dist-eval

Training RVT-B

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_base --data-path /path/to/imagenet --output_dir output --batch-size 32 --dist-eval

If you want to train RVT-Ti*, RVT-S* or RVT-B*, simply add --use_mask and --use_patch_aug to enable positon-aware attention scaling and patch-wise augmentation.

This repository contains PyTorch code for Robust Vision Transformers.

Related tags

Overview

RVT: Robust Vision Transformers

Usage

Training RVT-Ti

Training RVT-S

Training RVT-B

Owner

Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning using 🤗 transformers

Pytorch implementation of the paper Progressive Growing of Points with Tree-structured Generators (BMVC 2021)

Re-implementation of the Noise Contrastive Estimation algorithm for pyTorch, following "Noise-contrastive estimation: A new estimation principle for unnormalized statistical models." (Gutmann and Hyvarinen, AISTATS 2010)

ConE: Cone Embeddings for Multi-Hop Reasoning over Knowledge Graphs

Self-training with Weak Supervision (NAACL 2021)

PyTorch Implementation of Vector Quantized Variational AutoEncoders.

Similarity-based Gray-box Adversarial Attack Against Deep Face Recognition

Distance correlation and related E-statistics in Python

A Robust Unsupervised Ensemble of Feature-Based Explanations using Restricted Boltzmann Machines

Code for "AutoMTL: A Programming Framework for Automated Multi-Task Learning"

Implementation of StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation in PyTorch

Trajectory Prediction with Graph-based Dual-scale Context Fusion

Low-dose Digital Mammography with Deep Learning

Arbitrary Distribution Modeling with Censorship in Real Time 59 2 60 3 Bidding Advertising for KDD'21

Implementation of Monocular Direct Sparse Localization in a Prior 3D Surfel Map (DSL)

Codes for "Solving Long-tailed Recognition with Deep Realistic Taxonomic Classifier"

Laplace Redux -- Effortless Bayesian Deep Learning

A flexible framework of neural networks for deep learning

A neuroanatomy-based augmented reality experience powered by computer vision. Features 3D visuals of the Atlas Brain Map slices.

Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformers (arXiv2021)