RVT: Robust Vision Transformers

This repository contains PyTorch code for Robust Vision Transformers.

For details see Rethinking the Design Principles of Robust Vision Transformer by Xiaofeng Mao, Gege Qi, Yuefeng Chen, Yuan He and Hui Xue.

Usage

First, clone the repository locally:

git clone https://github.com/vtddggg/Robust-Vision-Transformer.git

Then, install PyTorch 1.7.0+ and torchvision 0.8.1+ and pytorch-image-models 0.3.2:

conda install -c pytorch pytorch torchvision
pip install timm==0.3.2

We use 4 nodes with 8 gpus to train RVT-Ti, RVT-S and RVT-B:

Training RVT-Ti

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_tiny --data-path /path/to/imagenet --output_dir output --dist-eval

Training RVT-S

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_small --data-path /path/to/imagenet --output_dir output --dist-eval

Training RVT-B

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_base --data-path /path/to/imagenet --output_dir output --batch-size 32 --dist-eval

If you want to train RVT-Ti*, RVT-S* or RVT-B*, simply add --use_mask and --use_patch_aug to enable positon-aware attention scaling and patch-wise augmentation.

This repository contains PyTorch code for Robust Vision Transformers.

Related tags

Overview

RVT: Robust Vision Transformers

Usage

Training RVT-Ti

Training RVT-S

Training RVT-B

Owner

Supervised Contrastive Learning for Downstream Optimized Sequence Representations

A collection of resources and papers on Diffusion Models, a darkhorse in the field of Generative Models

ALL Snow Removed: Single Image Desnowing Algorithm Using Hierarchical Dual-tree Complex Wavelet Representation and Contradict Channel Loss (HDCWNet)

HairCLIP: Design Your Hair by Text and Reference Image

Defending graph neural networks against adversarial attacks (NeurIPS 2020)

CVPR 2021: "The Spatially-Correlative Loss for Various Image Translation Tasks"

A general framework for inferring CNNs efficiently. Reduce the inference latency of MobileNet-V3 by 1.3x on an iPhone XS Max without sacrificing accuracy.

Grammar Induction using a Template Tree Approach

Code for the paper "Can Active Learning Preemptively Mitigate Fairness Issues?" presented at RAI 2021.

This is a simple framework to make object detection dataset very quickly

Simple tutorials on Pytorch DDP training

Code to reproduce the results for Compositional Attention

An atmospheric growth and evolution model based on the EVo degassing model and FastChem 2.0

Dynamic Bottleneck for Robust Self-Supervised Exploration

PassAPI is a password generator in hash format and fully developed in Python, with the aim of teaching how to handle and build

To model the probability of a soccer coach leave his/her team during Campeonato Brasileiro for 10 chosen teams and considering years 2018, 2019 and 2020.

Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples

Network Compression via Central Filter

Scheduling BilinearRewards

This repository contains the official MATLAB implementation of the TDA method for reverse image filtering