RVT: Robust Vision Transformers

This repository contains PyTorch code for Robust Vision Transformers.

For details see Rethinking the Design Principles of Robust Vision Transformer by Xiaofeng Mao, Gege Qi, Yuefeng Chen, Yuan He and Hui Xue.

Usage

First, clone the repository locally:

git clone https://github.com/vtddggg/Robust-Vision-Transformer.git

Then, install PyTorch 1.7.0+ and torchvision 0.8.1+ and pytorch-image-models 0.3.2:

conda install -c pytorch pytorch torchvision
pip install timm==0.3.2

We use 4 nodes with 8 gpus to train RVT-Ti, RVT-S and RVT-B:

Training RVT-Ti

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_tiny --data-path /path/to/imagenet --output_dir output --dist-eval

Training RVT-S

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_small --data-path /path/to/imagenet --output_dir output --dist-eval

Training RVT-B

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_base --data-path /path/to/imagenet --output_dir output --batch-size 32 --dist-eval

If you want to train RVT-Ti*, RVT-S* or RVT-B*, simply add --use_mask and --use_patch_aug to enable positon-aware attention scaling and patch-wise augmentation.

This repository contains PyTorch code for Robust Vision Transformers.

Related tags

Overview

RVT: Robust Vision Transformers

Usage

Training RVT-Ti

Training RVT-S

Training RVT-B

Owner

AI-Fitness-Tracker - AI Fitness Tracker With Python

SSPNet: Scale Selection Pyramid Network for Tiny Person Detection from UAV Images.

Relative Uncertainty Learning for Facial Expression Recognition

🛠️ SLAMcore SLAM Utilities

Implementation of Pooling by Sliced-Wasserstein Embedding (NeurIPS 2021)

🔥 Cogitare - A Modern, Fast, and Modular Deep Learning and Machine Learning framework for Python

PyArmadillo: an alternative approach to linear algebra in Python

MVFNet: Multi-View Fusion Network for Efficient Video Recognition (AAAI 2021)

UltraPose: Synthesizing Dense Pose with 1 Billion Points by Human-body Decoupling 3D Model

PyTorch implementation for our paper Learning Character-Agnostic Motion for Motion Retargeting in 2D, SIGGRAPH 2019

TPH-YOLOv5: Improved YOLOv5 Based on Transformer Prediction Head for Object Detection on Drone-Captured Scenarios

Predicting future trajectories of people in cameras of novel scenarios and views.

Implementation for Paper "Inverting Generative Adversarial Renderer for Face Reconstruction"

[ECCVW2020] Robust Long-Term Object Tracking via Improved Discriminative Model Prediction (RLT-DiMP)

Submanifold sparse convolutional networks

Gif-caption - A straightforward GIF Captioner written in Python

Research code for Arxiv paper "Camera Motion Agnostic 3D Human Pose Estimation"

[WWW 2021] Source code for "Graph Contrastive Learning with Adaptive Augmentation"

SOTR: Segmenting Objects with Transformers [ICCV 2021]

[SIGGRAPH 2022 Journal Track] AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars