RVT: Robust Vision Transformers

This repository contains PyTorch code for Robust Vision Transformers.

For details see Rethinking the Design Principles of Robust Vision Transformer by Xiaofeng Mao, Gege Qi, Yuefeng Chen, Yuan He and Hui Xue.

Usage

First, clone the repository locally:

git clone https://github.com/vtddggg/Robust-Vision-Transformer.git

Then, install PyTorch 1.7.0+ and torchvision 0.8.1+ and pytorch-image-models 0.3.2:

conda install -c pytorch pytorch torchvision
pip install timm==0.3.2

We use 4 nodes with 8 gpus to train RVT-Ti, RVT-S and RVT-B:

Training RVT-Ti

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_tiny --data-path /path/to/imagenet --output_dir output --dist-eval

Training RVT-S

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_small --data-path /path/to/imagenet --output_dir output --dist-eval

Training RVT-B

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_base --data-path /path/to/imagenet --output_dir output --batch-size 32 --dist-eval

If you want to train RVT-Ti*, RVT-S* or RVT-B*, simply add --use_mask and --use_patch_aug to enable positon-aware attention scaling and patch-wise augmentation.

This repository contains PyTorch code for Robust Vision Transformers.

Related tags

Overview

RVT: Robust Vision Transformers

Usage

Training RVT-Ti

Training RVT-S

Training RVT-B

Owner

MVSDF - Learning Signed Distance Field for Multi-view Surface Reconstruction

A TensorFlow implementation of SOFA, the Simulator for OFfline LeArning and evaluation.

2021 Artificial Intelligence Diabetes Datathon

PyTorch implementation of paper A Fast Knowledge Distillation Framework for Visual Recognition.

Look Who’s Talking: Active Speaker Detection in the Wild

This is a collection of our NAS and Vision Transformer work.

Generative Adversarial Text-to-Image Synthesis

Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Scheme (NeurIPS2021)

This is the official code of our paper "Diversity-based Trajectory and Goal Selection with Hindsight Experience Relay" (PRICAI 2021)

Mail classification with tensorflow and MS Exchange Server (ham or spam).

torchsummaryDynamic: support real FLOPs calculation of dynamic network or user-custom PyTorch ops

Official Implementation and Dataset of "PPR10K: A Large-Scale Portrait Photo Retouching Dataset with Human-Region Mask and Group-Level Consistency", CVPR 2021

Plugin for Gaffer providing direct acess to asset from PolyHaven.com. Only HDRIs at the moment, Cycles and Arnold supported

[NeurIPS 2021] “Improving Contrastive Learning on Imbalanced Data via Open-World Sampling”,

A curated list of automated deep learning (including neural architecture search and hyper-parameter optimization) resources.

Pywonderland - A tour in the wonderland of math with python.

A nutritional label for food for thought.

Realtime YOLO Monster Detection With Non Maximum Supression

Hand tracking demo for DIY Smart Glasses with a remote computer doing the work

Weight estimation in CT by multi atlas techniques