CrossViT
This repository is the official implementation of CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification. ArXiv
If you use the codes and models from this repo, please cite our work. Thanks!
@inproceedings{
chen2021crossvit,
title={{CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification}},
author={Chun-Fu (Richard) Chen and Quanfu Fan and Rameswar Panda},
booktitle={International Conference on Computer Vision (ICCV)},
year={2021}
}
Installation
To install requirements:
pip install -r requirements.txt
With conda:
conda create -n crossvit python=3.8
conda activate crossvit
conda install pytorch=1.7.1 torchvision cudatoolkit=11.0 -c pytorch -c nvidia
pip install -r requirements.txt
Data preparation
Download and extract ImageNet train and val images from http://image-net.org/. The directory structure is the standard layout for the torchvision datasets.ImageFolder, and the training and validation data is expected to be in the train/ folder and val folder respectively:
/path/to/imagenet/
train/
class1/
img1.jpeg
class2/
img2.jpeg
val/
class1/
img3.jpeg
class/2
img4.jpeg
Pretrained models
We provide models trained on ImageNet1K. You can find models here. And you can load pretrained weights into models by add --pretrained flag.
Training
To train crossvit_9_dagger_224 on ImageNet on a single node with 8 gpus for 300 epochs run:
python -m torch.distributed.launch --nproc_per_node=8 --use_env main.py --model crossvit_9_dagger_224 --batch-size 256 --data-path /path/to/imagenet
Other model names can be found at models/crossvit.py.
Multinode training
Distributed training is available via Slurm and submitit:
To train a crossvit_9_dagger_224 model on ImageNet on 4 nodes with 8 gpus each for 300 epochs:
python run_with_submitit.py --nodes 4 --model crossvit_9_dagger_224 --data-path /path/to/imagenet --batch-size 128 --warmup-epochs 30
Or you can start process on each machine maunally. E.g. 2 nodes, each with 8 gpus.
Machine 0:
python -m torch.distributed.launch --nproc_per_node=8 --master_addr=MACHINE_0_IP --master_port=AVAILABLE_PORT --nnodes=2 --node_rank=0 main.py --model crossvit_9_dagger_224 --batch-size 256 --data-path /path/to/imagenet
Machine 1:
python -m torch.distributed.launch --nproc_per_node=8 --master_addr=MACHINE_0_IP --master_port=AVAILABLE_PORT --nnodes=2 --node_rank=1 main.py --model crossvit_9_dagger_224 --batch-size 256 --data-path /path/to/imagenet
Note that: some slurm configurations might need to be changed based on your cluster.
Evaluation
To evaluate a pretrained model on crossvit_9_dagger_224:
python -m torch.distributed.launch --nproc_per_node=8 --use_env main.py --model crossvit_9_dagger_224 --batch-size 128 --data-path /path/to/imagenet --eval --pretrained
1 Dec 24, 2021
129 Dec 11, 2022
1.4k Dec 30, 2022
41 Jan 2, 2023
52 Nov 22, 2022
274 Dec 28, 2022
109 Dec 28, 2022
6 Sep 21, 2022
8 Oct 04, 2022
820 Dec 26, 2022
13 Dec 19, 2022
192 Jan 05, 2023
57 Nov 14, 2022
34 May 24, 2022
5.1k Jan 02, 2023
1.5k Dec 27, 2022
95 Jan 08, 2023
529 Dec 10, 2022
9 May 20, 2022
27 Dec 23, 2022
36 Dec 12, 2022
8 Nov 18, 2022
114 Dec 15, 2022
2.6k Jan 08, 2023
113 Dec 14, 2022
112 Jan 05, 2023
112 Dec 28, 2022