We propose to address the problem of few-shot classification by meta-learning “what to observe” and “where to attend” in a relational perspective. Our method leverages relational patterns within and between images via self-correlational representation (SCR) and cross-correlational attention (CCA). Within each image, the SCR module transforms a base feature map into a self-correlation tensor and learns to extract structural patterns from the tensor. Between the images, the CCA module computes cross-correlation between two image representations and learns to produce co-attention between them. (a), (b), and (c) visualize the activation maps of base features, self-correlational representation, and cross-correlational attention, respectively. Our Relational Embedding Network (RENet) combines the two relational modules to learn relational embedding in an end-to-end manner. In experimental evaluation, it achieves consistent improvements over state-of-the-art methods on four widely used few-shot classification benchmarks of miniImageNet, tieredImageNet, CUB-200-2011, and CIFAR-FS.

✔️ Requirements

• Ubuntu 16.04
• Python 3.7
• CUDA 11.0
• PyTorch 1.7.1

⚙️ Conda environmnet installation

conda env create --name renet_iccv21 --file environment.yml
conda activate renet_iccv21

📚 Datasets

cd datasets
bash download_miniimagenet.sh
bash download_cub.sh
bash download_cifar_fs.sh
bash download_tieredimagenet.sh

🌳 Authors' checkpoints

cd checkpoints
bash download_checkpoints_renet.sh

The file structure should be as follows:

renet/
├── datasets/
├── model/
├── scripts/
├── checkpoints/
│   ├── cifar_fs/
│   ├── cub/
│   ├── miniimagenet/
│   └── tieredimagenet/
train.py
test.py
README.md
environment.yml

📌 Quick start: testing scripts

To test in the 5-way K-shot setting:

bash scripts/test/{dataset_name}_5wKs.sh

For example, to test ReNet on the miniImagenet dataset in the 5-way 1-shot setting:

bash scripts/test/miniimagenet_5w1s.sh

🔥 Training scripts

To train in the 5-way K-shot setting:

bash scripts/train/{dataset_name}_5wKs.sh

For example, to train ReNet on the CUB dataset in the 5-way 1-shot setting:

bash scripts/train/cub_5w1s.sh

Training & testing a 5-way 1-shot model on the CUB dataset using a TitanRTX 3090 GPU takes 41m 30s.

🎨 Few-shot classification results

Experimental results on few-shot classification datasets with ResNet-12 backbone. We report average results with 2,000 randomly sampled episodes.

🔍 Related repos

Our project references the codes in the following repos:

• Zhang et al., DeepEMD.
• Ye et al., FEAT
• Wang et al., Non-local neural networks
• Ramachandran et al., Stand-alone self-attention
• Huang et al., DCCNet
• Yang et al., VCN

💌 Acknowledgement

We adopted the main code bases from DeepEMD, and we really appreciate it 😃 . We also sincerely thank all the ICCV reviewers, especially R#2, for valuable suggestions.

📜 Citing RENet

If you find our code or paper useful to your research work, please consider citing our work using the following bibtex:

@inproceedings{kang2021renet,
    author   = {Kang, Dahyun and Kwon, Heeseung and Min, Juhong and Cho, Minsu},
    title    = {Relational Embedding for Few-Shot Classification},
    booktitle= {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    year     = {2021}
}