Deformable 3D Convolution for Video Super-Resolution

Pytorch implementation of local motion and contrast prior driven deep network (MoCoPnet). [PDF]

Overview

Requirements

• Python 3
• pytorch >= 1.6
• numpy, PIL

Datasets

Training & test datasets

Download SAITD dataset.

SAITD dataset is a large-scale high-quality semi-synthetic dataset of infrared small target. We employ the 1st-50th sequences with target annotations as the test datasets and the remaining 300 sequences as the training datasets.

Download Hui and Anti-UAV.

Hui and Anti-UAV datasets are used as the test datasets to test the robustness of our MoCoPnet to real scenes. In Anti-UAV dataset, only the sequences with infrared small target (i.e., The target size is less than 0.12% of the image size) are selected as the test set (21 sequences in total). Note that, we only use the first 100 images of each sequence for test to balance computational/time cost and generalization performance.

For simplicity, you can also Download the test datasets in https://pan.baidu.com/s/1oobhklwIChvNJIBpTcdQRQ?pwd=1113 and put the folder in code/data.

Data format:

1. The training dataset is in code/data/train/SAITD.

train
  └── SAITD
       └── 1
              ├── 0.png
              ├── 1.png
              ├── ...
       └── 2
              ├── 00001
              ├── 00002
              ├── ...		
       ...

2. The test datasets are in code/data/test as below:

 test
  └── dataset_1
         └── scene_1
              ├── 0.png  
              ├── 1.png  
              ├── ...
              └── 100.png    
               
         ├── ...		  
         └── scene_M
  ├── ...    
  └── dataset_N      

Results

Quantitative Results of SR performance

Table 1. PSNR/SSIM achieved by different methods.

Table 2. SNR and CR results of different methods achieved on super-resolved LR images and super-resolved HR images.

Qualitative Results of SR performance

Figure 1. Visual results of different SR methods on LR images for 4x SR.

Figure 2. Visual results of different SR methods on LR images for 4x SR.

Quantitative Results of detection

Table 3. Quantitative results of Tophat, ILCM, IPI achieved on super-resolved LR images.

Table 4. Quantitative results of Tophat, ILCM, IPI achieved on super-resolved HR images.

Figure 3. ROC results of Tophat, ILCM and IPI achieved on super-resolved LR images.

Figure 4. ROC results of Tophat, ILCM and IPI achieved on super-resolved HR images.

Qualitative Results of detection

Figure 5. Qualitative results of super-resolved LR image and detection results.

Figure 6. Qualitative results of super-resolved HR image and detection results.

Citiation

@article{MoCoPnet,
  author = {Ying, Xinyi and Wang, Yingqian and Wang, Longguang and Sheng, Weidong and Liu, Li and Lin, Zaipin and Zhou, Shilin},
  title = {MoCoPnet: Exploring Local Motion and Contrast Priors for Infrared Small Target Super-Resolution},
  journal={arXiv preprint arXiv:2201.01014},
  year = {2020},
}

Contact

Please contact us at [email protected] for any question.