MoCoPnet - Deformable 3D Convolution for Video Super-Resolution

Related tags

Deep LearningMoCoPnet
Overview

Deformable 3D Convolution for Video Super-Resolution

Pytorch implementation of local motion and contrast prior driven deep network (MoCoPnet). [PDF]

Overview


Requirements

  • Python 3
  • pytorch >= 1.6
  • numpy, PIL

Datasets

Training & test datasets

Download SAITD dataset.

SAITD dataset is a large-scale high-quality semi-synthetic dataset of infrared small target. We employ the 1st-50th sequences with target annotations as the test datasets and the remaining 300 sequences as the training datasets.

Download Hui and Anti-UAV.

Hui and Anti-UAV datasets are used as the test datasets to test the robustness of our MoCoPnet to real scenes. In Anti-UAV dataset, only the sequences with infrared small target (i.e., The target size is less than 0.12% of the image size) are selected as the test set (21 sequences in total). Note that, we only use the first 100 images of each sequence for test to balance computational/time cost and generalization performance.

For simplicity, you can also Download the test datasets in https://pan.baidu.com/s/1oobhklwIChvNJIBpTcdQRQ?pwd=1113 and put the folder in code/data.

Data format:

  1. The training dataset is in code/data/train/SAITD.
train
  └── SAITD
       └── 1
              ├── 0.png
              ├── 1.png
              ├── ...
       └── 2
              ├── 00001
              ├── 00002
              ├── ...		
       ...
  1. The test datasets are in code/data/test as below:
 test
  └── dataset_1
         └── scene_1
              ├── 0.png  
              ├── 1.png  
              ├── ...
              └── 100.png    
               
         ├── ...		  
         └── scene_M
  ├── ...    
  └── dataset_N      

Results

Quantitative Results of SR performance

Table 1. PSNR/SSIM achieved by different methods.

Table 2. SNR and CR results of different methods achieved on super-resolved LR images and super-resolved HR images.

Qualitative Results of SR performance

Figure 1. Visual results of different SR methods on LR images for 4x SR.

Figure 2. Visual results of different SR methods on LR images for 4x SR.

Quantitative Results of detection

Table 3. Quantitative results of Tophat, ILCM, IPI achieved on super-resolved LR images.

Table 4. Quantitative results of Tophat, ILCM, IPI achieved on super-resolved HR images.

Figure 3. ROC results of Tophat, ILCM and IPI achieved on super-resolved LR images.

Figure 4. ROC results of Tophat, ILCM and IPI achieved on super-resolved HR images.

Qualitative Results of detection

Figure 5. Qualitative results of super-resolved LR image and detection results.

Figure 6. Qualitative results of super-resolved HR image and detection results.

Citiation

@article{MoCoPnet,
  author = {Ying, Xinyi and Wang, Yingqian and Wang, Longguang and Sheng, Weidong and Liu, Li and Lin, Zaipin and Zhou, Shilin},
  title = {MoCoPnet: Exploring Local Motion and Contrast Priors for Infrared Small Target Super-Resolution},
  journal={arXiv preprint arXiv:2201.01014},
  year = {2020},
}

Contact

Please contact us at [email protected] for any question.

Owner
Xinyi Ying
Her current research interests focus on image & video super-resolution and small target detection.
Xinyi Ying
This is the code related to "Sparse-to-dense Feature Matching: Intra and Inter domain Cross-modal Learning in Domain Adaptation for 3D Semantic Segmentation" (ICCV 2021).

Sparse-to-dense Feature Matching: Intra and Inter domain Cross-modal Learning in Domain Adaptation for 3D Semantic Segmentation This is the code relat

39 Sep 23, 2022
AdelaiDepth is an open source toolbox for monocular depth prediction.

AdelaiDepth is an open source toolbox for monocular depth prediction.

Adelaide Intelligent Machines (AIM) Group 743 Jan 01, 2023
TransNet V2: Shot Boundary Detection Neural Network

TransNet V2: Shot Boundary Detection Neural Network This repository contains code for TransNet V2: An effective deep network architecture for fast sho

Tomáš Souček 212 Dec 27, 2022
Personal project about genus-0 meshes, spherical harmonics and a cow

How to transform a cow into spherical harmonics ? Spot the cow, from Keenan Crane's blog Context In the field of Deep Learning, training on images or

3 Aug 22, 2022
Natural Intelligence is still a pretty good idea.

Human Learn Machine Learning models should play by the rules, literally. Project Goal Back in the old days, it was common to write rule-based systems.

vincent d warmerdam 641 Dec 26, 2022
Source code for NAACL 2021 paper "TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference"

TR-BERT Source code and dataset for "TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference". The code is based on huggaface's transformers.

THUNLP 37 Oct 30, 2022
Codes and scripts for "Explainable Semantic Space by Grounding Languageto Vision with Cross-Modal Contrastive Learning"

Visually Grounded Bert Language Model This repository is the official implementation of Explainable Semantic Space by Grounding Language to Vision wit

17 Dec 17, 2022
Public repo for the ICCV2021-CVAMD paper "Is it Time to Replace CNNs with Transformers for Medical Images?"

Is it Time to Replace CNNs with Transformers for Medical Images? Accepted at ICCV-2021: Workshop on Computer Vision for Automated Medical Diagnosis (C

Christos Matsoukas 80 Dec 27, 2022
KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control

KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control Tomas Jakab, Richard Tucker, Ameesh Makadia, Jiajun Wu, Noah Snavely, Angjoo Ka

Tomas Jakab 87 Nov 30, 2022
This GitHub repository contains code used for plots in NeurIPS 2021 paper 'Stochastic Multi-Armed Bandits with Control Variates.'

About Repository This repository contains code used for plots in NeurIPS 2021 paper 'Stochastic Multi-Armed Bandits with Control Variates.' About Code

Arun Verma 1 Nov 09, 2021
PyTorch Connectomics: segmentation toolbox for EM connectomics

Introduction The field of connectomics aims to reconstruct the wiring diagram of the brain by mapping the neural connections at the level of individua

Zudi Lin 132 Dec 26, 2022
Instance-wise Occlusion and Depth Orders in Natural Scenes (CVPR 2022)

Instance-wise Occlusion and Depth Orders in Natural Scenes Official source code. Appears at CVPR 2022 This repository provides a new dataset, named In

27 Dec 27, 2022
Generate pixel-style avatars with python.

face2pixel Generate pixel-style avatars with python. Run: Clone the project: git clone https://github.com/theodorecooper/face2pixel install requiremen

Theodore Cooper 2 May 11, 2022
Mixed Transformer UNet for Medical Image Segmentation

MT-UNet Update 2021/11/19 Thank you for your interest in our work. We have uploaded the code of our MTUNet to help peers conduct further research on i

dotman 92 Dec 25, 2022
Pytorch implementation for M^3L

Learning to Generalize Unseen Domains via Memory-based Multi-Source Meta-Learning for Person Re-Identification (CVPR 2021) Introduction This is the Py

Yuyang Zhao 45 Dec 26, 2022
ROS support for Velodyne 3D LIDARs

Overview Velodyne1 is a collection of ROS2 packages supporting Velodyne high definition 3D LIDARs3. Warning: The master branch normally contains code

ROS device drivers 543 Dec 30, 2022
IPATool-py: download ipa easily

IPATool-py Python version of IPATool! Installation pip3 install -r requirements.txt Usage Quickstart: download app with specific bundleId into DIR: p

159 Dec 30, 2022
Unofficial implementation of Point-Unet: A Context-Aware Point-Based Neural Network for Volumetric Segmentation

Point-Unet This is an unofficial implementation of the MICCAI 2021 paper Point-Unet: A Context-Aware Point-Based Neural Network for Volumetric Segment

Namt0d 9 Dec 07, 2022
Jittor Medical Segmentation Lib -- The assignment of Pattern Recognition course (2021 Spring) in Tsinghua University

THU模式识别2021春 -- Jittor 医学图像分割 模型列表 本仓库收录了课程作业中同学们采用jittor框架实现的如下模型: UNet SegNet DeepLab V2 DANet EANet HarDNet及其改动HarDNet_alter PSPNet OCNet OCRNet DL

48 Dec 26, 2022
OpenVisionAPI server

🚀 Quick start An instance of ova-server is free and publicly available here: https://api.openvisionapi.com Checkout ova-client for a quick demo. Inst

Open Vision API 93 Nov 24, 2022