Asymmetric Bilateral Motion Estimation for Video Frame Interpolation, ICCV2021

Related tags

Deep LearningABME
Overview

ABME (ICCV2021)

PWC PWC

Junheum Park, Chul Lee, and Chang-Su Kim

Official PyTorch Code for "Asymmetric Bilateral Motion Estimation for Video Frame Interpolation" [paper]

Requirements

  • PyTorch 1.7
  • CUDA 11.0
  • CuDNN 8.0.5
  • python 3.8

Installation

Create conda environment:

    $ conda create -n ABME python=3.8 anaconda
    $ conda activate ABME
    $ pip install opencv-python
    $ conda install pytorch==1.7 torchvision cudatoolkit=11.0 -c pytorch

Download repository:

    $ git clone https://github.com/JunHeum/ABME.git

Download pre-trained model parameters:

    $ unzip ABME_Weights.zip

Check your nvcc version:

    $ nvcc --version
  • To install correlation layer, you should match your nvcc version with cudatoolkit version of your conda environment. [nvcc_setting]

Install correlation layer:

    $ cd correlation_package
    $ python setup.py install

Quick Usage

Generate an intermediate frame on your pair of frames:

    $ python run.py --first images/im1.png --second images/im3.png --output images/im2.png

Test

  1. Download the datasets.
  2. Copy the path of the test dataset. (e.g., /hdd/vimeo_interp_test)
  3. Parse this path into the --dataset_root argument.
  4. (optional) You can ignore the --is_save. But, it yields a slightly different performance than evaluation on saved images.
    $ python test.py --name ABME --is_save --Dataset ucf101 --dataset_root /where/is/your/ucf101_dataset/path
    $ python test.py --name ABME --is_save --Dataset vimeo --dataset_root /where/is/your/vimeo_dataset/path
    $ python test.py --name ABME --is_save --Dataset SNU-FILM-all --dataset_root /where/is/your/FILM_dataset/path
    $ python test.py --name ABME --is_save --Dataset Xiph_HD --dataset_root /where/is/your/Xiph_dataset/path
    $ python test.py --name ABME --is_save --Dataset X4K1000FPS --dataset_root /where/is/your/X4K1000FPS_dataset/path

Experimental Results

We provide interpolated frames on test datasets for fast comparison or users with limited GPU memory. Especially, the test on X4K1000FPS requires at least 20GB of GPU memory.

Table

Train

We plan to share train codes soon!

Citation

Please cite the following paper if you feel this repository useful.

    @inproceedings{park2021ABME,
        author    = {Park, Junheum and Lee, Chul and Kim, Chang-Su}, 
        title     = {Asymmetric Bilateral Motion Estimation for Video Frame Interpolation}, 
        booktitle = {International Conference on Computer Vision},
        year      = {2021}
    }

License

See MIT License

Owner
Junheum Park
BS: EE, Korea University Grad: EE, Korea University (Current)
Junheum Park
Caffe models in TensorFlow

Caffe to TensorFlow Convert Caffe models to TensorFlow. Usage Run convert.py to convert an existing Caffe model to TensorFlow. Make sure you're using

Saumitro Dasgupta 2.8k Dec 31, 2022
Official PyTorch implementation of the Fishr regularization for out-of-distribution generalization

Fishr: Invariant Gradient Variances for Out-of-distribution Generalization Official PyTorch implementation of the Fishr regularization for out-of-dist

62 Dec 22, 2022
Build a medical knowledge graph based on Unified Language Medical System (UMLS)

UMLS-Graph Build a medical knowledge graph based on Unified Language Medical System (UMLS) Requisite Install MySQL Server 5.6 and import UMLS data int

Donghua Chen 6 Dec 25, 2022
Confidence Propagation Cluster aims to replace NMS-based methods as a better box fusion framework in 2D/3D Object detection

CP-Cluster Confidence Propagation Cluster aims to replace NMS-based methods as a better box fusion framework in 2D/3D Object detection, Instance Segme

Yichun Shen 41 Dec 08, 2022
Implementation for Curriculum DeepSDF

Curriculum-DeepSDF This repository is an implementation for Curriculum DeepSDF. Full paper is available here. Preparation Please follow original setti

Haidong Zhu 69 Dec 29, 2022
SweiNet is an uncertainty-quantifying shear wave speed (SWS) estimator for ultrasound shear wave elasticity (SWE) imaging.

SweiNet SweiNet is an uncertainty-quantifying shear wave speed (SWS) estimator for ultrasound shear wave elasticity (SWE) imaging. SweiNet takes as in

Felix Jin 3 Mar 31, 2022
Decensoring Hentai with Deep Neural Networks. Formerly named DeepMindBreak.

DeepCreamPy Decensoring Hentai with Deep Neural Networks. Formerly named DeepMindBreak. A deep learning-based tool to automatically replace censored a

616 Jan 06, 2023
Incomplete easy-to-use math solver and PDF generator.

Math Expert Let me do your work Preview preview.mp4 Introduction Math Expert is our (@salastro, @younis-tarek, @marawn-mogeb) math high school graduat

SalahDin Ahmed 22 Jul 11, 2022
Binary Passage Retriever (BPR) - an efficient passage retriever for open-domain question answering

BPR Binary Passage Retriever (BPR) is an efficient neural retrieval model for open-domain question answering. BPR integrates a learning-to-hash techni

Studio Ousia 147 Dec 07, 2022
Code for the paper Language as a Cognitive Tool to Imagine Goals in Curiosity Driven Exploration

IMAGINE: Language as a Cognitive Tool to Imagine Goals in Curiosity Driven Exploration This repo contains the code base of the paper Language as a Cog

Flowers Team 26 Dec 22, 2022
Task Transformer Network for Joint MRI Reconstruction and Super-Resolution (MICCAI 2021)

T2Net Task Transformer Network for Joint MRI Reconstruction and Super-Resolution (MICCAI 2021) [Paper][Code] Dependencies numpy==1.18.5 scikit_image==

64 Nov 23, 2022
Hitters Linear Regression - Hitters Linear Regression With Python

Hitters_Linear_Regression Kullanacağımız veri seti Carnegie Mellon Üniversitesi'

AyseBuyukcelik 2 Jan 26, 2022
Repository for "Space-Time Correspondence as a Contrastive Random Walk" (NeurIPS 2020)

Space-Time Correspondence as a Contrastive Random Walk This is the repository for Space-Time Correspondence as a Contrastive Random Walk, published at

A. Jabri 239 Dec 27, 2022
Code for MarioNette: Self-Supervised Sprite Learning, in NeurIPS 2021

MarioNette | Webpage | Paper | Video MarioNette: Self-Supervised Sprite Learning Dmitriy Smirnov, Michaël Gharbi, Matthew Fisher, Vitor Guizilini, Ale

Dima Smirnov 28 Nov 18, 2022
Off-policy continuous control in PyTorch, with RDPG, RTD3 & RSAC

arXiv technical report soon available. we are updating the readme to be as comprehensive as possible Please ask any questions in Issues, thanks. Intro

Zhihan 31 Dec 30, 2022
Official repository of the paper "GPR1200: A Benchmark for General-PurposeContent-Based Image Retrieval"

GPR1200 Dataset GPR1200: A Benchmark for General-Purpose Content-Based Image Retrieval (ArXiv) Konstantin Schall, Kai Uwe Barthel, Nico Hezel, Klaus J

Visual Computing Group 16 Nov 21, 2022
Monocular Depth Estimation - Weighted-average prediction from multiple pre-trained depth estimation models

merged_depth runs (1) AdaBins, (2) DiverseDepth, (3) MiDaS, (4) SGDepth, and (5) Monodepth2, and calculates a weighted-average per-pixel absolute dept

Pranav 39 Nov 21, 2022
Implementation of the paper Recurrent Glimpse-based Decoder for Detection with Transformer.

REGO-Deformable DETR By Zhe Chen, Jing Zhang, and Dacheng Tao. This repository is the implementation of the paper Recurrent Glimpse-based Decoder for

Zhe Chen 33 Nov 30, 2022
A PyTorch implementation of deep-learning-based registration

DiffuseMorph Implementation A PyTorch implementation of deep-learning-based registration. Requirements OS : Ubuntu / Windows Python 3.6 PyTorch 1.4.0

24 Jan 03, 2023
Using the provided dataset which includes various book features, in order to predict the price of books, using various proposed methods and models.

Using the provided dataset which includes various book features, in order to predict the price of books, using various proposed methods and models.

Nikolas Petrou 1 Jan 13, 2022