Neural Residual Flow Fields for Efficient Video Representations

1. Download MPI sintel dataset

Download MPI sintel dataset from here

2. GMA optical flow estimator

To obtain optical flow estimations for pretraining, we are using GMA from here. Note that it dose not have to do with our identity.

3. Training

Training neural residual flow fields (NRFF)

# frame 0 - 6
python train_video_flow_midkey.py --use-estimator --lr 0.0005 --training-step 30000 --data-dir {sintel dataset training directory} --video-name alley_1 --start-frame 0 --num-frames 7 --jpeg-quality 98 --hidden-features 96 --use-estimator --tag start0_jq98_hf96
# frame 7 - 13
python train_video_flow_midkey.py --use-estimator --lr 0.0005 --training-step 30000 --data-dir {sintel dataset training directory} --video-name alley_1 --start-frame 7 --num-frames 7 --jpeg-quality 98 --hidden-features 96 --use-estimator --tag start7_jq98_hf96
# frame 14 - 20
python train_video_flow_midkey.py --use-estimator --lr 0.0005 --training-step 30000 --data-dir {sintel dataset training directory} --video-name alley_1 --start-frame 14 --num-frames 7 --jpeg-quality 98 --hidden-features 96 --use-estimator --tag start14_jq98_hf96
# frame 21 - 27
python train_video_flow_midkey.py --use-estimator --lr 0.0005 --training-step 30000 --data-dir {sintel dataset training directory} --video-name alley_1 --start-frame 21 --num-frames 7 --jpeg-quality 98 --hidden-features 96 --use-estimator --tag start21_jq98_hf96

Training baseline (SIREN)

python train_video.py --data-dir {sintel dataset training directory} --video-name alley_1 --hidden-features 256 --num-frames 28 --lr 0.001 --training-step 30000 --tag baseline_siren_hf256

4. Examples

alley_2.mp4

HoneyBee.mp4

Eff video representation - Efficient video representation through neural fields

Related tags

Overview

Neural Residual Flow Fields for Efficient Video Representations

1. Download MPI sintel dataset

2. GMA optical flow estimator

3. Training

4. Examples

Owner

Tooling for converting STAC metadata to ODC data model

Embracing Single Stride 3D Object Detector with Sparse Transformer

LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation (NeurIPS2021 Benchmark and Dataset Track)

An addernet CUDA version

Run Effective Large Batch Contrastive Learning on Limited Memory GPU

A curated list of Machine Learning and Deep Learning tutorials in Jupyter Notebook format ready to run in Google Colaboratory

🤗 Push your spaCy pipelines to the Hugging Face Hub

[CVPR'22] COAP: Learning Compositional Occupancy of People

System-oriented IR evaluations are limited to rather abstract understandings of real user behavior

face property detection pytorch

Face Mask Detection is a project to determine whether someone is wearing mask or not, using deep neural network.

PyTorch implementation of "Simple and Deep Graph Convolutional Networks"

Unet network with mean teacher for altrasound image segmentation

Code for STFT Transformer used in BirdCLEF 2021 competition.

On Evaluation Metrics for Graph Generative Models

Pytorch implementation of the popular Improv RNN model originally proposed by the Magenta team.

Pytorch implementation of four neural network based domain adaptation techniques: DeepCORAL, DDC, CDAN and CDAN+E. Evaluated on benchmark dataset Office31.

Monocular Depth Estimation Using Laplacian Pyramid-Based Depth Residuals

PyTorch implementation of "Image-to-Image Translation Using Conditional Adversarial Networks".

SCAN: Learning to Classify Images without Labels, incl. SimCLR. [ECCV 2020]