Official implementation of Densely connected normalizing flows

Last update: Dec 12, 2022

Related tags

Overview

Densely connected normalizing flows

This repository is the official implementation of NeurIPS 2021 paper Densely connected normalizing flows. Poster available here.

Setup

CUDA 11.1
Python 3.8

pip install -r requirements.txt
pip install -e .

Training

cd ./experiments/image

CIFAR-10:

python train.py --epochs 400 --batch_size 64 --optimizer adamax --lr 1e-3  --gamma 0.9975 --warmup 5000  --eval_every 1 --check_every 10 --dataset cifar10 --augmentation eta --block_conf 6 4 1 --layers_conf  5 6 20  --layer_mid_chnls 48 48 48 --growth_rate 10  --name DF_74_10

python train_more.py --model ./log/cifar10_8bit/densenet-flow/expdecay/DF_74_10 --new_lr 2e-5 --new_epochs 420

ImageNet32:

python train.py --epochs 20 --batch_size 64 --optimizer adamax --lr 1e-3  --gamma 0.95 --warmup 5000  --eval_every 1 --check_every 10 --dataset imagenet32 --augmentation eta --block_conf 6 4 1 --layers_conf  5 6 20  --layer_mid_chnls 48 48 48 --growth_rate 10  --name DF_74_10

python train_more.py --model ./log/imagenet32_8bit/densenet-flow/expdecay/DF_74_10 --new_lr 2e-5 --new_epochs 22

ImageNet64:

python train.py --epochs 10 --batch_size 32 --optimizer adamax --lr 1e-3  --gamma 0.95 --warmup 5000  --eval_every 1 --check_every 10 --dataset imagenet64 --augmentation eta --block_conf 6 4 1 --layers_conf  5 6 20  --layer_mid_chnls 48 48 48 --growth_rate 10  --name DF_74_10

python train_more.py --model ./log/imagenet64_8bit/densenet-flow/expdecay/DF_74_10 --new_lr 2e-5 --new_epochs 11

CelebA:

python train.py --epochs 50 --batch_size 32 --optimizer adamax --lr 1e-3  --gamma 0.95 --warmup 5000  --eval_every 1 --check_every 10 --dataset celeba --augmentation horizontal_flip --block_conf 6 4 1 --layers_conf  5 6 20  --layer_mid_chnls 48 48 48 --growth_rate 10  --name DF_74_10

python train_more.py --model ./log/celeba_8bit/densenet-flow/expdecay/DF_74_10 --new_lr 2e-5 --new_epochs 55

Note: Download instructions for ImageNet and CelebA are defined in denseflow/data/datasets/image/{dataset}.py

Evaluation

CIFAR-10:

python eval_loglik.py --model PATH_TO_MODEL --k 1000 --kbs 50

ImageNet32:

python eval_loglik.py --model PATH_TO_MODEL --k 200 --kbs 50

ImageNet64 and CelebA:

python eval_loglik.py --model PATH_TO_MODEL --k 200 --kbs 25

Model weights

Model weights are stored here.

Samples generation

Generated samples are stored in PATH_TO_MODEL/samples

python eval_sample.py --model PATH_TO_MODEL

Note: PATH_TO_MODEL has to contain check directory.

ImageNet 32x32

ImageNet 64x64

CelebA

Acknowledgements

Significant part of this code benefited from SurVAE [1] code implementation, available under MIT license.

References

[1] Didrik Nielsen, Priyank Jaini, Emiel Hoogeboom, Ole Winther, and Max Welling. Survae flows: Surjections to bridge the gap between vaes and flows. InAdvances in Neural Information Processing Systems 33. Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020.

Official implementation of Densely connected normalizing flows

Related tags

Overview

Densely connected normalizing flows

Setup

Training

Evaluation

Model weights

Samples generation

ImageNet 32x32

ImageNet 64x64

CelebA

Acknowledgements

References

Owner

Matej Grcić

Improving Transferability of Representations via Augmentation-Aware Self-Supervision

Re-implememtation of MAE (Masked Autoencoders Are Scalable Vision Learners) using PyTorch.

OOD Dataset Curator and Benchmark for AI-aided Drug Discovery

Semantic Segmentation for Aerial Imagery using Convolutional Neural Network

Official pytorch implementation of "Scaling-up Disentanglement for Image Translation", ICCV 2021.

Semi-supervised Domain Adaptation via Minimax Entropy

Remote sensing change detection tool based on PaddlePaddle

Cycle Consistent Adversarial Domain Adaptation (CyCADA)

SpecAugmentPyTorch - A Pytorch (support batch and channel) implementation of GoogleBrain's SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition

Neural-Pull: Learning Signed Distance Functions from Point Clouds by Learning to Pull Space onto Surfaces(ICML 2021)

The spiritual successor to knockknock for PyTorch Lightning, get notified when your training ends

💊 A 3D Generative Model for Structure-Based Drug Design (NeurIPS 2021)

基于pytorch构建cyclegan示例

Predictive Modeling on Electronic Health Records(EHR) using Pytorch

Official Implementation of DE-CondDETR and DELA-CondDETR in "Towards Data-Efficient Detection Transformers"

Pytorch implementation of Depth-conditioned Dynamic Message Propagation forMonocular 3D Object Detection

Reproduced Code for Image Forgery Detection papers.

Read Like Humans: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Recognition

Scalable, event-driven, deep-learning-friendly backtesting library

DETReg: Unsupervised Pretraining with Region Priors for Object Detection