MMGeneration is a powerful toolkit for generative models, based on PyTorch and MMCV.

Overview

Documentation: https://mmgeneration.readthedocs.io/

Introduction

English | 简体中文

MMGeneration is a powerful toolkit for generative models, especially for GANs now. It is based on PyTorch and MMCV. The master branch works with PyTorch 1.5+.

Major Features

  • High-quality Training Performance: We currently support training on Unconditional GANs, Internal GANs, and Image Translation Models. Support for conditional models will come soon.
  • Powerful Application Toolkit: A plentiful toolkit containing multiple applications in GANs is provided to users. GAN interpolation, GAN projection, and GAN manipulations are integrated into our framework. It's time to play with your GANs! (Tutorial for applications)
  • Efficient Distributed Training for Generative Models: For the highly dynamic training in generative models, we adopt a new way to train dynamic models with MMDDP. (Tutorial for DDP)
  • New Modular Design for Flexible Combination: A new design for complex loss modules is proposed for customizing the links between modules, which can achieve flexible combination among different modules. (Tutorial for new modular design)
Training Visualization
GAN Interpolation
GAN Projector
GAN Manipulation

Highlight

  • Positional Encoding as Spatial Inductive Bias in GANs (CVPR2021) has been released in MMGeneration. [Config], [Project Page]

Changelog

v0.1.0 was released on 20/04/2021. Please refer to changelog.md for details and release history.

ModelZoo

These methods have been carefully studied and supported in our frameworks:

Unconditional GANs (click to collapse)
Image2Image Translation (click to collapse)
Internal Learing (click to collapse)

License

This project is released under the Apache 2.0 license. Some operations in MMGeneration are with other licenses instead of Apache2.0. Please refer to LICENSES.md for the careful check, if you are using our code for commercial matters.

Installation

Please refer to get_started.md for installation.

Getting Started

Please see get_started.md for the basic usage of MMGeneration. docs/quick_run.md can offer full guidance for quick run. For other details and tutorials, please go to our documentation.

Contributing

We appreciate all contributions to improve MMGeneration. Please refer to CONTRIBUTING.md in MMCV for more details about the contributing guideline.

Citation

If you find this project useful in your research, please consider cite:

@misc{2021mmgeneration,
    title={{MMGeneration}: OpenMMLab Generative Model Toolbox and Benchmark},
    author={MMGeneration Contributors},
    howpublished = {\url{https://github.com/open-mmlab/mmgeneration}},
    year={2021}
}

Projects in OpenMMLab

  • MMCV: OpenMMLab foundational library for computer vision.
  • MMClassification: OpenMMLab image classification toolbox and benchmark.
  • MMDetection: OpenMMLab detection toolbox and benchmark.
  • MMDetection3D: OpenMMLab's next-generation platform for general 3D object detection.
  • MMSegmentation: OpenMMLab semantic segmentation toolbox and benchmark.
  • MMAction2: OpenMMLab's next-generation action understanding toolbox and benchmark.
  • MMTracking: OpenMMLab video perception toolbox and benchmark.
  • MMPose: OpenMMLab pose estimation toolbox and benchmark.
  • MMEditing: OpenMMLab image and video editing toolbox.
  • MMOCR: A Comprehensive Toolbox for Text Detection, Recognition and Understanding.
  • MMGeneration: OpenMMLab's next-generation toolbox for generative models.
Comments
  • Fix s3 readme of 1.x branch

    Fix s3 readme of 1.x branch

    Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

    Motivation

    Please describe the motivation of this PR and the goal you want to achieve through this PR.

    Modification

    Please briefly describe what modification is made in this PR.

    Who can help? @ them here!

    BC-breaking (Optional)

    Does the modification introduce changes that break the backward-compatibility of the downstream repositories? If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.

    Use cases (Optional)

    If this PR introduces a new feature, it is better to list some use cases here, and update the documentation.

    Checklist

    Before PR:

    • [ ] I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
    • [ ] Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
    • [ ] Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
    • [ ] New functionalities are covered by complete unit tests. If not, please add more unit test to ensure the correctness.
    • [ ] The documentation has been modified accordingly, including docstring or example tutorials.

    After PR:

    • [ ] If the modification has potential influence on downstream or other related projects, this PR should be tested with some of those projects.
    • [ ] CLA has been signed and all committers have signed the CLA in this PR.
    opened by plyfager 0
  • Fix s3 readme dev-1.x

    Fix s3 readme dev-1.x

    Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

    Motivation

    Please describe the motivation of this PR and the goal you want to achieve through this PR.

    Modification

    Please briefly describe what modification is made in this PR.

    Who can help? @ them here!

    BC-breaking (Optional)

    Does the modification introduce changes that break the backward-compatibility of the downstream repositories? If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.

    Use cases (Optional)

    If this PR introduces a new feature, it is better to list some use cases here, and update the documentation.

    Checklist

    Before PR:

    • [ ] I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
    • [ ] Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
    • [ ] Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
    • [ ] New functionalities are covered by complete unit tests. If not, please add more unit test to ensure the correctness.
    • [ ] The documentation has been modified accordingly, including docstring or example tutorials.

    After PR:

    • [ ] If the modification has potential influence on downstream or other related projects, this PR should be tested with some of those projects.
    • [ ] CLA has been signed and all committers have signed the CLA in this PR.
    opened by plyfager 0
  • Fix s3 readme with refined metrics

    Fix s3 readme with refined metrics

    Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

    Motivation

    Please describe the motivation of this PR and the goal you want to achieve through this PR.

    Modification

    Please briefly describe what modification is made in this PR.

    Who can help? @ them here!

    BC-breaking (Optional)

    Does the modification introduce changes that break the backward-compatibility of the downstream repositories? If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.

    Use cases (Optional)

    If this PR introduces a new feature, it is better to list some use cases here, and update the documentation.

    Checklist

    Before PR:

    • [ ] I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
    • [ ] Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
    • [ ] Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
    • [ ] New functionalities are covered by complete unit tests. If not, please add more unit test to ensure the correctness.
    • [ ] The documentation has been modified accordingly, including docstring or example tutorials.

    After PR:

    • [ ] If the modification has potential influence on downstream or other related projects, this PR should be tested with some of those projects.
    • [ ] CLA has been signed and all committers have signed the CLA in this PR.
    opened by plyfager 0
  • Latent-Diffusion Models

    Latent-Diffusion Models

    Model/Dataset/Scheduler description

    Recent trends show that diffusion models, especially latent diffusion models, have been very successful for image/video generation tasks. While the current codebase supports IDDPM, I believe supporting LDM for tasks like unconditional image generation or even text-to-image generation would be very helpful for the openmmlab family.

    Open source status

    • [ ] The model implementation is available
    • [ ] The model weights are available.

    Provide useful links for the implementation

    https://github.com/CompVis/latent-diffusion

    opened by Adamdad 5
  • add colab page

    add colab page

    Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

    Motivation

    Please describe the motivation of this PR and the goal you want to achieve through this PR.

    Modification

    Please briefly describe what modification is made in this PR.

    Who can help? @ them here!

    BC-breaking (Optional)

    Does the modification introduce changes that break the backward-compatibility of the downstream repositories? If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.

    Use cases (Optional)

    If this PR introduces a new feature, it is better to list some use cases here, and update the documentation.

    Checklist

    Before PR:

    • [ ] I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
    • [ ] Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
    • [ ] Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
    • [ ] New functionalities are covered by complete unit tests. If not, please add more unit test to ensure the correctness.
    • [ ] The documentation has been modified accordingly, including docstring or example tutorials.

    After PR:

    • [ ] If the modification has potential influence on downstream or other related projects, this PR should be tested with some of those projects.
    • [ ] CLA has been signed and all committers have signed the CLA in this PR.
    opened by plyfager 0
Releases(v0.7.2)
  • v0.7.2(Sep 14, 2022)

    Improvements

    • [Docs] Fix typos in docs. by @RangeKing in https://github.com/open-mmlab/mmgeneration/pull/425
    • [Docs] Add doc for StyleGAN-Ada by @plyfager in https://github.com/open-mmlab/mmgeneration/pull/391
    • [Fix] Update MMCV_MAX to 1.7 by @LeoXing1996 in https://github.com/open-mmlab/mmgeneration/pull/436
    • [CI] update github workflow, circleci and github templates by @LeoXing1996 in https://github.com/open-mmlab/mmgeneration/pull/431

    New Contributors

    • @RangeKing made their first contribution in https://github.com/open-mmlab/mmgeneration/pull/425
    Source code(tar.gz)
    Source code(zip)
  • v1.0.0rc0(Sep 1, 2022)

    MMGeneration 1.0.0rc0 is the first version of MMGeneration 1.x, a part of the OpenMMLab 2.0 projects.

    Built upon the new training engine, MMGeneration 1.x unifies the interfaces of dataset, models, evaluation, and visualization.

    And there are some BC-breaking changes. Please check the migration tutorial for more details.

    Source code(tar.gz)
    Source code(zip)
  • v0.7.1(Apr 30, 2022)

    Fix bugs and Improvements

    • Support train_dataloader, val_dataloader and test_dataloader settings (#281)
    • Fix ada typo (#283)
    • Add chinese application tutorial (#284)
    • Add chinese document of ddp training (#286)

    Contributors

    @plyfager @LeoXing1996

    Source code(tar.gz)
    Source code(zip)
  • v0.7.0(Apr 2, 2022)

    Highlights

    • Support training of StyleGANv3 (#275, #277)
    • Support adaptive discriminator augmentation (#276)

    New Features

    • Support passing training arguments in static unconditional gan (#275)
    • Support dynamic EMA, now you can define momentum updating policy (#261)
    • Add multi machine distribute train (#267)

    Fix bugs and Improvements

    • Add brief installation steps in README (#270)
    • Support random seed for distributed sampler (#271)
    • Use hyphen for command line args in apps (#273)

    Contributors

    @plyfager @LeoXing1996

    Source code(tar.gz)
    Source code(zip)
  • v0.6.0(Mar 7, 2022)

    Highlights

    • Support StyleGANv3 (#247, #253, #258)
    • Support StyleCLIP (#236)

    New Features

    • Support training on CPU (#238)
    • Speed up training (#231)

    Fix bugs and Improvements

    • Fix bug in non-distributed training/testing (#239)
    • Fix typos and invalid links (#221, #226, #228, #244, #249)
    • Add part of Chinese documentation (#250, #257)

    Contributors

    @plyfager @LeoXing1996 @gvalvano @JimHeo @plutoyuxie

    Source code(tar.gz)
    Source code(zip)
  • v0.5.0(Jan 12, 2022)

    Highlights

    • Support BigGAN style's Spectral Norm and update BigGAN with best FID and IS (#159)
    • Support import projected latent and export video in interpolation (#167)
    • Support Improved-DDPM model (#205)

    New Features

    • Support evaluation in distributed mode (#151)
    • Support presistent_work in validation dataloader (#179)
    • Support dockerfile (#200)
    • Support mim (#176)

    Fix bugs and Improvements

    • Fix bug in SinGAN dataset (#192)
    • Fix SAGAN, SNGAN and BigGAN's default sn_style (#199, #213, #215, #217)

    Contributors

    @plyfager @LeoXing1996 @nbei @TommyZihao @JiangongWang

    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(Nov 3, 2021)

    Highlights

    • Add more experiments for conditional GANs: SNGAN, SAGAN, and BigGAN
    • Refact Translation Model (#88, #126, #127, #145)

    New Features

    • Use PyTorch Sphinx theme #123
    • Support torchserve for unconditional models #131

    Fix bugs and Improvements

    • Add CI for python3.9 #110
    • Add support for PyTorch1.9 #115
    • Add pre-commit hook for spell checking #135

    Contributors

    @plyfager , @LeoXing1996 , @ckkelvinchan , @nbei

    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Aug 2, 2021)

    Highlights

    • Support conditional GANs: Projection GAN, SNGAN, SAGAN, and BigGAN

    New Features

    • Add support for persistent_workers in PyTorch >= 1.7.0 #71
    • Support warm-up for EMA #55

    Fix bugs and Improvements

    • Fix failing to build docs #64
    • Revise the logic of num_classes in basic conditional gan #69
    • Support dynamic eval internal in eval hook #73
    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(May 30, 2021)

    Highlights

    • Support new methods: LSGAN, GGAN.
    • Support mixed-precision training (FP16): official PyTorch Implementation and APEX (#11, #20)

    New Features

    • Add the experiment of MNIST in DCGAN (#24)
    • Add support for uploading checkpoints to Ceph system (cloud server) (#27)
    • Add the functionality of saving the best checkpoint in GenerativeEvalHook (#21)

    Fix bugs and Improvements

    • Fix loss of sample-cfg argument (#13)
    • Add pbar to offline eval and fix bug in grayscale image evaluation/saving (#23)
    • Fix error when data_root option in val_cfg or test_cfg are set as None (#28)
    • Change latex in quick_run.md to svg url and fix number of checkpoints in modelzoo_statistics.md (#34)
    Source code(tar.gz)
    Source code(zip)
  • v0.1.0(Apr 23, 2021)

    Highlights

    • MMGeneration v0.1.0 is released.

    Main Features

    • High-quality Training Performance: We currently support training on Unconditional GANs(DCGAN, WGAN-GP,PGGAN, StyleGANV1, StyleGANV2, Positional Encoding in GANs), Internal GANs(SinGAN), and Image Translation Models(Pix2Pix, CycleGAN). Support for conditional models will come soon.
    • Powerful Application Toolkit: A plentiful toolkit containing multiple applications in GANs is provided to users. GAN interpolation, GAN projection, and GAN manipulations are integrated into our framework. It's time to play with your GANs!
    • Efficient Distributed Training for Generative Models: For the highly dynamic training in generative models, we adopt a new way to train dynamic models with MMDDP.
    • New Modular Design for Flexible Combination: A new design for complex loss modules is proposed for customizing the links between modules, which can achieve flexible combinations among different modules.
    Source code(tar.gz)
    Source code(zip)
Owner
OpenMMLab
OpenMMLab
Repository providing a wide range of self-supervised pretrained models for computer vision tasks.

Hierarchical Pretraining: Research Repository This is a research repository for reproducing the results from the project "Self-supervised pretraining

Colorado Reed 53 Nov 09, 2022
Official PyTorch implementation of "The Center of Attention: Center-Keypoint Grouping via Attention for Multi-Person Pose Estimation" (ICCV 21).

CenterGroup This the official implementation of our ICCV 2021 paper The Center of Attention: Center-Keypoint Grouping via Attention for Multi-Person P

Dynamic Vision and Learning Group 43 Dec 25, 2022
A curated list of the top 10 computer vision papers in 2021 with video demos, articles, code and paper reference.

The Top 10 Computer Vision Papers of 2021 The top 10 computer vision papers in 2021 with video demos, articles, code, and paper reference. While the w

Louis-François Bouchard 118 Dec 21, 2022
[NAACL & ACL 2021] SapBERT: Self-alignment pretraining for BERT.

SapBERT: Self-alignment pretraining for BERT This repo holds code for the SapBERT model presented in our NAACL 2021 paper: Self-Alignment Pretraining

Cambridge Language Technology Lab 104 Dec 07, 2022
Simple Tensorflow implementation of Toward Spatially Unbiased Generative Models (ICCV 2021)

Spatial unbiased GANs — Simple TensorFlow Implementation [Paper] : Toward Spatially Unbiased Generative Models (ICCV 2021) Abstract Recent image gener

Junho Kim 16 Apr 15, 2022
L-Verse: Bidirectional Generation Between Image and Text

Far beyond learning long-range interactions of natural language, transformers are becoming the de-facto standard for many vision tasks with their power and scalabilty

Kim, Taehoon 102 Dec 21, 2022
Fully Convolutional DenseNet (A.K.A 100 layer tiramisu) for semantic segmentation of images implemented in TensorFlow.

FC-DenseNet-Tensorflow This is a re-implementation of the 100 layer tiramisu, technically a fully convolutional DenseNet, in TensorFlow (Tiramisu). Th

Hasnain Raza 121 Oct 12, 2022
Plaything for Autistic Children (demo for PaddlePaddle/Wechaty/Mixlab project)

星星的孩子 - 一款为孤独症孩子设计的聊天机器人游戏 孤独症儿童是目前常常被忽视的一类群体。他们有着类似性格内向的特征,实际却受着广泛性发育障碍的折磨。 项目背景 这类儿童在与人交往时存在着沟通障碍,其特点表现在: 社交交流差,互动障碍明显 认知能力有限,被动认知 兴趣狭窄,重复刻板,缺乏变化和想象

Tianyi Pan 35 Nov 24, 2022
Using deep actor-critic model to learn best strategies in pair trading

Deep-Reinforcement-Learning-in-Stock-Trading Using deep actor-critic model to learn best strategies in pair trading Abstract Partially observed Markov

281 Dec 09, 2022
Audio-Visual Generalized Few-Shot Learning with Prototype-Based Co-Adaptation

Audio-Visual Generalized Few-Shot Learning with Prototype-Based Co-Adaptation The code repository for "Audio-Visual Generalized Few-Shot Learning with

Kaiaicy 3 Jun 27, 2022
Fine-Tune EleutherAI GPT-Neo to Generate Netflix Movie Descriptions in Only 47 Lines of Code Using Hugginface And DeepSpeed

GPT-Neo-2.7B Fine-Tuning Example Using HuggingFace & DeepSpeed Installation cd venv/bin ./pip install -r ../../requirements.txt ./pip install deepspe

Nikita 180 Jan 05, 2023
Few-shot Learning of GPT-3

Few-shot Learning With Language Models This is a codebase to perform few-shot "in-context" learning using language models similar to the GPT-3 paper.

Tony Z. Zhao 224 Dec 28, 2022
Open source repository for the code accompanying the paper 'PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations'.

PatchNets This is the official repository for the project "PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations". For details,

16 May 22, 2022
TorchDistiller - a collection of the open source pytorch code for knowledge distillation, especially for the perception tasks, including semantic segmentation, depth estimation, object detection and instance segmentation.

This project is a collection of the open source pytorch code for knowledge distillation, especially for the perception tasks, including semantic segmentation, depth estimation, object detection and i

yifan liu 147 Dec 03, 2022
PyTorch implementation of the implicit Q-learning algorithm (IQL)

Implicit-Q-Learning (IQL) PyTorch implementation of the implicit Q-learning algorithm IQL (Paper) Currently only implemented for online learning. Offl

Sebastian Dittert 27 Dec 30, 2022
The repo of Feedback Networks, CVPR17

Feedback Networks http://feedbacknet.stanford.edu/ Paper: Feedback Networks, CVPR 2017. Amir R. Zamir*,Te-Lin Wu*, Lin Sun, William B. Shen, Bertram E

Stanford Vision and Learning Lab 87 Nov 19, 2022
Losslandscapetaxonomy - Taxonomizing local versus global structure in neural network loss landscapes

Taxonomizing local versus global structure in neural network loss landscapes Int

Yaoqing Yang 8 Dec 30, 2022
Implementation of Research Paper "Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation"

Zero-DCE and Zero-DCE++(Lite architechture for Mobile and edge Devices) Papers Abstract The paper presents a novel method, Zero-Reference Deep Curve E

Tauhid Khan 15 Dec 10, 2022
:hot_pepper: R²SQL: "Dynamic Hybrid Relation Network for Cross-Domain Context-Dependent Semantic Parsing." (AAAI 2021)

R²SQL The PyTorch implementation of paper Dynamic Hybrid Relation Network for Cross-Domain Context-Dependent Semantic Parsing. (AAAI 2021) Requirement

huybery 60 Dec 31, 2022
Efficient electromagnetic solver based on rigorous coupled-wave analysis for 3D and 2D multi-layered structures with in-plane periodicity

Efficient electromagnetic solver based on rigorous coupled-wave analysis for 3D and 2D multi-layered structures with in-plane periodicity, such as gratings, photonic-crystal slabs, metasurfaces, surf

Alex Song 17 Dec 19, 2022