MODNet: Trimap-Free Portrait Matting in Real Time

Overview

MODNet: Trimap-Free Portrait Matting in Real Time

MODNet is a model for real-time portrait matting with only RGB image input.
MODNet是一个仅需RGB图片输入实时人像抠图模型。

Online Solution (在线方案) | Research Demo | Arxiv Preprint | Supplementary Video

Community | Code | PPM Benchmark | License | Acknowledgement | Citation | Contact

News: We create a repository for our new model MODNet-V that focuses on faster and better portrait video matting.
News: The PPM-100 benchmark is released in this repository.


Online Solution (在线方案)

The online solution for portrait matting is coming!
人像抠图在线方案发布了!

Portrait Image Matting Solution (图片抠像方案)

A Single Model! Only 7M! Process 2K resolution image with a Fast speed on common PCs or Mobiles!
单个模型!大小仅为7M!可以在普通PC或移动设备上快速处理具有2K分辨率的图像!

Now you can try our portrait image matting online via this website.
现在,您可以通过此网站在线使用我们的图片抠像功能。

Research Demo

All the models behind the following demos are trained on the datasets mentioned in our paper.

Portrait Image Matting

We provide an online Colab demo for portrait image matting.
It allows you to upload portrait images and predict/visualize/download the alpha mattes.

Portrait Video Matting

We provide two real-time portrait video matting demos based on WebCam. When using the demo, you can move the WebCam around at will. If you have an Ubuntu system, we recommend you to try the offline demo to get a higher fps. Otherwise, you can access the online Colab demo.
We also provide an offline demo that allows you to process custom videos.

Community

We share some cool applications/extentions of MODNet built by the community.

  • WebGUI for Portrait Image Matting
    You can try this WebGUI (hosted on Gradio) for portrait image matting from your browser without code!

  • Colab Demo of Bokeh (Blur Background)
    You can try this Colab demo (built by @eyaler) to blur the backgroud based on MODNet!

  • ONNX Version of MODNet
    You can convert the pre-trained MODNet to an ONNX model by using this code (provided by @manthan3C273). You can also try this Colab demo for MODNet image matting (ONNX version).

  • TorchScript Version of MODNet
    You can convert the pre-trained MODNet to an TorchScript model by using this code (provided by @yarkable).

  • TensorRT Version of MODNet
    You can access this Github repository to try the TensorRT version of MODNet (provided by @jkjung-avt).

There are some resources about MODNet from the community.

Code

We provide the code of MODNet training iteration, including:

  • Supervised Training: Train MODNet on a labeled matting dataset
  • SOC Adaptation: Adapt a trained MODNet to an unlabeled dataset

In the code comments, we provide examples for using the functions.

PPM Benchmark

The PPM benchmark is released in a separate repository PPM.

License

All resources in this repository (code, models, demos, etc.) are released under the Creative Commons Attribution NonCommercial ShareAlike 4.0 license.
The license will be changed to allow commercial use after our paper is accepted.

Acknowledgement

Citation

If this work helps your research, please consider to cite:

@article{MODNet,
  author = {Zhanghan Ke and Kaican Li and Yurou Zhou and Qiuhua Wu and Xiangyu Mao and Qiong Yan and Rynson W.H. Lau},
  title = {Is a Green Screen Really Necessary for Real-Time Portrait Matting?},
  journal={ArXiv},
  volume={abs/2011.11961},
  year = {2020},
}

Contact

This repository is currently maintained by Zhanghan Ke (@ZHKKKe).
For questions, please contact [email protected].

Owner
Zhanghan Ke
PhD Candidate @ CityU
Zhanghan Ke
Running AlphaFold2 (from ColabFold) in Azure Machine Learning

Running AlphaFold2 (from ColabFold) in Azure Machine Learning Colby T. Ford, Ph.D. Companion repository for Medium Post: How to predict many protein s

Colby T. Ford 3 Feb 18, 2022
Replication of Pix2Seq with Pretrained Model

Pretrained-Pix2Seq We provide the pre-trained model of Pix2Seq. This version contains new data augmentation. The model is trained for 300 epochs and c

peng gao 51 Nov 22, 2022
DVG-Face: Dual Variational Generation for Heterogeneous Face Recognition, TPAMI 2021

DVG-Face: Dual Variational Generation for HFR This repo is a PyTorch implementation of DVG-Face: Dual Variational Generation for Heterogeneous Face Re

52 Dec 30, 2022
A modular PyTorch library for optical flow estimation using neural networks

A modular PyTorch library for optical flow estimation using neural networks

neu-vig 113 Dec 20, 2022
DL course co-developed by YSDA, HSE and Skoltech

Deep learning course This repo supplements Deep Learning course taught at YSDA and HSE @fall'21. For previous iteration visit the spring21 branch. Lec

Yandex School of Data Analysis 1.3k Dec 30, 2022
Spline is a tool that is capable of running locally as well as part of well known pipelines like Jenkins (Jenkinsfile), Travis CI (.travis.yml) or similar ones.

Welcome to spline - the pipeline tool Important note: Since change in my job I didn't had the chance to continue on this project. My main new project

Thomas Lehmann 29 Aug 22, 2022
This is my research project for the Irving Center for Cancer Dynamics/Azizi Lab, Columbia University.

bayesian_uncertainty This is my research project for the Irving Center for Cancer Dynamics/Azizi Lab, Columbia University. In this project I build a s

Max David Gupta 1 Feb 13, 2022
Code to reproduce the experiments in the paper "Transformer Based Multi-Source Domain Adaptation" (EMNLP 2020)

Transformer Based Multi-Source Domain Adaptation Dustin Wright and Isabelle Augenstein To appear in EMNLP 2020. Read the preprint: https://arxiv.org/a

CopeNLU 36 Dec 05, 2022
Source codes for the paper "Local Additivity Based Data Augmentation for Semi-supervised NER"

LADA This repo contains codes for the following paper: Jiaao Chen*, Zhenghui Wang*, Ran Tian, Zichao Yang, Diyi Yang: Local Additivity Based Data Augm

GT-SALT 36 Dec 02, 2022
A project that uses optical flow and machine learning to detect aimhacking in video clips.

waldo-anticheat A project that aims to use optical flow and machine learning to visually detect cheating or hacking in video clips from fps games. Che

waldo.vision 542 Dec 03, 2022
Code to replicate the key results from Exploring the Limits of Out-of-Distribution Detection

Exploring the Limits of Out-of-Distribution Detection In this repository we're collecting replications for the key experiments in the Exploring the Li

Stanislav Fort 35 Jan 03, 2023
This is the official repository of XVFI (eXtreme Video Frame Interpolation)

XVFI This is the official repository of XVFI (eXtreme Video Frame Interpolation), https://arxiv.org/abs/2103.16206 Last Update: 20210607 We provide th

Jihyong Oh 195 Dec 29, 2022
A Simulated Optimal Intrusion Response Game

Optimal Intrusion Response An OpenAI Gym interface to a MDP/Markov Game model for optimal intrusion response of a realistic infrastructure simulated u

Kim Hammar 10 Dec 09, 2022
Pytorch Performace Tuning, WandB, AMP, Multi-GPU, TensorRT, Triton

Plant Pathology 2020 FGVC7 Introduction A deep learning model pipeline for training, experimentaiton and deployment for the Kaggle Competition, Plant

Bharat Giddwani 0 Feb 25, 2022
Efficiently computes derivatives of numpy code.

Note: Autograd is still being maintained but is no longer actively developed. The main developers (Dougal Maclaurin, David Duvenaud, Matt Johnson, and

Formerly: Harvard Intelligent Probabilistic Systems Group -- Now at Princeton 6.1k Jan 08, 2023
Individual Tree Crown classification on WorldView-2 Images using Autoencoder -- Group 9 Weak learners - Final Project (Machine Learning 2020 Course)

Created by Olga Sutyrina, Sarah Elemili, Abduragim Shtanchaev and Artur Bille Individual Tree Crown classification on WorldView-2 Images using Autoenc

2 Dec 08, 2022
PointCNN: Convolution On X-Transformed Points (NeurIPS 2018)

PointCNN: Convolution On X-Transformed Points Created by Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, and Baoquan Chen. Introduction PointCNN

Yangyan Li 1.3k Dec 21, 2022
Deep GPs built on top of TensorFlow/Keras and GPflow

GPflux Documentation | Tutorials | API reference | Slack What does GPflux do? GPflux is a toolbox dedicated to Deep Gaussian processes (DGP), the hier

Secondmind Labs 107 Nov 02, 2022
Hard cater examples from Hopper ICLR paper

CATER-h Honglu Zhou*, Asim Kadav, Farley Lai, Alexandru Niculescu-Mizil, Martin Renqiang Min, Mubbasir Kapadia, Hans Peter Graf (*Contact: honglu.zhou

NECLA ML Group 6 May 11, 2021
Python implementation of cover trees, near-drop-in replacement for scipy.spatial.kdtree

This is a Python implementation of cover trees, a data structure for finding nearest neighbors in a general metric space (e.g., a 3D box with periodic

Patrick Varilly 28 Nov 25, 2022