A PaddlePaddle version of Neural Renderer, refer to its PyTorch version

Last update: Jul 12, 2022

Overview

Neural 3D Mesh Renderer in PadddlePaddle

A PaddlePaddle version of Neural Renderer, refer to its PyTorch version

Install

Run:

pip install neural-renderer-paddle

Usage

Check examples folder for usage.

Note

Unittest module is not compatiable with PaddlePaddle. If you want to run test, run:

git clone <THIS REPO> neural_renderer_paddle
pip install neural_renderer_paddle/
cd neural_renderer_paddle/tests
python -m unittest test_*.py

Original repo README:

Neural 3D Mesh Renderer (CVPR 2018)

This repo contains a PyTorch implementation of the paper Neural 3D Mesh Renderer by Hiroharu Kato, Yoshitaka Ushiku, and Tatsuya Harada. It is a port of the original Chainer implementation released by the authors. Currently the API is the same as in the original implementation with some smalls additions (e.g. render using a general 3x4 camera matrix, lens distortion coefficients etc.). However it is possible that it will change in the future.

The library is fully functional and it passes all the test cases supplied by the authors of the original library. Detailed documentation will be added in the near future.

Requirements

Python 2.7+ and PyTorch 0.4.0.

The code has been tested only with PyTorch 0.4.0, there are no guarantees that it is compatible with older versions. Currently the library has both Python 3 and Python 2 support.

Note: In some newer PyTorch versions you might see some compilation errors involving AT_ASSERT. In these cases you can use the version of the code that is in the branch at_assert_fix. These changes will be merged into master in the near future.

Installation

You can install the package by running

pip install neural_renderer_pytorch

Since running install.py requires PyTorch, make sure to install PyTorch before running the above command.

Running examples

python ./examples/example1.py
python ./examples/example2.py
python ./examples/example3.py
python ./examples/example4.py

Example 1: Drawing an object from multiple viewpoints

Example 2: Optimizing vertices

Transforming the silhouette of a teapot into a rectangle. The loss function is the difference between the rendered image and the reference image.

Reference image, optimization, and the result.

Example 3: Optimizing textures

Matching the color of a teapot with a reference image.

Reference image, result.

Example 4: Finding camera parameters

The derivative of images with respect to camera pose can be computed through this renderer. In this example the position of the camera is optimized by gradient descent.

From left to right: reference image, initial state, and optimization process.

Citation

@InProceedings{kato2018renderer
    title={Neural 3D Mesh Renderer},
    author={Kato, Hiroharu and Ushiku, Yoshitaka and Harada, Tatsuya},
    booktitle={The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year={2018}
}

You might also like...

Object detection and instance segmentation toolkit based on PaddlePaddle.

A PaddlePaddle version of Neural Renderer, refer to its PyTorch version

Related tags

Overview

Neural 3D Mesh Renderer in PadddlePaddle

Install

Usage

Note

Neural 3D Mesh Renderer (CVPR 2018)

Requirements

Installation

Running examples

Example 1: Drawing an object from multiple viewpoints

Example 2: Optimizing vertices

Example 3: Optimizing textures

Example 4: Finding camera parameters

Citation

You might also like...

Object detection and instance segmentation toolkit based on PaddlePaddle.

Paddle-Adversarial-Toolbox (PAT) is a Python library for Deep Learning Security based on PaddlePaddle.

Plaything for Autistic Children (demo for PaddlePaddle/Wechaty/Mixlab project)

Official PaddlePaddle implementation of Paint Transformer

PaddleViT: State-of-the-art Visual Transformer and MLP Models for PaddlePaddle 2.0+

Remote sensing change detection tool based on PaddlePaddle

End-to-end image segmentation kit based on PaddlePaddle.

Large-scale open domain KNOwledge grounded conVERsation system based on PaddlePaddle

Classical OCR DCNN reproduction based on PaddlePaddle framework.

Releases(1.1.3)

1.1.3(Dec 9, 2021)

Owner

AgentMaker

High performance, easy-to-use, and scalable machine learning (ML) package, including linear model (LR), factorization machines (FM), and field-aware factorization machines (FFM) for Python and CLI interface.

Soft actor-critic is a deep reinforcement learning framework for training maximum entropy policies in continuous domains.

Code and data of the ACL 2021 paper: Few-Shot Text Ranking with Meta Adapted Synthetic Weak Supervision

In this project we investigate the performance of the SetCon model on realistic video footage. Therefore, we implemented the model in PyTorch and tested the model on two example videos.

一个多语言支持、易使用的 OCR 项目。An easy-to-use OCR project with multilingual support.

Semi-supervised semantic segmentation needs strong, varied perturbations

Research Artifact of USENIX Security 2022 Paper: Automated Side Channel Analysis of Media Software with Manifold Learning

DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generative Transformers

Breaking Shortcut: Exploring Fully Convolutional Cycle-Consistency for Video Correspondence Learning

An implementation of Geoffrey Hinton's paper "How to represent part-whole hierarchies in a neural network" in Pytorch.

Virtual hand gesture mouse using a webcam

Deeprl - Standard DQN and dueling network for simple games

official code for dynamic convolution decomposition

WRENCH: Weak supeRvision bENCHmark

Code for the paper "Functional Regularization for Reinforcement Learning via Learned Fourier Features"

DeepCAD: A Deep Generative Network for Computer-Aided Design Models

The repo for the paper "I3CL: Intra- and Inter-Instance Collaborative Learning for Arbitrary-shaped Scene Text Detection".

Code for "Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networks", CVPR 2021

Music source separation is a task to separate audio recordings into individual sources

A Peer-to-peer Platform for Secure, Privacy-preserving, Decentralized Data Science