An executor that wraps 3D mesh models and encodes 3D content documents to d-dimension vector.

Last update: Dec 14, 2022

Related tags

Overview

3D Mesh Encoder

An Executor that receives Documents containing point sets data in its blob attribute, with shape (N, 3) and encodes it to embeddings of shape (D,). Now, the following pretrained models are ready to be used to create embeddings:

PointConv-Shapenet-d512: A PointConv model resulted in 512 dimension of embeddings, which is finetuned based on ShapeNet dataset.
PointConv-Shapenet-d1024: A PointConv model resulted in 1024 dimension of embeddings, which is finetuned based on ShapeNet dataset.

Usage

via Docker image (recommended)

from jina import Flow

f = Flow().add(uses='jinahub+docker://3DMeshEncoder', \
               uses_with={'pretrained_model': 'PointConv-Shapenet-d512'})

via source code

from jina import Flow

f = Flow().add(uses='jinahub://3DMeshEncoder', \
               uses_with={'pretrained_model': 'PointConv-Shapenet-d512'})

This Executor offers a GPU tag to speed up encoding. For more information on how to run the executor on GPU, check out the documentation.

How to finetune pretrained-model?

install finetuner

$ pip install finetuner

prepare dataset

TBD...

finetuning model with labeled dataset

$ python finetune.py --help

$ python finetune.py --model_name pointconv \
    --train_dataset /path/to/train.bin \
    --eval_dataset /path/to/eval.bin \
    --batch_size 128 \
    --epochs 50

finetuning model with unlabeled dataset

$ python finetune.py --model_name pointconv \
    --train_dataset /path/to/unlabeled_data.bin \
    --interactive

References

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
PointConv: Deep Convolutional Networks on 3D Point Clouds

Comments

which version should the `finetuner` module be?
I was preparing the environment of this repo. However, I found it confused that the python import code in finetune.py gives error message:

from finetuner.tuner.callback import BestModelCheckpoint from finetuner.tuner.pytorch.losses import TripletLoss from finetuner.tuner.pytorch.miner import TripletEasyHardMiner

It seems that there's no finetuner.tuner in finetuner module but finetuner.finetuner. Besides, callback/pytorch is not in finetuner.finetuner now.

Now my finetuner module installed by running pip install -U finetuner is 0.5.0. Is there any solution? Maybe you can freeze the version in requirements.txt.
opened by elenacliu 4
fix: fix previous structure

I got pretrain and finetune process mixed up, and previously I pretrained the models using finetune model structure, which might lead to worse results. The fix code is in process and may be finished in a few days.

opened by elenacliu 2
Add dataset, preprocess and pointnet2

Here is the modification of the structure. There are still some bugs in my pointnet2 codes. I will solve the possible problems after you comment.

@numb3r3 @jemmyshin

opened by elenacliu 1
In `farthest_point_sample` function, the first centroid sampled is not random.

I'm reading your codes.

https://github.com/jina-ai/executor-3d-encoder/blob/main/executor/models/modules.py#L75

The randint line is commented, but in the original implementation and the algorithm itself, it should be random? Maybe we can provide a fixed seed if we want to get the same result?

opened by elenacliu 1
Provide diffusion encoder

https://github.com/nmwsharp/diffusion-net

I'm unfamiliar with the jina ecosystem. So this is my first issue.

Final goal is to have dalle generate 3d digital human meshes like https://github.com/jina-ai/dalle-flow

opened by fire 2

Releases(v0.3.0)

v0.3.0(May 6, 2022)

Source code(tar.gz)
Source code(zip)
v0.2.0(Feb 21, 2022)

Source code(tar.gz)
Source code(zip)

Owner

Jina AI

A Neural Search Company. We provide the cloud-native neural search solution powered by state-of-the-art AI technology.

GitHub Repository

Simple program to easily view Euler parameters in 3D.

5 Aug 20, 2021

Make your master artistic punk avatar through machine learning world famous paintings

Master-art-punk Make your master artistic punk avatar through machine learning world famous paintings. 通过机器学习世界名画制作属于你的大师级艺术朋克头像 Nowadays, NFT is beco

23 Jan 04, 2022

A ray tracing render implemented using Taichi language.

45 Oct 23, 2022

Simple mathematical operations on image, point and surface layers.

napari-math This package provides a GUI interfrace for simple mathematical operations on image, point and surface layers. addition subtraction multipl

2 Jan 18, 2022

QR fixer part is standalone but for image to FQR conversion

f-qr-fixer QR fixer part is standalone but for image to FQR conversion it requires Pillow (can be installed with easy_install), qrtools (on ubuntu the

2 Nov 22, 2022

Python implementation of image filters (such as brightness, contrast, saturation, etc.)

PyPhotoshop Python implementation of image filters Use Python to adjust brightness and contrast, add blur, and detect edges! Follow along tutorial: ht

87 Dec 15, 2022

A simple programme for converting url into a qr code (.png file)

QrTk A simple lightweight programme for converting url into a qr code (.png file) Pre-Requisites Before installing the programme , you need to run the

4 Nov 08, 2021

Advance Image Steganography

Chaya Advance Image Steganography Using LSB-LPS + AES-256-GCM + FLIF Right To Privacy! United Nations Declaration of Human Rights (UDHR) 1948, Article

113 Dec 02, 2022

Convert any image into greyscale ASCII art.

Image-to-ASCII Convert any image into greyscale ASCII art.

12 Jan 15, 2022

A QR Code encode and decode python module

4 Feb 10, 2022

An API that renders HTML/CSS content to PNG using Chromium

html_png An API that renders HTML/CSS content to PNG using Chromium Disclaimer I am not responsible if you happen to make your own instance of this AP

10 Aug 08, 2022

QR Code Generator

In this project, we'll be using some libraries to instantly generate authentic QR Codes and export them in various formats

3 Jun 02, 2022

This will help to read QR codes using Raspberry Pi and Pi Camera

Raspberry-Pi-Generate-and-Read-QR-code This will help to read QR codes using Raspberry Pi and Pi Camera Install the required libraries first in your T

2 Nov 06, 2021

SALaD (Semi-Automatic Landslide Detection) is a landslide mapping system

SALaD (Semi-Automatic Landslide Detection) is a landslide mapping system. SALaD utilizes Object-based Image Analysis and Random Forest to map landslides.

14 Jan 04, 2023

This is an app that allows users to upload photos and display and store the photos in a file until the user deletes them.

Qt Photo App This is an app that allows users to upload photos and display and store the photos in a file until the user deletes them. Setup python3 -

5 Jan 22, 2022