Vis2Mesh: Efficient Mesh Reconstruction from Unstructured Point Clouds of Large Scenes with Learned Virtual View Visibility ICCV2021

Last update: Dec 25, 2022

Related tags

Deep Learning vis2mesh

Overview

Vis2Mesh

This is the offical repository of the paper:

Vis2Mesh: Efficient Mesh Reconstruction from Unstructured Point Clouds of Large Scenes with Learned Virtual View Visibility

https://arxiv.org/abs/2108.08378

@misc{song2021vis2mesh,
      title={Vis2Mesh: Efficient Mesh Reconstruction from Unstructured Point Clouds of Large Scenes with Learned Virtual View Visibility}, 
      author={Shuang Song and Zhaopeng Cui and Rongjun Qin},
      year={2021},
      eprint={2108.08378},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Updates

2021/9/6: Intialize all in one project. Only this version only supports inferencing with our pre-trained weights. We will release Dockerfile to relief deploy efforts.

TODO

Ground truth generation and network training.
Evaluation scripts

Build With Docker (Recommended)

Install nvidia-docker2

# Add the package repositories
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list

sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
sudo systemctl restart docker

Build docker image

docker build . -t vis2mesh

Build on Ubuntu

Please create a conda environment with pytorch and check out our setup script:

./setup_tools.sh

Usage

Get pretrained weights and examples

pip install gdown
./checkpoints/get_pretrained.sh
./example/get_example.sh

Run example

The main command for surface reconstruction, the result will be copied as $(CLOUDFILE)_vis2mesh.ply.

python inference.py example/example1.ply --cam cam0

We suggested to use docker, either in interactive mode or single shot mode.

xhost +
name=vis2mesh
# Run in interactive mode
docker run -it \
--mount type=bind,source="$PWD/checkpoints",target=/workspace/checkpoints \
--mount type=bind,source="$PWD/example",target=/workspace/example \
--privileged \
--net=host \
-e NVIDIA_DRIVER_CAPABILITIES=all \
-e DISPLAY=unix$DISPLAY \
-v $XAUTH:/root/.Xauthority \
-v /tmp/.X11-unix:/tmp/.X11-unix:rw \
--device=/dev/dri \
--gpus all $name

cd /workspace
python inference.py example/example1.ply --cam cam0

# Run with single shot call
docker run \
--mount type=bind,source="$PWD/checkpoints",target=/workspace/checkpoints \
--mount type=bind,source="$PWD/example",target=/workspace/example \
--privileged \
--net=host \
-e NVIDIA_DRIVER_CAPABILITIES=all \
-e DISPLAY=unix$DISPLAY \
-v $XAUTH:/root/.Xauthority \
-v /tmp/.X11-unix:/tmp/.X11-unix:rw \
--device=/dev/dri \
--gpus all $name \
/workspace/inference.py example/example1.ply --cam cam0

Run with Customize Views

python inference.py example/example1.ply Run the command without --cam flag, you can add virtual views interactively with the following GUI. Your views will be recorded in example/example1.ply_WORK/cam*.json.

Navigate in 3D viewer and click key [Space] to record current view. Click key [Q] to close the window and continue meshing process.

Vis2Mesh: Efficient Mesh Reconstruction from Unstructured Point Clouds of Large Scenes with Learned Virtual View Visibility ICCV2021

Related tags

Overview

Vis2Mesh

Updates

TODO

Build With Docker (Recommended)

Install nvidia-docker2

Build docker image

Build on Ubuntu

Usage

Get pretrained weights and examples

Run example

Run with Customize Views

Owner

Unofficial implementation of Google "CutPaste: Self-Supervised Learning for Anomaly Detection and Localization" in PyTorch

Few-Shot Graph Learning for Molecular Property Prediction

LLVM-based compiler for LightGBM gradient-boosted trees. Speeds up prediction by ≥10x.

🚗 INGI Dakar 2K21 - Be the first one on the finish line ! 🚗

Open source implementation of AceNAS: Learning to Rank Ace Neural Architectures with Weak Supervision of Weight Sharing

Keepsake is a Python library that uploads files and metadata (like hyperparameters) to Amazon S3 or Google Cloud Storage

DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort

StyleTransfer - Open source style transfer project, based on VGG19

An implementation of the methods presented in Causal-BALD: Deep Bayesian Active Learning of Outcomes to Infer Treatment-Effects from Observational Data.

This project hosts the code for implementing the ISAL algorithm for object detection and image classification

Unoffical implementation about Image Super-Resolution via Iterative Refinement by Pytorch

Code for Emergent Translation in Multi-Agent Communication

[Arxiv preprint] Causality-inspired Single-source Domain Generalization for Medical Image Segmentation (code&data-processing pipeline)

Classify bird species based on their songs using SIamese Networks and 1D dilated convolutions.

A machine learning project which can detect and predict the skin disease through image recognition.

Evaluating Cross-lingual Sentence Representations

Official code of ICCV2021 paper "Residual Attention: A Simple but Effective Method for Multi-Label Recognition"

Speech Enhancement Generative Adversarial Network Based on Asymmetric AutoEncoder

CondNet: Conditional Classifier for Scene Segmentation

Collection of machine learning related notebooks to share.