Procedural 3D data generation pipeline for architecture

Overview

Synthetic Dataset Generator

Authors:

This is a tool that generates a dataset of synthetic buildings of different typologies.

Arxiv Website Samples

The generated data includes:

  • Mesh files of generated buildings, .obj format
  • Rendered images of the mesh, .png format
  • Rendered segmentation masks, .png format
  • Depth annotation, .png and .exr format
  • Surface normals annotation, .png format
  • Point cloud files, .ply format (the number of points by default is 2048, can be changed in dataset_config.py)

How To Use

  • Install Blender>=2.90. After installation make sure to add blender as an Environment variable.
  • Download the package as a .zip file or:
git clone https://github.com/CDInstitute/CompoNET

*Navigate to the Building-Dataset-Generator folder.

pip install -r requirements.txt

To create completely synthetic buildings use:

run.bat

Or:

blender setup.blend --python dataset.py

Unfortunately, it is not possible to use Blender in background mode as it will not render the image masks correctly.

Note: all the parameters related to the dataset (including any specific parameters for your buildings (e.g. max and min height / width / length)) are to be provided in dataset_config.py. Default values adhere to international standards (min) and most common European values (max):

  • minimum height 3m
  • minimum length and width 6m
  • maximum length, width, height 30 m Other values to set:
  • number of dataset samples
  • building types
  • component materials
  • rendered image dimensions
  • number of points in the point clouds
  • paths to store the generated data
  • option to save the .exr files

Annotation structure

{'img': 'images/0.png', 'category': 'building', 'img_size': (256, 256), '2d_keypoints': [], 'mask': 'masks/0.png', 'img_source': 'synthetic', 'model': 'models/0.obj', 'point_cloud': 'PointCloud/0.ply', 'model_source': 'synthetic', 'trans_mat': 0, 'focal_length': 35.0, 'cam_position': (0.0, 0.0, 0.0), 'inplane_rotation': 0, 'truncated': False, 'occluded': False, 'slightly_occluded': False, 'bbox': [0.0, 0.0, 0.0, 0.0], 'material': ['concrete', 'brick']}

Performance

We ran the dataset generation algorithm for 100 model samples with different input parameters on Windows 10 OS on CPU and GPU using AMD Ryzen 7 3800-X 8-Core Processor and GeForce GTX 1080. Here we report the results for the multiview generation (3 views per model):

GPU Multiview Time (h)
1.7
2.7
0.34
0.8

Citation

Bibtex format

@inproceedings{fedorova2021synthetic,
      title={Synthetic 3D Data Generation Pipeline for Geometric Deep Learning in Architecture}, 
      author={Stanislava Fedorova and Alberto Tono and Meher Shashwat Nigam and Jiayao Zhang and Amirhossein Ahmadnia and Cecilia Bolognesi and Dominik L. Michels},
      year={2021},
}

Generated Image Samples

Owner
Computational Design Institute
501(c)(3) Research Nonprofit for Digital and Humanities
Computational Design Institute
FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks

FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks Image Classification Dataset: Google Landmark, COCO, ImageNet Model: Efficient

FedML-AI 62 Dec 10, 2022
WaveFake: A Data Set to Facilitate Audio DeepFake Detection

WaveFake: A Data Set to Facilitate Audio DeepFake Detection This is the code repository for our NeurIPS 2021 (Track on Datasets and Benchmarks) paper

Chair for Sys­tems Se­cu­ri­ty 27 Dec 22, 2022
A state-of-the-art semi-supervised method for image recognition

Mean teachers are better role models Paper ---- NIPS 2017 poster ---- NIPS 2017 spotlight slides ---- Blog post By Antti Tarvainen, Harri Valpola (The

Curious AI 1.4k Jan 06, 2023
Photo2cartoon - 人像卡通化探索项目 (photo-to-cartoon translation project)

人像卡通化 (Photo to Cartoon) 中文版 | English Version 该项目为小视科技卡通肖像探索项目。您可使用微信扫描下方二维码或搜索“AI卡通秀”小程序体验卡通化效果。

Minivision_AI 3.5k Dec 30, 2022
Unofficial Implement PU-Transformer

PU-Transformer-pytorch Pytorch unofficial implementation of PU-Transformer (PU-Transformer: Point Cloud Upsampling Transformer) https://arxiv.org/abs/

Lee Hyung Jun 7 Sep 21, 2022
A Lighting Pytorch Framework for Recommendation System, Easy-to-use and Easy-to-extend.

Torch-RecHub A Lighting Pytorch Framework for Recommendation Models, Easy-to-use and Easy-to-extend. 安装 pip install torch-rechub 主要特性 scikit-learn风格易用

Mincai Lai 67 Jan 04, 2023
Revitalizing CNN Attention via Transformers in Self-Supervised Visual Representation Learning

Revitalizing CNN Attention via Transformers in Self-Supervised Visual Representation Learning

ChongjianGE 89 Dec 02, 2022
Context Decoupling Augmentation for Weakly Supervised Semantic Segmentation

Context Decoupling Augmentation for Weakly Supervised Semantic Segmentation The code of: Context Decoupling Augmentation for Weakly Supervised Semanti

54 Dec 12, 2022
ML model to classify between cats and dogs

Cats-and-dogs-classifier This is my first ML model which can classify between cats and dogs. Here the accuracy is around 75%, however , the accuracy c

Sharath V 4 Aug 20, 2021
Model parallel transformers in Jax and Haiku

Mesh Transformer Jax A haiku library using the new(ly documented) xmap operator in Jax for model parallelism of transformers. See enwik8_example.py fo

Ben Wang 4.8k Jan 01, 2023
SCNet: Learning Semantic Correspondence

SCNet Code Region matching code is contributed by Kai Han ([email protected]). Dense

Kai Han 34 Sep 06, 2022
A DNN inference latency prediction toolkit for accurately modeling and predicting the latency on diverse edge devices.

Note: This is an alpha (preview) version which is still under refining. nn-Meter is a novel and efficient system to accurately predict the inference l

Microsoft 244 Jan 06, 2023
Boosted CVaR Classification (NeurIPS 2021)

Boosted CVaR Classification Runtian Zhai, Chen Dan, Arun Sai Suggala, Zico Kolter, Pradeep Ravikumar NeurIPS 2021 Table of Contents Quick Start Train

Runtian Zhai 4 Feb 15, 2022
Implementation of the final project of the course DDA6309 Probabilistic Graphical Model

Task-aware Joint CWS and POS (TCwsPos) This is the implementation of the final project of the course DDA6309 Probabilistic Graphical Models, The Chine

Peng 1 Dec 26, 2021
LoL Runes Recommender With Python

LoL-Runes-Recommender Para ejecutar la aplicación se debe llamar a execute_app.p

Sebastián Salinas 1 Jan 10, 2022
Keras community contributions

keras-contrib : Keras community contributions Keras-contrib is deprecated. Use TensorFlow Addons. The future of Keras-contrib: We're migrating to tens

Keras 1.6k Dec 21, 2022
Code for generating the figures in the paper "Capacity of Group-invariant Linear Readouts from Equivariant Representations: How Many Objects can be Linearly Classified Under All Possible Views?"

Code for running simulations for the paper "Capacity of Group-invariant Linear Readouts from Equivariant Representations: How Many Objects can be Lin

Matthew Farrell 1 Nov 22, 2022
Genetic feature selection module for scikit-learn

sklearn-genetic Genetic feature selection module for scikit-learn Genetic algorithms mimic the process of natural selection to search for optimal valu

Manuel Calzolari 260 Dec 14, 2022
Codebase for INVASE: Instance-wise Variable Selection - 2019 ICLR

Codebase for "INVASE: Instance-wise Variable Selection" Authors: Jinsung Yoon, James Jordon, Mihaela van der Schaar Paper: Jinsung Yoon, James Jordon,

Jinsung Yoon 50 Nov 11, 2022
Face2webtoon - Despite its importance, there are few previous works applying I2I translation to webtoon.

Despite its importance, there are few previous works applying I2I translation to webtoon. I collected dataset from naver webtoon 연애혁명 and tried to transfer human faces to webtoon domain.

이상윤 64 Oct 19, 2022