This is the code related to "Sparse-to-dense Feature Matching: Intra and Inter domain Cross-modal Learning in Domain Adaptation for 3D Semantic Segmentation" (ICCV 2021).

Related tags

Deep LearningDsCML
Overview

Sparse-to-dense Feature Matching: Intra and Inter domain Cross-modal Learning in Domain Adaptation for 3D Semantic Segmentation

This is the code related to "Sparse-to-dense Feature Matching: Intra and Inter domain Cross-modal Learning in Domain Adaptation for 3D Semantic Segmentation" (ICCV 2021).

1. Paper

Sparse-to-dense Feature Matching: Intra and Inter domain Cross-modal Learning in Domain Adaptation for 3D Semantic Segmentation
IEEE International Conference on Computer Vision (ICCV 2021)

If you find it helpful to your research, please cite as follows:

@inproceedings{peng2021sparse,
  title={Sparse-to-dense Feature Matching: Intra and Inter domain Cross-modal Learning in Domain Adaptation for 3D Semantic Segmentation},
  author={Peng, Duo and Lei, Yinjie and Li, Wen and Zhang, Pingping and Guo, Yulan},
  booktitle={Proceedings of the International Conference on Computer Vision (ICCV)},
  year={2021},
  publisher={IEEE}
}

2. Preparation

You can follow the next steps to install the requairmented environment. This code is mainly modified from xMUDA, you can also refer to its README if the installation isn't going well.

2.1 Setup a Conda environment:

First, you are recommended to create a new Conda environment named nuscenes.

conda create --name nuscenes python=3.7

You can enable the virtual environment using:

conda activate nuscenes 

To deactivate the virtual environment, use:

source deactivate

2.2 Install nuscenes-devkit:

Download the devkit to your computer, decompress and enter it.

Add the python-sdk directory to your PYTHONPATH environmental variable, by adding the following to your ~/.bashrc:

export PYTHONPATH="${PYTHONPATH}:$HOME/nuscenes-devkit/python-sdk"

Using cmd (make sure the environment "nuscenes" is activated) to install the base environment:

pip install -r setup/requirements.txt

Setup environment variable:

export NUSCENES="/data/sets/nuscenes"

Using the cmd to finally install it:

pip install nuscenes-devkit

After the above steps, the devikit is installed, for any question you can refer to devikit_installation_help

If you meet the error with "pycocotools", you can try following steps:

(1) Install Cython in your environment:

sudo apt-get installl Cython
pip install cython

(2) Download the cocoapi to your computer, decompress and enter it.

(3) Using cmd to enter the path under "PythonAPI", type:

make

(4) Type:

pip install pycocotools

2.3 Install SparseConveNet:

Download the SparseConveNet to your computer, decompress, enter and develop it:

cd SparseConvNet/
bash develop.sh

3. Datasets Preparation

For Dataset preprocessing, the code and steps are highly borrowed from xMUDA, you can see more preprocessing details from this Link. We summarize the preprocessing as follows:

3.1 NuScenes

Download Nuscenes from NuScenes website and extract it.

Before training, you need to perform preprocessing to generate the data first. Please edit the script DsCML/data/nuscenes/preprocess.py as follows and then run it.

root_dir should point to the root directory of the NuScenes dataset

out_dir should point to the desired output directory to store the pickle files

3.2 A2D2

Download the A2D2 Semantic Segmentation dataset and Sensor Configuration from the Audi website

Similar to NuScenes preprocessing, please save all points that project into the front camera image as well as the segmentation labels to a pickle file.

Please edit the script DsCML/data/a2d2/preprocess.py as follows and then run it.

root_dir should point to the root directory of the A2D2 dataset

out_dir should point to the desired output directory to store the undistorted images and pickle files.

It should be set differently than the root_dir to prevent overwriting of images.

3.3 SemanticKITTI

Download the files from the SemanticKITTI website and additionally the color data from the Kitti Odometry website. Extract everything into the same folder.

Please edit the script DsCML/data/semantic_kitti/preprocess.py as follows and then run it.

root_dir should point to the root directory of the SemanticKITTI dataset out_dir should point to the desired output directory to store the pickle files

4. Usage

You can training the DsCML by using cmd or IDE such as Pycharm.

python DsCML/train_DsCML.py --cfg=../configs/nuscenes/day_night/xmuda.yaml

The output will be written to /home/<user>/workspace by default. You can change the path OUTPUT_DIR in the config file in (e.g. configs/nuscenes/day_night/xmuda.yaml)

You can start the trainings on the other UDA scenarios (USA/Singapore and A2D2/SemanticKITTI):

python DsCML/train_DsCML.py --cfg=../configs/nuscenes/usa_singapore/xmuda.yaml
python DsCML/train_DsCML.py --cfg=../configs/a2d2_semantic_kitti/xmuda.yaml

5. Results

We present several qualitative results reported in our paper.

Update Status

The code of CMAL is updated. (2021-10-04)

Predict stock movement with Machine Learning and Deep Learning algorithms

Project Overview Stock market movement prediction using LSTM Deep Neural Networks and machine learning algorithms Software and Library Requirements Th

Naz Delam 46 Sep 13, 2022
Multi-Stage Spatial-Temporal Convolutional Neural Network (MS-GCN)

Multi-Stage Spatial-Temporal Convolutional Neural Network (MS-GCN) This code implements the skeleton-based action segmentation MS-GCN model from Autom

Benjamin Filtjens 8 Nov 29, 2022
GDSC-ML Team Interview Task

GDSC-ML-Team---Interview-Task Task 1 : Clean or Messy room In this task we have to classify the given test images as clean or messy. - Link for datase

Aayush. 1 Jan 19, 2022
Validated, scalable, community developed variant calling, RNA-seq and small RNA analysis

Validated, scalable, community developed variant calling, RNA-seq and small RNA analysis. You write a high level configuration file specifying your in

Blue Collar Bioinformatics 917 Jan 03, 2023
PyTorch implementation of the supervised learning experiments from the paper Model-Agnostic Meta-Learning (MAML)

pytorch-maml This is a PyTorch implementation of the supervised learning experiments from the paper Model-Agnostic Meta-Learning (MAML): https://arxiv

Kate Rakelly 516 Jan 05, 2023
Deep Learning Datasets Maker is a QGIS plugin to make datasets creation easier for raster and vector data.

Deep Learning Dataset Maker Deep Learning Datasets Maker is a QGIS plugin to make datasets creation easier for raster and vector data. How to use Down

deepbands 25 Dec 15, 2022
Inference pipeline for our participation in the FeTA challenge 2021.

feta-inference Inference pipeline for our participation in the FeTA challenge 2021. Team name: TRABIT Installation Download the two folders in https:/

Lucas Fidon 2 Apr 13, 2022
2021 credit card consuming recommendation

2021 credit card consuming recommendation

Wang, Chung-Che 7 Mar 08, 2022
SpinalNet: Deep Neural Network with Gradual Input

SpinalNet: Deep Neural Network with Gradual Input This repository contains scripts for training different variations of the SpinalNet and its counterp

H M Dipu Kabir 142 Dec 30, 2022
This is the research repository for Vid2Doppler: Synthesizing Doppler Radar Data from Videos for Training Privacy-Preserving Activity Recognition.

Vid2Doppler: Synthesizing Doppler Radar Data from Videos for Training Privacy-Preserving Activity Recognition This is the research repository for Vid2

Future Interfaces Group (CMU) 26 Dec 24, 2022
Quick program made to generate alpha and delta tables for Hidden Markov Models

HMM_Calc Functions for generating Alpha and Delta tables from a Hidden Markov Model. Parameters: a: Matrix of transition probabilities. a[i][j] = a_{i

Adem Odza 1 Dec 04, 2021
A modular application for performing anomaly detection in networks

Deep-Learning-Models-for-Network-Annomaly-Detection The modular app consists for mainly three annomaly detection algorithms. The system supports model

Shivam Patel 1 Dec 09, 2021
PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud, CVPR 2019.

PointRCNN PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud Code release for the paper PointRCNN:3D Object Proposal Generation a

Shaoshuai Shi 1.5k Dec 27, 2022
CARLA: A Python Library to Benchmark Algorithmic Recourse and Counterfactual Explanation Algorithms

CARLA - Counterfactual And Recourse Library CARLA is a python library to benchmark counterfactual explanation and recourse models. It comes out-of-the

Carla Recourse 200 Dec 28, 2022
Explaining neural decisions contrastively to alternative decisions.

Contrastive Explanations for Model Interpretability This is the repository for the paper "Contrastive Explanations for Model Interpretability", about

AI2 16 Oct 16, 2022
This repository contains the source codes for the paper AtlasNet V2 - Learning Elementary Structures.

AtlasNet V2 - Learning Elementary Structures This work was build upon Thibault Groueix's AtlasNet and 3D-CODED projects. (you might want to have a loo

Théo Deprelle 123 Nov 11, 2022
[Preprint] ConvMLP: Hierarchical Convolutional MLPs for Vision, 2021

Convolutional MLP ConvMLP: Hierarchical Convolutional MLPs for Vision Preprint link: ConvMLP: Hierarchical Convolutional MLPs for Vision By Jiachen Li

SHI Lab 143 Jan 03, 2023
Official PyTorch implementation of Spatial Dependency Networks.

Spatial Dependency Networks: Neural Layers for Improved Generative Image Modeling Đorđe Miladinović   Aleksandar Stanić   Stefan Bauer   Jürgen Schmid

Djordje Miladinovic 34 Jan 19, 2022
Video lie detector using xgboost - A video lie detector using OpenFace and xgboost

video_lie_detector_using_xgboost a video lie detector using OpenFace and xgboost

2 Jan 11, 2022
Crowd-sourced Annotation of Human Motion.

Motion Annotation Tool Live: https://motion-annotation.humanoids.kit.edu Paper: The KIT Motion-Language Dataset Installation Start by installing all P

Matthias Plappert 4 May 25, 2020