This is the code related to "Sparse-to-dense Feature Matching: Intra and Inter domain Cross-modal Learning in Domain Adaptation for 3D Semantic Segmentation" (ICCV 2021).

Related tags

Deep LearningDsCML
Overview

Sparse-to-dense Feature Matching: Intra and Inter domain Cross-modal Learning in Domain Adaptation for 3D Semantic Segmentation

This is the code related to "Sparse-to-dense Feature Matching: Intra and Inter domain Cross-modal Learning in Domain Adaptation for 3D Semantic Segmentation" (ICCV 2021).

1. Paper

Sparse-to-dense Feature Matching: Intra and Inter domain Cross-modal Learning in Domain Adaptation for 3D Semantic Segmentation
IEEE International Conference on Computer Vision (ICCV 2021)

If you find it helpful to your research, please cite as follows:

@inproceedings{peng2021sparse,
  title={Sparse-to-dense Feature Matching: Intra and Inter domain Cross-modal Learning in Domain Adaptation for 3D Semantic Segmentation},
  author={Peng, Duo and Lei, Yinjie and Li, Wen and Zhang, Pingping and Guo, Yulan},
  booktitle={Proceedings of the International Conference on Computer Vision (ICCV)},
  year={2021},
  publisher={IEEE}
}

2. Preparation

You can follow the next steps to install the requairmented environment. This code is mainly modified from xMUDA, you can also refer to its README if the installation isn't going well.

2.1 Setup a Conda environment:

First, you are recommended to create a new Conda environment named nuscenes.

conda create --name nuscenes python=3.7

You can enable the virtual environment using:

conda activate nuscenes 

To deactivate the virtual environment, use:

source deactivate

2.2 Install nuscenes-devkit:

Download the devkit to your computer, decompress and enter it.

Add the python-sdk directory to your PYTHONPATH environmental variable, by adding the following to your ~/.bashrc:

export PYTHONPATH="${PYTHONPATH}:$HOME/nuscenes-devkit/python-sdk"

Using cmd (make sure the environment "nuscenes" is activated) to install the base environment:

pip install -r setup/requirements.txt

Setup environment variable:

export NUSCENES="/data/sets/nuscenes"

Using the cmd to finally install it:

pip install nuscenes-devkit

After the above steps, the devikit is installed, for any question you can refer to devikit_installation_help

If you meet the error with "pycocotools", you can try following steps:

(1) Install Cython in your environment:

sudo apt-get installl Cython
pip install cython

(2) Download the cocoapi to your computer, decompress and enter it.

(3) Using cmd to enter the path under "PythonAPI", type:

make

(4) Type:

pip install pycocotools

2.3 Install SparseConveNet:

Download the SparseConveNet to your computer, decompress, enter and develop it:

cd SparseConvNet/
bash develop.sh

3. Datasets Preparation

For Dataset preprocessing, the code and steps are highly borrowed from xMUDA, you can see more preprocessing details from this Link. We summarize the preprocessing as follows:

3.1 NuScenes

Download Nuscenes from NuScenes website and extract it.

Before training, you need to perform preprocessing to generate the data first. Please edit the script DsCML/data/nuscenes/preprocess.py as follows and then run it.

root_dir should point to the root directory of the NuScenes dataset

out_dir should point to the desired output directory to store the pickle files

3.2 A2D2

Download the A2D2 Semantic Segmentation dataset and Sensor Configuration from the Audi website

Similar to NuScenes preprocessing, please save all points that project into the front camera image as well as the segmentation labels to a pickle file.

Please edit the script DsCML/data/a2d2/preprocess.py as follows and then run it.

root_dir should point to the root directory of the A2D2 dataset

out_dir should point to the desired output directory to store the undistorted images and pickle files.

It should be set differently than the root_dir to prevent overwriting of images.

3.3 SemanticKITTI

Download the files from the SemanticKITTI website and additionally the color data from the Kitti Odometry website. Extract everything into the same folder.

Please edit the script DsCML/data/semantic_kitti/preprocess.py as follows and then run it.

root_dir should point to the root directory of the SemanticKITTI dataset out_dir should point to the desired output directory to store the pickle files

4. Usage

You can training the DsCML by using cmd or IDE such as Pycharm.

python DsCML/train_DsCML.py --cfg=../configs/nuscenes/day_night/xmuda.yaml

The output will be written to /home/<user>/workspace by default. You can change the path OUTPUT_DIR in the config file in (e.g. configs/nuscenes/day_night/xmuda.yaml)

You can start the trainings on the other UDA scenarios (USA/Singapore and A2D2/SemanticKITTI):

python DsCML/train_DsCML.py --cfg=../configs/nuscenes/usa_singapore/xmuda.yaml
python DsCML/train_DsCML.py --cfg=../configs/a2d2_semantic_kitti/xmuda.yaml

5. Results

We present several qualitative results reported in our paper.

Update Status

The code of CMAL is updated. (2021-10-04)

Deep Learning for Time Series Forecasting.

nixtlats:Deep Learning for Time Series Forecasting [nikstla] (noun, nahuatl) Period of time. State-of-the-art time series forecasting for pytorch. Nix

Nixtla 5 Dec 06, 2022
PyTorch implementation of Neural Combinatorial Optimization with Reinforcement Learning.

neural-combinatorial-rl-pytorch PyTorch implementation of Neural Combinatorial Optimization with Reinforcement Learning. I have implemented the basic

Patrick E. 454 Jan 06, 2023
Training Cifar-10 Classifier Using VGG16

opevcvdl-hw3 This project uses pytorch and Qt to achieve the requirements. Version Python 3.6 opencv-contrib-python 3.4.2.17 Matplotlib 3.1.1 pyqt5 5.

Kenny Cheng 3 Aug 17, 2022
Fake News Detection Using Machine Learning Methods

Fake-News-Detection-Using-Machine-Learning-Methods Fake news is always a real and dangerous issue. However, with the presence and abundance of various

Achraf Safsafi 1 Jan 11, 2022
NeurIPS-2021: Neural Auto-Curricula in Two-Player Zero-Sum Games.

NAC Official PyTorch implementation of NAC from the paper: Neural Auto-Curricula in Two-Player Zero-Sum Games. We release code for: Gradient based ora

Xidong Feng 19 Nov 11, 2022
Predicting Student Attentiveness using OpenCV

Predicting-Student-Attentiveness-using-OpenCV The model will predict if a student is attentive or not through facial parameter received through the st

Johann Pinto 2 Aug 20, 2022
CMP 414/765 course repository for Spring 2022 semester

CMP414/765: Artificial Intelligence Spring2021 This is the GitHub repository for course CMP 414/765: Artificial Intelligence taught at The City Univer

ch00226855 4 May 16, 2022
A Moonraker plug-in for real-time compensation of frame thermal expansion

Frame Expansion Compensation A Moonraker plug-in for real-time compensation of frame thermal expansion. Installation Credit to protoloft, from whom I

58 Jan 02, 2023
Discerning Decision-Making Process of Deep Neural Networks with Hierarchical Voting Transformation

Configurations Change HOME_PATH in CONFIG.py as the current path Data Prepare CENSINCOME Download data Put census-income.data and census-income.test i

2 Aug 14, 2022
Numenta Platform for Intelligent Computing is an implementation of Hierarchical Temporal Memory (HTM), a theory of intelligence based strictly on the neuroscience of the neocortex.

NuPIC Numenta Platform for Intelligent Computing The Numenta Platform for Intelligent Computing (NuPIC) is a machine intelligence platform that implem

Numenta 6.3k Dec 30, 2022
Gesture-controlled Video Game. Just swing your finger and play the game without touching your PC

Gesture Controlled Video Game Detailed Blog : https://www.analyticsvidhya.com/blog/2021/06/gesture-controlled-video-game/ Introduction This project is

Devbrat Anuragi 35 Jan 06, 2023
Implementation of Hierarchical Transformer Memory (HTM) for Pytorch

Hierarchical Transformer Memory (HTM) - Pytorch Implementation of Hierarchical Transformer Memory (HTM) for Pytorch. This Deepmind paper proposes a si

Phil Wang 63 Dec 29, 2022
Selective Wavelet Attention Learning for Single Image Deraining

SWAL Code for Paper "Selective Wavelet Attention Learning for Single Image Deraining" Prerequisites Python 3 PyTorch Models We provide the models trai

Bobo 9 Jun 17, 2022
A PyTorch implementation of DenseNet.

A PyTorch Implementation of DenseNet This is a PyTorch implementation of the DenseNet-BC architecture as described in the paper Densely Connected Conv

Brandon Amos 771 Dec 15, 2022
The official implementation of NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation [ICLR-2021]. https://arxiv.org/pdf/2101.12378.pdf

NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation [ICLR-2021] Release Notes The offical PyTorch implementation of NeMo, p

Angtian Wang 76 Nov 23, 2022
Check out the StyleGAN repo and place it in the same directory hierarchy as the present repo

Variational Model Inversion Attacks Kuan-Chieh Wang, Yan Fu, Ke Li, Ashish Khisti, Richard Zemel, Alireza Makhzani Most commands are in run_scripts. W

Jackson Wang 15 Dec 26, 2022
A project that uses optical flow and machine learning to detect aimhacking in video clips.

waldo-anticheat A project that aims to use optical flow and machine learning to visually detect cheating or hacking in video clips from fps games. Che

waldo.vision 542 Dec 03, 2022
The code for our NeurIPS 2021 paper "Kernelized Heterogeneous Risk Minimization".

Kernelized-HRM Jiashuo Liu, Zheyuan Hu The code for our NeurIPS 2021 paper "Kernelized Heterogeneous Risk Minimization"[1]. This repo contains the cod

Liu Jiashuo 8 Nov 20, 2022
Rainbow DQN implementation that outperforms the paper's results on 40% of games using 20x less data 🌈

Rainbow 🌈 An implementation of Rainbow DQN which reaches a median HNS of 205.7 after only 10M frames (the original Rainbow from Hessel et al. 2017 re

Dominik Schmidt 31 Dec 21, 2022