Struct-MDC

(click the above buttons for redirection!)

Official page of "Struct-MDC: Mesh-Refined Unsupervised Depth Completion Leveraging Structural Regularities from Visual SLAM", which is accepted in IEEE RA-L'22 (IROS'22 are still being under-reviewed.)

• Depth completion from Visual(-inertial) SLAM using point & line features.

README & code & Dataset are still being edited.

• Code (including source code, utility code for visualization) & Dataset will be finalized & released soon! (goal: I'm still organizing the code structure, until publish date)
• version info
  • (04/20) docker image has been uploaded.
  • (04/21) Dataset has been uploaded.
  • (04/21) Visusal-SLAM module (modified UV-SLAM) has been uploaded.

Results

• 3D Depth estimation results
  • VOID (left three columns) and NYUv2 (right three columns)
  • detected features (top row), estimation from baseline (middle row) and ours (bottom row)

• 2D Depth estimation results
  • NYUv2

Ground truth	Baseline	Struct-MDC (Ours)

Installation

1. Prerequisites (we've validated our code in the following environment!)

• Common
  • Ubuntu 18.04
  • ROS melodic
• Visual-SLAM module
  • OpenCV 3.2.0 (under 3.4.1)
  • Ceres Solver-1.14.0
  • Eigen-3.3.9
  • CDT library

    git clone https://github.com/artem-ogre/CDT.git
    cd CDT
    mkdir build && cd build
    cmake -DCDT_USE_AS_COMPILED_LIBRARY=ON -DCDT_USE_BOOST=ON ..
    cmake --build . && cmake --install .
    sudo make install
    

• Depth completion module
  • Python 3.7.7
  • PyTorch 1.5.0 (you can easily reproduce equivalent environment using our docker image)

2. Build

• Visual-SLAM module

  • As visual-SLAM, we modified the UV-SLAM, which is implemented in ROS environment.
  • make sure that your catkin workspace has following cmake args: -DCMAKE_BUILD_TYPE=Release

  cd ~/$(PATH_TO_YOUR_ROS_WORKSPACE)/src
  git clone --recursive https://github.com/url-kaist/Struct-MDC
  cd ..
  catkin build
  source ~/$(PATH_TO_YOUR_ROS_WORKSPACE)/devel/setup.bash
  

• Depth completion module

  • Our depth compeltion module is based on the popular Deep-Learning framework, PyTorch.
  • For your convenience, we share our environment as Docker image. We assume that you have already installed the Docker. For Docker installation, please refer here

  # pull our docker image into your local machine
  docker pull zinuok/nvidia-torch:latest
  
  # run the image mounting our source
  docker run -it --gpus "device=0" -v $(PATH_TO_YOUR_LOCAL_FOLER):/workspace zinuok/nvidia-torch:latest bash
  

3. Trouble shooting

• any issues found will be updated in this section.
• if you've found any other issues, please post it on Issues tab. We'll do our best to resolve your issues.