A Collection of LiDAR-Camera-Calibration Papers, Toolboxes and Notes

Overview

Awesome-LiDAR-Camera-Calibration

Awesome

A Collection of LiDAR-Camera-Calibration Papers, Toolboxes and Notes.

Outline

0. Introduction

For applications such as autonomous driving, robotics, navigation systems, and 3-D scene reconstruction, data of the same scene is often captured using both lidar and camera sensors. To accurately interpret the objects in a scene, it is necessary to fuse the lidar and the camera outputs together. Lidar camera calibration estimates a rigid transformation matrix (extrinsics, rotation+translation, 6 DoF) that establishes the correspondences between the points in the 3-D lidar plane and the pixels in the image plane.

Example

1. Target-based methods

Paper Target Feature Optimization Toolbox Note
Extrinsic Calibration of a Camera and Laser Range Finder (improves camera calibration), 2004 checkerboard C:Plane (a), L: pts in plane (m) point-to-plane CamLaserCalibraTool CN
Fast Extrinsic Calibration of a Laser Rangefinder to a Camera, 2005 checkerboard C: Plane (a), L: Plane (m) plane(n/d) correspondence, point-to-plane LCCT *
Extrinsic calibration of a 3D laser scanner and an omnidirectional camera, 2010 checkerboard C: plane (a), L: pts in plane (m) point-to-plane cam_lidar_calib *
LiDAR-Camera Calibration using 3D-3D Point correspondences, 2017 cardboard + ArUco C: 3D corners (a), L: 3D corners (m) ICP lidar_camera_calibration *
Reflectance Intensity Assisted Automatic and Accurate Extrinsic Calibration of 3D LiDAR and Panoramic Camera Using a Printed Chessboard, 2017 checkerboard C: 2D corners (a), L: 3D corners (a) PnP, angle difference ILCC *
Extrinsic Calibration of Lidar and Camera with Polygon, 2018 regular cardboard C: 2D edge, corners (a), L: 3D edge, pts in plane (a) point-to-line, point-inside-polygon ram-lab/plycal *
Automatic Extrinsic Calibration of a Camera and a 3D LiDAR using Line and Plane Correspondences, 2018 checkerboard C: 3D edge, plane(a), L: 3D edge, pts in plane (a) direcion/normal, point-to-line, point-to-plane Matlab LiDAR Toolbox *
Improvements to Target-Based 3D LiDAR to Camera Calibration, 2020 cardboard with ArUco C: 2d corners (a), L: 3D corners (a) PnP, IOU github *
ACSC: Automatic Calibration for Non-repetitive Scanning Solid-State LiDAR and Camera Systems, 2020 checkerboard C: 2D corners (a), L: 3D corners (a) PnP ACSC *
Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups, 2021 cardboard with circle & Aruco C: 3D points (a), L: 3D points (a) ICP velo2cam_ calibration *

C: camera, L: LiDAR, a: automaic, m: manual

2. Targetless methods

2.1. Motion-based methods

Paper Feature Optimization Toolbox Note
LiDAR and Camera Calibration Using Motions Estimated by Sensor Fusion Odometry, 2018 C: motion (ICP), L: motion (VO) hand-eye calibration * *

2.2. Scene-based methods

2.2.1. Traditional methods

Paper Feature Optimization Toolbox Note
Automatic Targetless Extrinsic Calibration of a 3D Lidar and Camera by Maximizing Mutual Information, 2012 C:grayscale, L: reflectivity mutual information, BB steepest gradient ascent Extrinsic Calib *
Automatic Calibration of Lidar and Camera Images using Normalized Mutual Information, 2013 C:grayscale, L: reflectivity, noraml normalized MI, particle swarm * *
Automatic Online Calibration of Cameras and Lasers, 2013 C: Canny edge, L: depth-discontinuous edge correlation, grid search * *
SOIC: Semantic Online Initialization and Calibration for LiDAR and Camera, 2020 semantic centroid PnP * *
A Low-cost and Accurate Lidar-assisted Visual SLAM System, 2021 C: edge(grayscale), L: edge (reflectivity, depth projection) ICP, coordinate descent algorithms CamVox *
Pixel-level Extrinsic Self Calibration of High Resolution LiDAR and Camera in Targetless Environments,2021 C:Canny edge(grayscale), L: depth-continuous edge point-to-line, Gaussian-Newton livox_camera_calib *
CRLF: Automatic Calibration and Refinement based on Line Feature for LiDAR and Camera in Road Scenes, 2021 C:straight line, L: straight line perspective3-lines (P3L) * CN

2.2.2. Deep-learning methods

Pape Toolbox Note
RegNet: Multimodal sensor registration using deep neural networks, 2017,IV regnet *
CalibNet: Geometrically supervised extrinsic calibration using 3d spatial transformer networks,2018,IROS CalibNet *

3. Other toolboxes

Toolbox Introduction Note
Apollo sensor calibration tools targetless method, no source code CN
Autoware camera lidar calibrator pick points mannually, PnP *
Autoware calibration camera lidar checkerboard, similar to LCCT CN
livox_camera_lidar_calibration pick points mannually, PnP *
(CVPR 2021) Back-tracing Representative Points for Voting-based 3D Object Detection in Point Clouds

BRNet Introduction This is a release of the code of our paper Back-tracing Representative Points for Voting-based 3D Object Detection in Point Clouds,

86 Oct 05, 2022
A PyTorch implementation of "From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network" (ICCV2021)

From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network The official code of VisionLAN (ICCV2021). VisionLAN successfully a

81 Dec 12, 2022
ComputerVision - This repository aims at realized easy network architecture

ComputerVision This repository aims at realized easy network architecture Colori

DongDong 4 Dec 14, 2022
PyTorch implementation of the ideas presented in the paper Interaction Grounded Learning (IGL)

Interaction Grounded Learning This repository contains a simple PyTorch implementation of the ideas presented in the paper Interaction Grounded Learni

Arthur Juliani 4 Aug 31, 2022
Nodule Generation Algorithm Baseline and template code for node21 generation track

Nodule Generation Algorithm This codebase implements a simple baseline model, by following the main steps in the paper published by Litjens et al. for

node21challenge 10 Apr 21, 2022
[AAAI22] Reliable Propagation-Correction Modulation for Video Object Segmentation

Reliable Propagation-Correction Modulation for Video Object Segmentation (AAAI22) Preview version paper of this work is available at: https://arxiv.or

Xiaohao Xu 70 Dec 04, 2022
[ICML 2020] DrRepair: Learning to Repair Programs from Error Messages

DrRepair: Learning to Repair Programs from Error Messages This repo provides the source code & data of our paper: Graph-based, Self-Supervised Program

Michihiro Yasunaga 155 Jan 08, 2023
MAg: a simple learning-based patient-level aggregation method for detecting microsatellite instability from whole-slide images

MAg Paper Abstract File structure Dataset prepare Data description How to use MAg? Why not try the MAg_lib! Trained models Experiment and results Some

Calvin Pang 3 Apr 08, 2022
Mahadi-Now - This Is Pakistani Just Now Login Tools

PAKISTANI JUST NOW LOGIN TOOLS Install apt update apt upgrade apt install python

MAHADI HASAN AFRIDI 19 Apr 06, 2022
An open-source outlier detection package by Getcontact Data Team

pyfbad The pyfbad library supports anomaly detection projects. An end-to-end anomaly detection application can be written using the source codes of th

Teknasyon Tech 41 Dec 27, 2022
Learning Efficient Online 3D Bin Packing on Packing Configuration Trees

Learning Efficient Online 3D Bin Packing on Packing Configuration Trees This repository is being continuously updated, please stay tuned! Any code con

86 Dec 28, 2022
Template repository to build PyTorch projects from source on any version of PyTorch/CUDA/cuDNN.

The Ultimate PyTorch Source-Build Template Translations: 한국어 TL;DR PyTorch built from source can be x4 faster than a naïve PyTorch install. This repos

Joonhyung Lee/이준형 651 Dec 12, 2022
Repository for training material for the 2022 SDSC HPC/CI User Training Course

hpc-training-2022 Repository for training material for the 2022 SDSC HPC/CI Training Series HPC/CI Training Series home https://www.sdsc.edu/event_ite

sdsc-hpc-training-org 21 Jul 27, 2022
A deep neural networks for images using CNN algorithm.

Example-CNN-Project This is a simple project showing how to implement deep neural networks using CNN algorithm. The dataset is taken from this link: h

Mohammad Amin Dadgar 3 Sep 16, 2022
Optimal Camera Position for a Practical Application of Gaze Estimation on Edge Devices,

Optimal Camera Position for a Practical Application of Gaze Estimation on Edge Devices, Linh Van Ma, Tin Trung Tran, Moongu Jeon, ICAIIC 2022 (The 4th

Linh 11 Oct 10, 2022
🤗 Transformers: State-of-the-art Natural Language Processing for Pytorch, TensorFlow, and JAX.

English | 简体中文 | 繁體中文 | 한국어 State-of-the-art Natural Language Processing for Jax, PyTorch and TensorFlow 🤗 Transformers provides thousands of pretrai

Hugging Face 77.4k Jan 05, 2023
Code for CVPR 2018 paper --- Texture Mapping for 3D Reconstruction with RGB-D Sensor

G2LTex This repository contains the implementation of "Texture Mapping for 3D Reconstruction with RGB-D Sensor (CVPR2018)" based on mvs-texturing. Due

Fu Yanping(付燕平) 129 Dec 30, 2022
PyTorch implementation of Pointnet2/Pointnet++

Pointnet2/Pointnet++ PyTorch Project Status: Unmaintained. Due to finite time, I have no plans to update this code and I will not be responding to iss

Erik Wijmans 1.2k Dec 29, 2022
This repository contains answers of the Shopify Summer 2022 Data Science Intern Challenge.

Data-Science-Intern-Challenge This repository contains answers of the Shopify Summer 2022 Data Science Intern Challenge. Summer 2022 Data Science Inte

1 Jan 11, 2022
This is a model made out of Neural Network specifically a Convolutional Neural Network model

This is a model made out of Neural Network specifically a Convolutional Neural Network model. This was done with a pre-built dataset from the tensorflow and keras packages. There are other alternativ

9 Oct 18, 2022