AirCode: A Robust Object Encoding Method

Overview

AirCode

This repo contains source codes for the arXiv preprint "AirCode: A Robust Object Encoding Method"

Demo

Object matching comparison when the objects are non-rigid and the view is changed, left is the result of our method while right is the result of NetVLAD

Relocalization on KITTI datasets

Dependencies

  • Python
  • PyTorch
  • OpenCV
  • Matplotlib
  • NumPy
  • Yaml

Data

Four datasets are used in our experiments.

KITTI Odometry

For relocalization experiment. Three sequences are selected, and they are "00", "05" and "06".

KITTI Tracking

For multi-object matching experiment. Four sequences are selected, and they are "0002", "0003", "0006", "0010".

VOT Datasets

For single-object matching experiment. We select three sequences from VOT2019 datasets and they are "bluecar", "bus6" and "humans_corridor_occ_2_A", because the tracked objects in these sequences are included in coco datasets, which are the data we used to train mask-rcnn.

OTB Datasets

For single-object matching experiment. We select five sequences and they are "BlurBody", "BlurCar2", "Human2", "Human7" and "Liquor".

Examples

Relocalization on KITTI Datasets

  1. Extract object descrptors

    python experiments/place_recogination/online_relocalization.py -c config/experiment_tracking.yaml -g 1 -s PATH_TO_SAVE_MIDDLE_RESULTS -d PATH_TO_DATASET -m PATH_TO_MODELS
    
  2. Compute precision-recall curves

    python experiments/place_recogination/offline_process.py -c config/experiment_tracking.yaml -g 1 -d PATH_TO_DATASET -n PATH_TO_MIDDLE_RESULTS -s PATH_TO_SAVE_RESULTS
    
  3. Compute top-K relocalization results

    python experiments/place_recogination/offline_topK.py -c config/experiment_tracking.yaml -g 1 -d PATH_TO_DATASET -n PATH_TO_MIDDLE_RESULTS -s PATH_TO_SAVE_RESULTS
    

Object Matching on OTB, VOT or KITTI Tracking Datasets

  • Run multi-object matching experiment in KITTI Tracking Datasets Modify the config file and run

    python experiments/object_tracking/object_tracking.py -c config/experiment_tracking.yaml -g 1 -s PATH_TO_SAVE_RESULTS -d PATH_TO_DATASET -m PATH_TO_MODELS 
    
  • Run single-object matching experiment in OTB or VOT Datasets Modify the config file and run

    python experiments/object_tracking/single_object_tracking.py -c config/experiment_tracking.yaml -g 1 -s PATH_TO_SAVE_RESULTS -d PATH_TO_DATASET -m PATH_TO_MODELS 
    
You might also like...
PyTorch implementation of Rethinking Positional Encoding in Language Pre-training
PyTorch implementation of Rethinking Positional Encoding in Language Pre-training

TUPE PyTorch implementation of Rethinking Positional Encoding in Language Pre-training. Quickstart Clone this repository. git clone https://github.com

Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding (CVPR2022)
Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding (CVPR2022)

Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding by Qiaole Dong*, Chenjie Cao*, Yanwei Fu Paper and Supple

A Robust Non-IoU Alternative to Non-Maxima Suppression in Object Detection
A Robust Non-IoU Alternative to Non-Maxima Suppression in Object Detection

Confluence: A Robust Non-IoU Alternative to Non-Maxima Suppression in Object Detection 1. 介绍 用以替代 NMS,在所有 bbox 中挑选出最优的集合。 NMS 仅考虑了 bbox 的得分,然后根据 IOU 来

[ECCVW2020] Robust Long-Term Object Tracking via Improved Discriminative Model Prediction (RLT-DiMP)
[ECCVW2020] Robust Long-Term Object Tracking via Improved Discriminative Model Prediction (RLT-DiMP)

Feel free to visit my homepage Robust Long-Term Object Tracking via Improved Discriminative Model Prediction (RLT-DIMP) [ECCVW2020 paper] Presentation

 Robust Instance Segmentation through Reasoning about Multi-Object Occlusion [CVPR 2021]
Robust Instance Segmentation through Reasoning about Multi-Object Occlusion [CVPR 2021]

Robust Instance Segmentation through Reasoning about Multi-Object Occlusion [CVPR 2021] Abstract Analyzing complex scenes with DNN is a challenging ta

Code release for our paper,
Code release for our paper, "SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo"

SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo Thomas Kollar, Michael Laskey, Kevin Stone, Brijen Thananjeyan

object detection; robust detection; ACM MM21 grand challenge; Security AI Challenger Phase VII
object detection; robust detection; ACM MM21 grand challenge; Security AI Challenger Phase VII

赛题背景 在商品知识产权领域,知识产权体现为在线商品的设计和品牌。不幸的是,在每一天,存在着非法商户通过一些对抗手段干扰商标识别来逃避侵权,这带来了很高的知识产权风险和财务损失。为了促进先进的多媒体人工智能技术的发展,以保护企业来之不易的创作和想法免受恶意使用和剽窃,因此提出了鲁棒性标识检测挑战赛

Code and models for ICCV2021 paper
Code and models for ICCV2021 paper "Robust Object Detection via Instance-Level Temporal Cycle Confusion".

Robust Object Detection via Instance-Level Temporal Cycle Confusion This repo contains the implementation of the ICCV 2021 paper, Robust Object Detect

Official code for 'Robust Siamese Object Tracking for Unmanned Aerial Manipulator' and offical introduction to UAMT100 benchmark
Official code for 'Robust Siamese Object Tracking for Unmanned Aerial Manipulator' and offical introduction to UAMT100 benchmark

SiamSA: Robust Siamese Object Tracking for Unmanned Aerial Manipulator Demo video 📹 Our video on Youtube and bilibili demonstrates the evaluation of

Comments
  • how can I get *.pth files?

    how can I get *.pth files?

    Hello, I am a beginner. When I run python experiments/place_recogination/online_relocalization.py -c config/experiment_tracking.yaml -g 1 -s results/ -d /media/jixingwu/datasetj/KITTI/Odom/data_odometry_color/sequences -m models/, points_model.pth file is needed. So how can I get it? Thank you!

    opened by jixingwu 5
  • Unable to load model under CPU-only configuration

    Unable to load model under CPU-only configuration

    Hi, I want to run object tracking on KITTI tracking datasets with only CPU using the following terminal prompt:

      python experiments/object_tracking/object_tracking.py -c config/experiment_tracking.yaml -g 1 -s ./results -d /data/datasets/SLAM_dataset/training/ -m ./weights
    

    with configuration in object_tracking.py updated with

    configs['use_gpu'] = 0
    

    However, when running with the configuration above with gcn_model.pth, maskrcnn_model.pth, points_model.pth model files in release v2.0.0, the following error occurs:

    (aircode) [email protected]:~/workspace/AirCode$ python experiments/object_tracking/object_tracking.py -c config/experiment_tracking.yaml -g 1 -s ./results -d /data/datasets/SLAM_dataset/training/ -m ./weights
    experiments/object_tracking/object_tracking.py:371: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
      configs = yaml.load(configs)
    Traceback (most recent call last):
      File "experiments/object_tracking/object_tracking.py", line 384, in <module>
        main()
      File "experiments/object_tracking/object_tracking.py", line 381, in main
        show_object_tracking(configs)
      File "experiments/object_tracking/object_tracking.py", line 272, in show_object_tracking
        superpoint_model = build_superpoint_model(configs, requires_grad=False)
      File "./model/build_model.py", line 101, in build_superpoint_model
        model.load_state_dict(model_dict)
      File "/home/yutianc/minicondas/envs/aircode/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1052, in load_state_dict
        self.__class__.__name__, "\n\t".join(error_msgs)))
    RuntimeError: Error(s) in loading state_dict for VggLike:
            Unexpected key(s) in state_dict: "module.pretrained_net.features.0.weight", "module.pretrained_net.features.0.bias", "module.pretrained_net.features.2.weight", "module.pretrained_net.features.2.bias", "module.pretrained_net.features.5.weight", "module.pretrained_net.features.5.bias", "module.pretrained_net.features.7.weight", "module.pretrained_net.features.7.bias", "module.pretrained_net.features.10.weight", "module.pretrained_net.features.10.bias", "module.pretrained_net.features.12.weight", "module.pretrained_net.features.12.bias", "module.pretrained_net.features.14.weight", "module.pretrained_net.features.14.bias", "module.pretrained_net.features.17.weight", "module.pretrained_net.features.17.bias", "module.pretrained_net.features.19.weight", "module.pretrained_net.features.19.bias", "module.pretrained_net.features.21.weight", "module.pretrained_net.features.21.bias", "module.pretrained_net.features.24.weight", "module.pretrained_net.features.24.bias", "module.pretrained_net.features.26.weight", "module.pretrained_net.features.26.bias", "module.pretrained_net.features.28.weight", "module.pretrained_net.features.28.bias", "module.convPa.weight", "module.convPa.bias", "module.bnPa.weight", "module.bnPa.bias", "module.bnPa.running_mean", "module.bnPa.running_var", "module.bnPa.num_batches_tracked", "module.convPb.weight", "module.convPb.bias", "module.bnPb.weight", "module.bnPb.bias", "module.bnPb.running_mean", "module.bnPb.running_var", "module.bnPb.num_batches_tracked", "module.convDa.weight", "module.convDa.bias", "module.bnDa.weight", "module.bnDa.bias", "module.bnDa.running_mean", "module.bnDa.running_var", "module.bnDa.num_batches_tracked", "module.convDb.weight", "module.convDb.bias", "module.bnDb.weight", "module.bnDb.bias", "module.bnDb.running_mean", "module.bnDb.running_var", "module.bnDb.num_batches_tracked".
    

    Running object_tracking.py with CUDA seems to load models successfully. Is there something wrong with the model loading when GPU is disabled?

    opened by MarkChenYutian 4
  • Why RGB image is converted into grayscale image with 3 channels?

    Why RGB image is converted into grayscale image with 3 channels?

    Hi, I'm trying to use AirCode to do object matching on complete KITTI sequences and I'm reading the code in experiments/show_object_matching.py.

    While reading the code, I noticed that the current code is reading RGB image sequence, convert it into grayscale image, and then duplicate the image into 3-channel each with same value (as following):

    https://github.com/wang-chen/AirCode/blob/5e23e9f5322d2e4ee119d5326a6b6112cef0e6bd/experiments/show_object_matching/show_object_matching.py#L172-L176

    I'm a bit unsure about the reason why this operation is performed here as the original RGB image should contain more information about the object comparing to grayscale image. For instance, it should be easier to distinguish objects with different color but similar shape if the RGB value is preserved.

    opened by MarkChenYutian 2
Owner
Chen Wang
I am engaged in delivering simple and efficient source code.
Chen Wang
JugLab 33 Dec 30, 2022
This code is an unofficial implementation of HiFiSinger.

HiFiSinger This code is an unofficial implementation of HiFiSinger. The algorithm is based on the following papers: Chen, J., Tan, X., Luan, J., Qin,

Heejo You 87 Dec 23, 2022
Free-duolingo-plus - Duolingo account creator that uses your invite code to get you free duolingo plus

free-duolingo-plus duolingo account creator that uses your invite code to get yo

1 Jan 06, 2022
A very short and easy implementation of Quantile Regression DQN

Quantile Regression DQN Quantile Regression DQN a Minimal Working Example, Distributional Reinforcement Learning with Quantile Regression (https://arx

Arsenii Senya Ashukha 80 Sep 17, 2022
✨✨✨An awesome open source toolbox for stereo matching.

OpenStereo This is an awesome open source toolbox for stereo matching. Supported Methods: BM SGM(T-PAMI'07) GCNet(ICCV'17) PSMNet(CVPR'18) StereoNet(E

Wang Qingyu 6 Nov 04, 2022
Graph-total-spanning-trees - A Python script to get total number of Spanning Trees in a Graph

Total number of Spanning Trees in a Graph This is a python script just written f

Mehdi I. 0 Jul 18, 2022
Deep Learning agent of Starcraft2, similar to AlphaStar of DeepMind except size of network.

Introduction This repository is for Deep Learning agent of Starcraft2. It is very similar to AlphaStar of DeepMind except size of network. I only test

Dohyeong Kim 136 Jan 04, 2023
CIFAR-10_train-test - training and testing codes for dataset CIFAR-10

CIFAR-10_train-test - training and testing codes for dataset CIFAR-10

Frederick Wang 3 Apr 26, 2022
Learning 3D Part Assembly from a Single Image

Learning 3D Part Assembly from a Single Image This repository contains a PyTorch implementation of the paper: Learning 3D Part Assembly from A Single

18 Dec 21, 2022
Joint deep network for feature line detection and description

SOLD² - Self-supervised Occlusion-aware Line Description and Detection This repository contains the implementation of the paper: SOLD² : Self-supervis

Computer Vision and Geometry Lab 427 Dec 27, 2022
Train emoji embeddings based on emoji descriptions.

emoji2vec This is my attempt to train, visualize and evaluate emoji embeddings as presented by Ben Eisner, Tim Rocktäschel, Isabelle Augenstein, Matko

Miruna Pislar 17 Sep 03, 2022
This is a vision-based 3d model manipulation and control UI

Manipulation of 3D Models Using Hand Gesture This program allows user to manipulation 3D models (.obj format) with their hands. The project support bo

Cortic Technology Corp. 43 Oct 23, 2022
Generating Radiology Reports via Memory-driven Transformer

R2Gen This is the implementation of Generating Radiology Reports via Memory-driven Transformer at EMNLP-2020. Citations If you use or extend our work,

CUHK-SZ NLP Group 101 Dec 13, 2022
Tensorflow implementation of DeepLabv2

TF-deeplab This is a Tensorflow implementation of DeepLab, compatible with Tensorflow 1.2.1. Currently it supports both training and testing the ResNe

Chenxi Liu 21 Sep 27, 2022
A task Provided by A respective Artenal Ai and Ml based Company to complete it

A task Provided by A respective Alternal Ai and Ml based Company to complete it .

Parth Madan 1 Jan 25, 2022
details on efforts to dump the Watermelon Games Paprium cart

Reminder, if you like these repos, fork them so they don't disappear https://github.com/ArcadeHustle/WatermelonPapriumDump/fork Big thanks to Fonzie f

Hustle Arcade 29 Dec 11, 2022
A minimal TPU compatible Jax implementation of NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis

NeRF Minimal Jax implementation of NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. Result of Tiny-NeRF RGB Depth

Soumik Rakshit 11 Jul 24, 2022
Implementation for our ICCV 2021 paper: Dual-Camera Super-Resolution with Aligned Attention Modules

DCSR: Dual Camera Super-Resolution Implementation for our ICCV 2021 oral paper: Dual-Camera Super-Resolution with Aligned Attention Modules paper | pr

Tengfei Wang 110 Dec 20, 2022
An adaptive hierarchical energy management strategy for hybrid electric vehicles

An adaptive hierarchical energy management strategy This project contains the source code of an adaptive hierarchical EMS combining heuristic equivale

19 Dec 13, 2022
This is the pytorch re-implementation of the IterNorm

IterNorm-pytorch Pytorch reimplementation of the IterNorm methods, which is described in the following paper: Iterative Normalization: Beyond Standard

Lei Huang 32 Dec 27, 2022