AirCode: A Robust Object Encoding Method

Overview

AirCode

This repo contains source codes for the arXiv preprint "AirCode: A Robust Object Encoding Method"

Demo

Object matching comparison when the objects are non-rigid and the view is changed, left is the result of our method while right is the result of NetVLAD

Relocalization on KITTI datasets

Dependencies

  • Python
  • PyTorch
  • OpenCV
  • Matplotlib
  • NumPy
  • Yaml

Data

Four datasets are used in our experiments.

KITTI Odometry

For relocalization experiment. Three sequences are selected, and they are "00", "05" and "06".

KITTI Tracking

For multi-object matching experiment. Four sequences are selected, and they are "0002", "0003", "0006", "0010".

VOT Datasets

For single-object matching experiment. We select three sequences from VOT2019 datasets and they are "bluecar", "bus6" and "humans_corridor_occ_2_A", because the tracked objects in these sequences are included in coco datasets, which are the data we used to train mask-rcnn.

OTB Datasets

For single-object matching experiment. We select five sequences and they are "BlurBody", "BlurCar2", "Human2", "Human7" and "Liquor".

Examples

Relocalization on KITTI Datasets

  1. Extract object descrptors

    python experiments/place_recogination/online_relocalization.py -c config/experiment_tracking.yaml -g 1 -s PATH_TO_SAVE_MIDDLE_RESULTS -d PATH_TO_DATASET -m PATH_TO_MODELS
    
  2. Compute precision-recall curves

    python experiments/place_recogination/offline_process.py -c config/experiment_tracking.yaml -g 1 -d PATH_TO_DATASET -n PATH_TO_MIDDLE_RESULTS -s PATH_TO_SAVE_RESULTS
    
  3. Compute top-K relocalization results

    python experiments/place_recogination/offline_topK.py -c config/experiment_tracking.yaml -g 1 -d PATH_TO_DATASET -n PATH_TO_MIDDLE_RESULTS -s PATH_TO_SAVE_RESULTS
    

Object Matching on OTB, VOT or KITTI Tracking Datasets

  • Run multi-object matching experiment in KITTI Tracking Datasets Modify the config file and run

    python experiments/object_tracking/object_tracking.py -c config/experiment_tracking.yaml -g 1 -s PATH_TO_SAVE_RESULTS -d PATH_TO_DATASET -m PATH_TO_MODELS 
    
  • Run single-object matching experiment in OTB or VOT Datasets Modify the config file and run

    python experiments/object_tracking/single_object_tracking.py -c config/experiment_tracking.yaml -g 1 -s PATH_TO_SAVE_RESULTS -d PATH_TO_DATASET -m PATH_TO_MODELS 
    
You might also like...
PyTorch implementation of Rethinking Positional Encoding in Language Pre-training
PyTorch implementation of Rethinking Positional Encoding in Language Pre-training

TUPE PyTorch implementation of Rethinking Positional Encoding in Language Pre-training. Quickstart Clone this repository. git clone https://github.com

Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding (CVPR2022)
Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding (CVPR2022)

Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding by Qiaole Dong*, Chenjie Cao*, Yanwei Fu Paper and Supple

A Robust Non-IoU Alternative to Non-Maxima Suppression in Object Detection
A Robust Non-IoU Alternative to Non-Maxima Suppression in Object Detection

Confluence: A Robust Non-IoU Alternative to Non-Maxima Suppression in Object Detection 1. 介绍 用以替代 NMS,在所有 bbox 中挑选出最优的集合。 NMS 仅考虑了 bbox 的得分,然后根据 IOU 来

[ECCVW2020] Robust Long-Term Object Tracking via Improved Discriminative Model Prediction (RLT-DiMP)
[ECCVW2020] Robust Long-Term Object Tracking via Improved Discriminative Model Prediction (RLT-DiMP)

Feel free to visit my homepage Robust Long-Term Object Tracking via Improved Discriminative Model Prediction (RLT-DIMP) [ECCVW2020 paper] Presentation

 Robust Instance Segmentation through Reasoning about Multi-Object Occlusion [CVPR 2021]
Robust Instance Segmentation through Reasoning about Multi-Object Occlusion [CVPR 2021]

Robust Instance Segmentation through Reasoning about Multi-Object Occlusion [CVPR 2021] Abstract Analyzing complex scenes with DNN is a challenging ta

Code release for our paper,
Code release for our paper, "SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo"

SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo Thomas Kollar, Michael Laskey, Kevin Stone, Brijen Thananjeyan

object detection; robust detection; ACM MM21 grand challenge; Security AI Challenger Phase VII
object detection; robust detection; ACM MM21 grand challenge; Security AI Challenger Phase VII

赛题背景 在商品知识产权领域,知识产权体现为在线商品的设计和品牌。不幸的是,在每一天,存在着非法商户通过一些对抗手段干扰商标识别来逃避侵权,这带来了很高的知识产权风险和财务损失。为了促进先进的多媒体人工智能技术的发展,以保护企业来之不易的创作和想法免受恶意使用和剽窃,因此提出了鲁棒性标识检测挑战赛

Code and models for ICCV2021 paper
Code and models for ICCV2021 paper "Robust Object Detection via Instance-Level Temporal Cycle Confusion".

Robust Object Detection via Instance-Level Temporal Cycle Confusion This repo contains the implementation of the ICCV 2021 paper, Robust Object Detect

Official code for 'Robust Siamese Object Tracking for Unmanned Aerial Manipulator' and offical introduction to UAMT100 benchmark
Official code for 'Robust Siamese Object Tracking for Unmanned Aerial Manipulator' and offical introduction to UAMT100 benchmark

SiamSA: Robust Siamese Object Tracking for Unmanned Aerial Manipulator Demo video 📹 Our video on Youtube and bilibili demonstrates the evaluation of

Comments
  • how can I get *.pth files?

    how can I get *.pth files?

    Hello, I am a beginner. When I run python experiments/place_recogination/online_relocalization.py -c config/experiment_tracking.yaml -g 1 -s results/ -d /media/jixingwu/datasetj/KITTI/Odom/data_odometry_color/sequences -m models/, points_model.pth file is needed. So how can I get it? Thank you!

    opened by jixingwu 5
  • Unable to load model under CPU-only configuration

    Unable to load model under CPU-only configuration

    Hi, I want to run object tracking on KITTI tracking datasets with only CPU using the following terminal prompt:

      python experiments/object_tracking/object_tracking.py -c config/experiment_tracking.yaml -g 1 -s ./results -d /data/datasets/SLAM_dataset/training/ -m ./weights
    

    with configuration in object_tracking.py updated with

    configs['use_gpu'] = 0
    

    However, when running with the configuration above with gcn_model.pth, maskrcnn_model.pth, points_model.pth model files in release v2.0.0, the following error occurs:

    (aircode) [email protected]:~/workspace/AirCode$ python experiments/object_tracking/object_tracking.py -c config/experiment_tracking.yaml -g 1 -s ./results -d /data/datasets/SLAM_dataset/training/ -m ./weights
    experiments/object_tracking/object_tracking.py:371: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
      configs = yaml.load(configs)
    Traceback (most recent call last):
      File "experiments/object_tracking/object_tracking.py", line 384, in <module>
        main()
      File "experiments/object_tracking/object_tracking.py", line 381, in main
        show_object_tracking(configs)
      File "experiments/object_tracking/object_tracking.py", line 272, in show_object_tracking
        superpoint_model = build_superpoint_model(configs, requires_grad=False)
      File "./model/build_model.py", line 101, in build_superpoint_model
        model.load_state_dict(model_dict)
      File "/home/yutianc/minicondas/envs/aircode/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1052, in load_state_dict
        self.__class__.__name__, "\n\t".join(error_msgs)))
    RuntimeError: Error(s) in loading state_dict for VggLike:
            Unexpected key(s) in state_dict: "module.pretrained_net.features.0.weight", "module.pretrained_net.features.0.bias", "module.pretrained_net.features.2.weight", "module.pretrained_net.features.2.bias", "module.pretrained_net.features.5.weight", "module.pretrained_net.features.5.bias", "module.pretrained_net.features.7.weight", "module.pretrained_net.features.7.bias", "module.pretrained_net.features.10.weight", "module.pretrained_net.features.10.bias", "module.pretrained_net.features.12.weight", "module.pretrained_net.features.12.bias", "module.pretrained_net.features.14.weight", "module.pretrained_net.features.14.bias", "module.pretrained_net.features.17.weight", "module.pretrained_net.features.17.bias", "module.pretrained_net.features.19.weight", "module.pretrained_net.features.19.bias", "module.pretrained_net.features.21.weight", "module.pretrained_net.features.21.bias", "module.pretrained_net.features.24.weight", "module.pretrained_net.features.24.bias", "module.pretrained_net.features.26.weight", "module.pretrained_net.features.26.bias", "module.pretrained_net.features.28.weight", "module.pretrained_net.features.28.bias", "module.convPa.weight", "module.convPa.bias", "module.bnPa.weight", "module.bnPa.bias", "module.bnPa.running_mean", "module.bnPa.running_var", "module.bnPa.num_batches_tracked", "module.convPb.weight", "module.convPb.bias", "module.bnPb.weight", "module.bnPb.bias", "module.bnPb.running_mean", "module.bnPb.running_var", "module.bnPb.num_batches_tracked", "module.convDa.weight", "module.convDa.bias", "module.bnDa.weight", "module.bnDa.bias", "module.bnDa.running_mean", "module.bnDa.running_var", "module.bnDa.num_batches_tracked", "module.convDb.weight", "module.convDb.bias", "module.bnDb.weight", "module.bnDb.bias", "module.bnDb.running_mean", "module.bnDb.running_var", "module.bnDb.num_batches_tracked".
    

    Running object_tracking.py with CUDA seems to load models successfully. Is there something wrong with the model loading when GPU is disabled?

    opened by MarkChenYutian 4
  • Why RGB image is converted into grayscale image with 3 channels?

    Why RGB image is converted into grayscale image with 3 channels?

    Hi, I'm trying to use AirCode to do object matching on complete KITTI sequences and I'm reading the code in experiments/show_object_matching.py.

    While reading the code, I noticed that the current code is reading RGB image sequence, convert it into grayscale image, and then duplicate the image into 3-channel each with same value (as following):

    https://github.com/wang-chen/AirCode/blob/5e23e9f5322d2e4ee119d5326a6b6112cef0e6bd/experiments/show_object_matching/show_object_matching.py#L172-L176

    I'm a bit unsure about the reason why this operation is performed here as the original RGB image should contain more information about the object comparing to grayscale image. For instance, it should be easier to distinguish objects with different color but similar shape if the RGB value is preserved.

    opened by MarkChenYutian 2
Owner
Chen Wang
I am engaged in delivering simple and efficient source code.
Chen Wang
EmoTag helps you train emotion detection model for Chinese audios

emoTag emoTag helps you train emotion detection model for Chinese audios. Environment pip install -r requirement.txt Data We used Emotional Speech Dat

_zza 4 Sep 07, 2022
Implementation of "Semi-supervised Domain Adaptive Structure Learning"

Semi-supervised Domain Adaptive Structure Learning - ASDA This repo contains the source code and dataset for our ASDA paper. Illustration of the propo

3 Dec 13, 2021
[AAAI2022] Source code for our paper《Suppressing Static Visual Cues via Normalizing Flows for Self-Supervised Video Representation Learning》

SSVC The source code for paper [Suppressing Static Visual Cues via Normalizing Flows for Self-Supervised Video Representation Learning] samples of the

7 Oct 26, 2022
The codes and models in 'Gaze Estimation using Transformer'.

GazeTR We provide the code of GazeTR-Hybrid in "Gaze Estimation using Transformer". We recommend you to use data processing codes provided in GazeHub.

65 Dec 27, 2022
Official implementation of Densely connected normalizing flows

Densely connected normalizing flows This repository is the official implementation of NeurIPS 2021 paper Densely connected normalizing flows. Poster a

Matej Grcić 31 Dec 12, 2022
Code of paper "Compositionally Generalizable 3D Structure Prediction"

Compositionally Generalizable 3D Structure Prediction In this work, We bring in the concept of compositional generalizability and factorizes the 3D sh

Songfang Han 30 Dec 17, 2022
Make Watson Assistant send messages to your Discord Server

Make Watson Assistant send messages to your Discord Server Prerequisites Sign up for an IBM Cloud account. Fill in the required information and press

1 Jan 10, 2022
Keeper for Ricochet Protocol, implemented with Apache Airflow

Ricochet Keeper This repository contains Apache Airflow DAGs for executing keeper operations for Ricochet Exchange. Usage You will need to run this us

Ricochet Exchange 5 May 24, 2022
MusicYOLO framework uses the object detection model, YOLOx, to locate notes in the spectrogram.

MusicYOLO MusicYOLO framework uses the object detection model, YOLOX, to locate notes in the spectrogram. Its performance on the ISMIR2014 dataset, MI

Xianke Wang 2 Aug 02, 2022
Learnable Multi-level Frequency Decomposition and Hierarchical Attention Mechanism for Generalized Face Presentation Attack Detection

LMFD-PAD Note This is the official repository of the paper: LMFD-PAD: Learnable Multi-level Frequency Decomposition and Hierarchical Attention Mechani

28 Dec 02, 2022
Technical Indicators implemented in Python only using Numpy-Pandas as Magic - Very Very Fast! Very tiny! Stock Market Financial Technical Analysis Python library . Quant Trading automation or cryptocoin exchange

MyTT Technical Indicators implemented in Python only using Numpy-Pandas as Magic - Very Very Fast! to Stock Market Financial Technical Analysis Python

dev 34 Dec 27, 2022
Patch-Based Deep Autoencoder for Point Cloud Geometry Compression

Patch-Based Deep Autoencoder for Point Cloud Geometry Compression Overview The ever-increasing 3D application makes the point cloud compression unprec

17 Dec 05, 2022
[NeurIPS'20] Multiscale Deep Equilibrium Models

Multiscale Deep Equilibrium Models 💥 💥 💥 💥 This repo is deprecated and we will soon stop actively maintaining it, as a more up-to-date (and simple

CMU Locus Lab 221 Dec 26, 2022
This is the repo for the paper "Improving the Accuracy-Memory Trade-Off of Random Forests Via Leaf-Refinement".

Improving the Accuracy-Memory Trade-Off of Random Forests Via Leaf-Refinement This is the repository for the paper "Improving the Accuracy-Memory Trad

3 Dec 29, 2022
Code for Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks Under construction. Description Code for Phase diagram of S

Rodrigo Veiga 3 Nov 24, 2022
The codes I made while I practiced various TensorFlow examples

TensorFlow_Exercises The codes I made while I practiced various TensorFlow examples About the codes I didn't create these codes by myself, but re-crea

Terry Taewoong Um 614 Dec 08, 2022
ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

ONNX Runtime is a cross-platform inference and training machine-learning accelerator. ONNX Runtime inference can enable faster customer experiences an

Microsoft 8k Jan 04, 2023
[NeurIPS 2021] Garment4D: Garment Reconstruction from Point Cloud Sequences

Garment4D [PDF] | [OpenReview] | [Project Page] Overview This is the codebase for our NeurIPS 2021 paper Garment4D: Garment Reconstruction from Point

Fangzhou Hong 112 Dec 23, 2022
DyNet: The Dynamic Neural Network Toolkit

The Dynamic Neural Network Toolkit General Installation C++ Python Getting Started Citing Releases and Contributing General DyNet is a neural network

Chris Dyer's lab @ LTI/CMU 3.3k Jan 06, 2023
A repository for the updated version of CoinRun used to collect MUGEN, a multimodal video-audio-text dataset.

A repository for the updated version of CoinRun used to collect MUGEN, a multimodal video-audio-text dataset. This repo contains scripts to train RL agents to navigate the closed world and collect vi

MUGEN 11 Oct 22, 2022