AirCode: A Robust Object Encoding Method

Overview

AirCode

This repo contains source codes for the arXiv preprint "AirCode: A Robust Object Encoding Method"

Demo

Object matching comparison when the objects are non-rigid and the view is changed, left is the result of our method while right is the result of NetVLAD

Relocalization on KITTI datasets

Dependencies

  • Python
  • PyTorch
  • OpenCV
  • Matplotlib
  • NumPy
  • Yaml

Data

Four datasets are used in our experiments.

KITTI Odometry

For relocalization experiment. Three sequences are selected, and they are "00", "05" and "06".

KITTI Tracking

For multi-object matching experiment. Four sequences are selected, and they are "0002", "0003", "0006", "0010".

VOT Datasets

For single-object matching experiment. We select three sequences from VOT2019 datasets and they are "bluecar", "bus6" and "humans_corridor_occ_2_A", because the tracked objects in these sequences are included in coco datasets, which are the data we used to train mask-rcnn.

OTB Datasets

For single-object matching experiment. We select five sequences and they are "BlurBody", "BlurCar2", "Human2", "Human7" and "Liquor".

Examples

Relocalization on KITTI Datasets

  1. Extract object descrptors

    python experiments/place_recogination/online_relocalization.py -c config/experiment_tracking.yaml -g 1 -s PATH_TO_SAVE_MIDDLE_RESULTS -d PATH_TO_DATASET -m PATH_TO_MODELS
    
  2. Compute precision-recall curves

    python experiments/place_recogination/offline_process.py -c config/experiment_tracking.yaml -g 1 -d PATH_TO_DATASET -n PATH_TO_MIDDLE_RESULTS -s PATH_TO_SAVE_RESULTS
    
  3. Compute top-K relocalization results

    python experiments/place_recogination/offline_topK.py -c config/experiment_tracking.yaml -g 1 -d PATH_TO_DATASET -n PATH_TO_MIDDLE_RESULTS -s PATH_TO_SAVE_RESULTS
    

Object Matching on OTB, VOT or KITTI Tracking Datasets

  • Run multi-object matching experiment in KITTI Tracking Datasets Modify the config file and run

    python experiments/object_tracking/object_tracking.py -c config/experiment_tracking.yaml -g 1 -s PATH_TO_SAVE_RESULTS -d PATH_TO_DATASET -m PATH_TO_MODELS 
    
  • Run single-object matching experiment in OTB or VOT Datasets Modify the config file and run

    python experiments/object_tracking/single_object_tracking.py -c config/experiment_tracking.yaml -g 1 -s PATH_TO_SAVE_RESULTS -d PATH_TO_DATASET -m PATH_TO_MODELS 
    
You might also like...
PyTorch implementation of Rethinking Positional Encoding in Language Pre-training
PyTorch implementation of Rethinking Positional Encoding in Language Pre-training

TUPE PyTorch implementation of Rethinking Positional Encoding in Language Pre-training. Quickstart Clone this repository. git clone https://github.com

Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding (CVPR2022)
Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding (CVPR2022)

Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding by Qiaole Dong*, Chenjie Cao*, Yanwei Fu Paper and Supple

A Robust Non-IoU Alternative to Non-Maxima Suppression in Object Detection
A Robust Non-IoU Alternative to Non-Maxima Suppression in Object Detection

Confluence: A Robust Non-IoU Alternative to Non-Maxima Suppression in Object Detection 1. 介绍 用以替代 NMS,在所有 bbox 中挑选出最优的集合。 NMS 仅考虑了 bbox 的得分,然后根据 IOU 来

[ECCVW2020] Robust Long-Term Object Tracking via Improved Discriminative Model Prediction (RLT-DiMP)
[ECCVW2020] Robust Long-Term Object Tracking via Improved Discriminative Model Prediction (RLT-DiMP)

Feel free to visit my homepage Robust Long-Term Object Tracking via Improved Discriminative Model Prediction (RLT-DIMP) [ECCVW2020 paper] Presentation

 Robust Instance Segmentation through Reasoning about Multi-Object Occlusion [CVPR 2021]
Robust Instance Segmentation through Reasoning about Multi-Object Occlusion [CVPR 2021]

Robust Instance Segmentation through Reasoning about Multi-Object Occlusion [CVPR 2021] Abstract Analyzing complex scenes with DNN is a challenging ta

Code release for our paper,
Code release for our paper, "SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo"

SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo Thomas Kollar, Michael Laskey, Kevin Stone, Brijen Thananjeyan

object detection; robust detection; ACM MM21 grand challenge; Security AI Challenger Phase VII
object detection; robust detection; ACM MM21 grand challenge; Security AI Challenger Phase VII

赛题背景 在商品知识产权领域,知识产权体现为在线商品的设计和品牌。不幸的是,在每一天,存在着非法商户通过一些对抗手段干扰商标识别来逃避侵权,这带来了很高的知识产权风险和财务损失。为了促进先进的多媒体人工智能技术的发展,以保护企业来之不易的创作和想法免受恶意使用和剽窃,因此提出了鲁棒性标识检测挑战赛

Code and models for ICCV2021 paper
Code and models for ICCV2021 paper "Robust Object Detection via Instance-Level Temporal Cycle Confusion".

Robust Object Detection via Instance-Level Temporal Cycle Confusion This repo contains the implementation of the ICCV 2021 paper, Robust Object Detect

Official code for 'Robust Siamese Object Tracking for Unmanned Aerial Manipulator' and offical introduction to UAMT100 benchmark
Official code for 'Robust Siamese Object Tracking for Unmanned Aerial Manipulator' and offical introduction to UAMT100 benchmark

SiamSA: Robust Siamese Object Tracking for Unmanned Aerial Manipulator Demo video 📹 Our video on Youtube and bilibili demonstrates the evaluation of

Comments
  • how can I get *.pth files?

    how can I get *.pth files?

    Hello, I am a beginner. When I run python experiments/place_recogination/online_relocalization.py -c config/experiment_tracking.yaml -g 1 -s results/ -d /media/jixingwu/datasetj/KITTI/Odom/data_odometry_color/sequences -m models/, points_model.pth file is needed. So how can I get it? Thank you!

    opened by jixingwu 5
  • Unable to load model under CPU-only configuration

    Unable to load model under CPU-only configuration

    Hi, I want to run object tracking on KITTI tracking datasets with only CPU using the following terminal prompt:

      python experiments/object_tracking/object_tracking.py -c config/experiment_tracking.yaml -g 1 -s ./results -d /data/datasets/SLAM_dataset/training/ -m ./weights
    

    with configuration in object_tracking.py updated with

    configs['use_gpu'] = 0
    

    However, when running with the configuration above with gcn_model.pth, maskrcnn_model.pth, points_model.pth model files in release v2.0.0, the following error occurs:

    (aircode) [email protected]:~/workspace/AirCode$ python experiments/object_tracking/object_tracking.py -c config/experiment_tracking.yaml -g 1 -s ./results -d /data/datasets/SLAM_dataset/training/ -m ./weights
    experiments/object_tracking/object_tracking.py:371: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
      configs = yaml.load(configs)
    Traceback (most recent call last):
      File "experiments/object_tracking/object_tracking.py", line 384, in <module>
        main()
      File "experiments/object_tracking/object_tracking.py", line 381, in main
        show_object_tracking(configs)
      File "experiments/object_tracking/object_tracking.py", line 272, in show_object_tracking
        superpoint_model = build_superpoint_model(configs, requires_grad=False)
      File "./model/build_model.py", line 101, in build_superpoint_model
        model.load_state_dict(model_dict)
      File "/home/yutianc/minicondas/envs/aircode/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1052, in load_state_dict
        self.__class__.__name__, "\n\t".join(error_msgs)))
    RuntimeError: Error(s) in loading state_dict for VggLike:
            Unexpected key(s) in state_dict: "module.pretrained_net.features.0.weight", "module.pretrained_net.features.0.bias", "module.pretrained_net.features.2.weight", "module.pretrained_net.features.2.bias", "module.pretrained_net.features.5.weight", "module.pretrained_net.features.5.bias", "module.pretrained_net.features.7.weight", "module.pretrained_net.features.7.bias", "module.pretrained_net.features.10.weight", "module.pretrained_net.features.10.bias", "module.pretrained_net.features.12.weight", "module.pretrained_net.features.12.bias", "module.pretrained_net.features.14.weight", "module.pretrained_net.features.14.bias", "module.pretrained_net.features.17.weight", "module.pretrained_net.features.17.bias", "module.pretrained_net.features.19.weight", "module.pretrained_net.features.19.bias", "module.pretrained_net.features.21.weight", "module.pretrained_net.features.21.bias", "module.pretrained_net.features.24.weight", "module.pretrained_net.features.24.bias", "module.pretrained_net.features.26.weight", "module.pretrained_net.features.26.bias", "module.pretrained_net.features.28.weight", "module.pretrained_net.features.28.bias", "module.convPa.weight", "module.convPa.bias", "module.bnPa.weight", "module.bnPa.bias", "module.bnPa.running_mean", "module.bnPa.running_var", "module.bnPa.num_batches_tracked", "module.convPb.weight", "module.convPb.bias", "module.bnPb.weight", "module.bnPb.bias", "module.bnPb.running_mean", "module.bnPb.running_var", "module.bnPb.num_batches_tracked", "module.convDa.weight", "module.convDa.bias", "module.bnDa.weight", "module.bnDa.bias", "module.bnDa.running_mean", "module.bnDa.running_var", "module.bnDa.num_batches_tracked", "module.convDb.weight", "module.convDb.bias", "module.bnDb.weight", "module.bnDb.bias", "module.bnDb.running_mean", "module.bnDb.running_var", "module.bnDb.num_batches_tracked".
    

    Running object_tracking.py with CUDA seems to load models successfully. Is there something wrong with the model loading when GPU is disabled?

    opened by MarkChenYutian 4
  • Why RGB image is converted into grayscale image with 3 channels?

    Why RGB image is converted into grayscale image with 3 channels?

    Hi, I'm trying to use AirCode to do object matching on complete KITTI sequences and I'm reading the code in experiments/show_object_matching.py.

    While reading the code, I noticed that the current code is reading RGB image sequence, convert it into grayscale image, and then duplicate the image into 3-channel each with same value (as following):

    https://github.com/wang-chen/AirCode/blob/5e23e9f5322d2e4ee119d5326a6b6112cef0e6bd/experiments/show_object_matching/show_object_matching.py#L172-L176

    I'm a bit unsure about the reason why this operation is performed here as the original RGB image should contain more information about the object comparing to grayscale image. For instance, it should be easier to distinguish objects with different color but similar shape if the RGB value is preserved.

    opened by MarkChenYutian 2
Owner
Chen Wang
I am engaged in delivering simple and efficient source code.
Chen Wang
A short and easy PyTorch implementation of E(n) Equivariant Graph Neural Networks

Simple implementation of Equivariant GNN A short implementation of E(n) Equivariant Graph Neural Networks for HOMO energy prediction. Just 50 lines of

Arsenii Senya Ashukha 97 Dec 23, 2022
Full-featured Decision Trees and Random Forests learner.

CID3 This is a full-featured Decision Trees and Random Forests learner. It can save trees or forests to disk for later use. It is possible to query tr

Alejandro Penate-Diaz 3 Aug 15, 2022
Official PyTorch implementation of our AAAI22 paper: TransMEF: A Transformer-Based Multi-Exposure Image Fusion Framework via Self-Supervised Multi-Task Learning. Code will be available soon.

Official-PyTorch-Implementation-of-TransMEF Official PyTorch implementation of our AAAI22 paper: TransMEF: A Transformer-Based Multi-Exposure Image Fu

117 Dec 27, 2022
Official repository for "Intriguing Properties of Vision Transformers" (2021)

Intriguing Properties of Vision Transformers Muzammal Naseer, Kanchana Ranasinghe, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, & Ming-Hsuan Yang P

Muzammal Naseer 155 Dec 27, 2022
An implementation of MobileFormer

MobileFormer An implementation of MobileFormer proposed by Yinpeng Chen, Xiyang Dai et al. Including [1] Mobile-Former proposed in:

slwang9353 62 Dec 28, 2022
Code for Active Learning at The ImageNet Scale.

Code for Active Learning at The ImageNet Scale. This repository implements many popular active learning algorithms and allows training with torch's DDP.

Zeyad Emam 47 Dec 12, 2022
MAUS: A Dataset for Mental Workload Assessment Using Wearable Sensor - Baseline system

MAUS: A Dataset for Mental Workload Assessment Using Wearable Sensor - Baseline system Getting started To start working on this assignment, you should

2 Aug 06, 2022
The source code for Generating Training Data with Language Models: Towards Zero-Shot Language Understanding.

SuperGen The source code for Generating Training Data with Language Models: Towards Zero-Shot Language Understanding. Requirements Before running, you

Yu Meng 38 Dec 12, 2022
Development kit for MIT Scene Parsing Benchmark

Development Kit for MIT Scene Parsing Benchmark [NEW!] Our PyTorch implementation is released in the following repository: https://github.com/hangzhao

MIT CSAIL Computer Vision 424 Dec 01, 2022
Official implementation for “Unsupervised Low-Light Image Enhancement via Histogram Equalization Prior”

HEP Unsupervised Low-Light Image Enhancement via Histogram Equalization Prior Implementation Python3 PyTorch=1.0 NVIDIA GPU+CUDA Training process The

FengZhang 34 Dec 04, 2022
Fast Scattering Transform with CuPy/PyTorch

Announcement 11/18 This package is no longer supported. We have now released kymatio: http://www.kymat.io/ , https://github.com/kymatio/kymatio which

Edouard Oyallon 289 Dec 07, 2022
code from "Tensor decomposition of higher-order correlations by nonlinear Hebbian plasticity"

Code associated with the paper "Tensor decomposition of higher-order correlations by nonlinear Hebbian learning," Ocker & Buice, Neurips 2021. "plot_f

Gabriel Koch Ocker 4 Oct 16, 2022
This implements one of result networks from Large-scale evolution of image classifiers

Exotic structured image classifier This implements one of result networks from Large-scale evolution of image classifiers by Esteban Real, et. al. Req

54 Nov 25, 2022
PyTorch implementation of Densely Connected Time Delay Neural Network

Densely Connected Time Delay Neural Network PyTorch implementation of Densely Connected Time Delay Neural Network (D-TDNN) in our paper "Densely Conne

Ya-Qi Yu 64 Oct 11, 2022
TorchDistiller - a collection of the open source pytorch code for knowledge distillation, especially for the perception tasks, including semantic segmentation, depth estimation, object detection and instance segmentation.

This project is a collection of the open source pytorch code for knowledge distillation, especially for the perception tasks, including semantic segmentation, depth estimation, object detection and i

yifan liu 147 Dec 03, 2022
Solution to the Weather4cast 2021 challenge

This code was used for the entry by the team "antfugue" for the Weather4cast 2021 Challenge. Below, you can find the instructions for generating predi

Jussi Leinonen 13 Jan 03, 2023
A PyTorch implementation of the Relational Graph Convolutional Network (RGCN).

Torch-RGCN Torch-RGCN is a PyTorch implementation of the RGCN, originally proposed by Schlichtkrull et al. in Modeling Relational Data with Graph Conv

Thiviyan Singam 66 Nov 30, 2022
A PyTorch Implementation of ViT (Vision Transformer)

ViT - Vision Transformer This is an implementation of ViT - Vision Transformer by Google Research Team through the paper "An Image is Worth 16x16 Word

Quan Nguyen 7 May 11, 2022
This implementation contains the application of GPlearn's symbolic transformer on a commodity futures sector of the financial market.

GPlearn_finiance_stock_futures_extension This implementation contains the application of GPlearn's symbolic transformer on a commodity futures sector

Chengwei <a href=[email protected]"> 189 Dec 25, 2022