The official implementation of paper Siamese Transformer Pyramid Networks for Real-Time UAV Tracking, accepted by WACV22

Last update: Nov 25, 2022

Related tags

Deep Learning SiamTPNTracker

Overview

SiamTPN

Introduction

This is the official implementation of the SiamTPN (WACV2022). The tracker intergrates pyramid feature network and transformer into Siamese network, achieving state-of-the-art performance (better than DiMP) while runing 30 FPS on a single CPU. The tracker optimized with ONXX and openvino could run at 45 FPS on cpu end, leading promising performance when deploying on drones for tracking.

[Paper] [Raw Results] [Drone Tracking Videos] [Models]

Training

prepare data

change the path in lib/train/admin/local.py to your data location

# Distributed training withh 4 nodes 
python -m torch.distributed.launch --nproc_per_node 4 tools/run_training.py --config shufflenet_l345_192

# single gpu training for test purpose
python tools/run_training.py --config shufflenet_l345_192

Test and evaluate SiamTPN

prepare data

change the path in lib/test/evaluation/local.py to your data location

running on cpu

# Download the pretrain model and put it under ./results/checkpoints/train/SiamTPN/ folder

python tools/test.py siamtpn shufflenet_l345_192 --dataset_name got10k_val --debug 1 --cpu 1 --epoch 100 --sequence GOT-10k_Val_000001

running on cpu with onnx optimized

The debug mode will show tracking results, more details refer to tools/test.py

Currently, onnx only support cpu version

First, you need to install onxx and onxxruningtime:

pip install onxx
# for onxx runining time, download the openvino version from release [page](https://github.com/intel/onnxruntime/releases/tag/v3.1) and install with
pip install onnxruntime_openvino-1.9.0-cp37-cp37m-linux_x86_64.whl

# please refer the [page](https://github.com/intel/onnxruntime/releases/tag/v3.1) for openvino installation details.

# Download the converted onnx model and put it under ./results/onnx/ folder
# or conver your own model with 
python tools/onnx_search.py
python tools/onnx_template.py

python tools/test.py siamtpn_onnx shufflenet_l345_192 --dataset_name got10k_val --debug 1 --cpu 1 --epoch 100 --sequence GOT-10k_Val_000001

Citation

Acknowledge

Our code is implemented based on the following libraries:

The official implementation of paper Siamese Transformer Pyramid Networks for Real-Time UAV Tracking, accepted by WACV22

Related tags

Overview

SiamTPN

Introduction

Training

prepare data

Test and evaluate SiamTPN

prepare data

running on cpu

running on cpu with onnx optimized

Citation

Acknowledge

Owner

Robotics and Intelligent Systems Control @ NYUAD

Rewrite ultralytics/yolov5 v6.0 opencv inference code based on numpy, no need to rely on pytorch

Implementation of PersonaGPT Dialog Model

Integrated physics-based and ligand-based modeling.

This is a pytorch implementation of the NeurIPS paper GAN Memory with No Forgetting.

Changing the Mind of Transformers for Topically-Controllable Language Generation

ICCV2021: Code for 'Spatial Uncertainty-Aware Semi-Supervised Crowd Counting'

使用深度学习框架提取视频硬字幕；docker容器免安装深度学习库，使用本地api接口使得界面和后端识别分离；

Task-related Saliency Network For Few-shot learning

Code accompanying the paper "How Tight Can PAC-Bayes be in the Small Data Regime?"

Shuwa Gesture Toolkit is a framework that detects and classifies arbitrary gestures in short videos

A Neural Net Training Interface on TensorFlow, with focus on speed + flexibility

Implementation of Kronecker Attention in Pytorch

Extracting knowledge graphs from language models as a diagnostic benchmark of model performance.

Perfect implement. Model shared. x0.5 (Top1:60.646) and 1.0x (Top1:69.402).

Deep-learning X-Ray Micro-CT image enhancement, pore-network modelling and continuum modelling

QHack—the quantum machine learning hackathon

7th place solution of Human Protein Atlas - Single Cell Classification on Kaggle

A PyTorch Implementation of "Watch Your Step: Learning Node Embeddings via Graph Attention" (NeurIPS 2018).

Code for the paper "Asymptotics of ℓ2 Regularized Network Embeddings"

Includes PyTorch -> Keras model porting code for ConvNeXt family of models with fine-tuning and inference notebooks.