Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking (CVPR 2021)
Pytorch implementation of the ArTIST motion model. In this repo, there are
- Training script for the Moving Agent network
- Training script for the ArTIST motion model
- Demo script for Inferring the likelihood of current observations (detections)
- Demo script for Inpainting the missing observation/detections
Demo 1: Likelihood estimation of observation
Run:
python3 demo_scoring.py
This will generate the output in the temp/ar/log_p directory, look like this: 
This demo gets as input a pretrained model of the Moving Agent Network (MA-Net), a pretrained model of ArTIST, the centroids (obtain centroids via the script in the utils), a demo test sample index and the number of clusters.
The model then evaluates the log-likelihood (lower the better) of all detections as the continuation of the observed sequence.
Demo 2: Sequence inpainting
Run:
python3 demo_inpainting.py
This will generate the multiple plauusible continuations of an observed motion, stored in the temp/ar/inpainting directory. One example looks like this: 
This demo gets as input a pretrained model of the Moving Agent Network (MA-Net), a pretrained model of ArTIST, the centroids (obtain centroids via the script in the utils), a demo test sample index and the number of samples we wish to generate.
For each generated future sequence, it computes the IoU between the last generated bounding box and the last groundtruth bounding box, as well as the mean IoU for the entire generated sequence and the groundtruth sequence.
Utilities
In this repo, there are a number of scripts to generate the required data to train/evaluate ArTIST.
prepare_data: Given the annotations of a dataset (e.g., MOT17), it extracts the motion sequences as well as the IDs of the social tracklets living the life span of the corresponding sequence, and stores it as a dictionary. If there are multiple tracking datasets that you wish to combine, you can use themerge_datasets()function inside this script.clustering: Given the output dictionary ofprepare_datascript, this script performs the K-Means clustering and stores the centroids which are then used in the ArTIST model.dataloader_aeanddataloader_ar: Given the post-processes version of the dataset dictionary (which can be done by running thepost_processscript), these two scripts define the dataloaders for training the MA-Net and ArTIST. Note that the dataloader of ArTIST uses the MA-Net to compute the social information. This can also be done jointly in an end-to-end fashion, which we observed almost no difference.create_demo_test_subset: In order to run the demo scripts, you need to run this script. However, the demo test subset has been produced and stored indata/demo_test_subset.npy.
Data
You can download the required data from the Release and put it in data/ directory.
Citation
If you find this work useful in your own research, please consider citing:
@inproceedings{saleh2021probabilistic,
author={Saleh, Fatemeh and Aliakbarian, Sadegh and Rezatofighi, Hamid and Salzmann, Mathieu and Gould, Stephen},
title = {Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking},
booktitle={Proceedings of the IEEE/CVF conference on computer vision and pattern recognition},
year = {2021}
}
9 Nov 8, 2022
4 Apr 3, 2022
64 Dec 17, 2022
1 Dec 26, 2021
1 Dec 29, 2021
443 Dec 6, 2022
1.6k Jan 4, 2023
432 Dec 17, 2022
1 Apr 09, 2022
3 Jun 03, 2021
12 Oct 24, 2022
83 Jan 07, 2023
80 Dec 08, 2022
1k Jan 04, 2023
7 Dec 22, 2022
656 Dec 28, 2022
149 Dec 22, 2022
134 Dec 20, 2022
13 Dec 13, 2022
97 Dec 13, 2022
13 May 11, 2022
1 May 17, 2022
11.7k Jan 09, 2023
16 Nov 01, 2022
1 Nov 17, 2021
83 Nov 27, 2022
393 Dec 22, 2022
353 Dec 30, 2022