Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking (CVPR 2021)
Pytorch implementation of the ArTIST motion model. In this repo, there are
- Training script for the Moving Agent network
- Training script for the ArTIST motion model
- Demo script for Inferring the likelihood of current observations (detections)
- Demo script for Inpainting the missing observation/detections
Demo 1: Likelihood estimation of observation
Run:
python3 demo_scoring.py
This will generate the output in the temp/ar/log_p directory, look like this: 
This demo gets as input a pretrained model of the Moving Agent Network (MA-Net), a pretrained model of ArTIST, the centroids (obtain centroids via the script in the utils), a demo test sample index and the number of clusters.
The model then evaluates the log-likelihood (lower the better) of all detections as the continuation of the observed sequence.
Demo 2: Sequence inpainting
Run:
python3 demo_inpainting.py
This will generate the multiple plauusible continuations of an observed motion, stored in the temp/ar/inpainting directory. One example looks like this: 
This demo gets as input a pretrained model of the Moving Agent Network (MA-Net), a pretrained model of ArTIST, the centroids (obtain centroids via the script in the utils), a demo test sample index and the number of samples we wish to generate.
For each generated future sequence, it computes the IoU between the last generated bounding box and the last groundtruth bounding box, as well as the mean IoU for the entire generated sequence and the groundtruth sequence.
Utilities
In this repo, there are a number of scripts to generate the required data to train/evaluate ArTIST.
prepare_data: Given the annotations of a dataset (e.g., MOT17), it extracts the motion sequences as well as the IDs of the social tracklets living the life span of the corresponding sequence, and stores it as a dictionary. If there are multiple tracking datasets that you wish to combine, you can use themerge_datasets()function inside this script.clustering: Given the output dictionary ofprepare_datascript, this script performs the K-Means clustering and stores the centroids which are then used in the ArTIST model.dataloader_aeanddataloader_ar: Given the post-processes version of the dataset dictionary (which can be done by running thepost_processscript), these two scripts define the dataloaders for training the MA-Net and ArTIST. Note that the dataloader of ArTIST uses the MA-Net to compute the social information. This can also be done jointly in an end-to-end fashion, which we observed almost no difference.create_demo_test_subset: In order to run the demo scripts, you need to run this script. However, the demo test subset has been produced and stored indata/demo_test_subset.npy.
Data
You can download the required data from the Release and put it in data/ directory.
Citation
If you find this work useful in your own research, please consider citing:
@inproceedings{saleh2021probabilistic,
author={Saleh, Fatemeh and Aliakbarian, Sadegh and Rezatofighi, Hamid and Salzmann, Mathieu and Gould, Stephen},
title = {Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking},
booktitle={Proceedings of the IEEE/CVF conference on computer vision and pattern recognition},
year = {2021}
}
9 Nov 8, 2022
4 Apr 3, 2022
64 Dec 17, 2022
1 Dec 26, 2021
1 Dec 29, 2021
443 Dec 6, 2022
1.6k Jan 4, 2023
432 Dec 17, 2022
60 Dec 24, 2022
1.2k Dec 30, 2022
27 Nov 07, 2022
2.5k Jan 04, 2023
1 Nov 17, 2021
2 Apr 12, 2022
40 Dec 22, 2022
371 Jan 06, 2023
173 Dec 25, 2022
752 Nov 18, 2022
134 Dec 19, 2022
23 Jun 09, 2022
4 Aug 18, 2022
56 Dec 12, 2022
17 Dec 12, 2022
759 Jan 08, 2023
4.5k Jan 03, 2023
116 Nov 24, 2022
0 Nov 19, 2021