Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
Chatbot with python code!

Chatbot Python Chatbot with python! How to Run Installation requirements. pip install -r requirements.txt Sample Chatbot The required files must be d

Mohammad Dori 3 Jul 15, 2022
A project in order to analyze user's favorite musics, artists and genre

Spotify-Wrapped This is a project about Spotify Wrapped (which is an extra option for premium accounts, but you don't need to be premium here) This pr

Hossein Mohseni 19 Jan 04, 2023
Ulaavi for nuke, helps to keep our stocl elements organised.

Ulaavi Ulaavi for nuke, helps to keep our stock elements organised. Installation Downlaod ffmpeg from ffmpeg.org linux : https://johnvansickle.com/ffm

Arun Subramaniyam 17 Aug 24, 2022
It is a useful project for developers that includes useful tools for Instagram

InstagramIG It is a useful project for developers that includes useful tools for Instagram Installation : pip install InstagramIG Logan Usage from In

Sidra ELEzz 14 Mar 14, 2022
Account Profiles Dumper for Fortnite.

Fortnite Profile Dumper This program allows you to dump your Fortnite account profiles. How to use it? After starting the FortniteProfileDumper.py, yo

PRO100KatYT 12 Jul 28, 2022
SpaceManJax's open-source Discord Bot. Now on Github!

SpaceManBot This is SpaceManJax's open-source Discord.py Bot. Now on Github! This bot runs on Repl.it, which is a free online code editor. It can do a

Jack 1 Nov 16, 2021
A Discord BOT that uses Google Sheets for storing the roles and permissions of a discord server.

Discord Role Manager Bot Role Manager is a discord BOT that utilizes Google Sheets for the organization of a server's hierarchy and permissions. Detai

Dion Rigatos 17 Oct 13, 2022
Telegram bot for Whisper Message.

Whisper Bot @WhisperStarkBot A star ⭐ from you means a lot to us! Telegram bot for Whisper Message. Usage Deploy to Heroku Tap on above button and fil

Stark Bots 33 Nov 24, 2022
Telegram Bot for generating and decoding QR-codes

Telegram openqrgen_bot Telegram Bot that generates from user's messages and decodes QR-codes from photos. Also contains rickroll detection :) Just typ

2 Nov 14, 2021
Wanna play on the Overwatch NA servers?

OverwatchRegionSwapper-NA- Wanna play on the Overwatch NA servers? Use at you own risk. LIST OF IPs EU Netherlands: From: 5.42.168.0 to: 5.42.175.255

1 Jun 08, 2022
Python Client for MLflow Tracking Server

Python Client for MLflow Python client for MLflow REST API. Features: Unlike MLflow Tracking client all REST API methods are exposed to user. All clas

MTS 35 Dec 23, 2022
Spore REST API asyncio client

Spore REST API asyncio client

LEv145 16 Aug 02, 2022
A file-based quote bot written in Python

Let's Write a Python Quote Bot! This repository will get you started with building a quote bot in Python. It's meant to be used along with the Learnin

Jyoti prakash Rout 1 Jan 08, 2022
Criando Lambda Functions para Ingerir Dados de APIs com AWS CDK

LIVE001 - AWS Lambda para Ingerir Dados de APIs Fazer o deploy de uma função lambda com infraestrutura como código Lambda vai numa API externa e extra

Andre Sionek 12 Nov 20, 2022
An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.

TWINT - Twitter Intelligence Tool No authentication. No API. No limits. Twint is an advanced Twitter scraping tool written in Python that allows for s

TWINT Project 14.2k Jan 03, 2023
EduuRobot Telegram bot source code.

EduuRobot A multipurpose Telegram Bot made with Pyrogram and asynchronous programming. Requirements Python 3.6+ An Unix-like operating system (Running

Amano Team 119 Dec 23, 2022
4 Oct 28, 2021
Repository to access information of stocks in Bombay Stock Exchange.

BSE Repository to access information of stocks in Bombay Stock Exchange. The code in this repository uses BSE API and conclusions made using the code

1 Nov 13, 2021
A basic Ubisoft API wrapper created in python.

UbisoftAPI A basic Ubisoft API wrapper created in python. I will be updating this with more endpoints as time goes on. Please note that this is my fir

Ethan 2 Oct 31, 2021