Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
A file-based quote bot written in Python

Let's Write a Python Quote Bot! This repository will get you started with building a quote bot in Python. It's meant to be used along with the Learnin

1 Feb 03, 2022
This is a simple bot that can be used to upload images to a third-party cloud (image hosting). Currently, only the imgbb.com website supports the bot. I Will do future updates

TGImageHosting This is a simple bot that can be used to upload images to a third party cloud (image hosting). Currently, only the imgbb.com website su

Abhijith N T 0 Dec 26, 2021
Python interface to the LinkedIn API

Python LinkedIn Python interface to the LinkedIn API This library provides a pure Python interface to the LinkedIn Profile, Group, Company, Jobs, Sear

ozgur 844 Dec 27, 2022
"Nesse projeto criei uma automação para abrir as tarefas no Jira em massa pegando de uma determinada fila do Zendesk."

automacao-Zendesk "Nesse projeto criei uma automação para abrir as tarefas no Jira em massa pegando de uma determinada fila do Zendesk." en-us "In thi

tokoyamy 1 Dec 20, 2021
Automated crypto trading bot as adapted from Algovibes.

crypto-trading-bot Automated crypto trading bot as adapted from Algovibes. Pre-requisites Ensure that you have created a Binance API key before procee

Kai Koh 33 Nov 01, 2022
Ross Virtual Assistant is a programme which can play Music, search Wikipedia, open Websites and much more.

Ross-Virtual-Assistant Ross Virtual Assistant is a programme which can play Music, search Wikipedia, open Websites and much more. Installation Downloa

Jehan Patel 4 Nov 08, 2021
just a program i made cuz a friend got tokenlogged and spammed me with these scam/phishing links so i made a programm to spam these websides with fake logins

scam-webside-spammer just a program i made cuz a friend got tokenlogged and spammed me with these scam/phishing links so i made a programm to spam the

TerrificTable 3 Sep 23, 2022
Robot to convert files to direct links, hosting files on Telegram servers, unlimited and without restrictions

stream-cloud demo : downloader_star_bot Run : Docker : install docker , docker-compose set Environment or edit Config/init.py docker-compose up Heroku

53 Dec 21, 2022
A Python library for PagerDuty.

Pygerduty Python Library for PagerDuty's REST API and Events API. This library was originally written to support v1 and is currently being updated to

Dropbox 164 Dec 20, 2022
Most Powerful Chatbot On Telegram Bot

About Hello, I am Lycia [リュキア], An Intelligent ChatBot. If You Are Feeling Lonely, You can Always Come to me and Chat With Me! How To Host The easiest

RedAura 8 May 26, 2021
My attempt at weaponizing Discord.

MayorbotC2 This is my Discord C2 bot. There are many like it, but this one is mine. MayorbotC2 is a project I absolutely forgot about until I was pilf

Joe Helle 19 May 16, 2022
A GETTR API client written in Python.

GUTTR A GETTR client library written in Python. I rushed to get this out so it's a bit janky. Open an issue if something is broken or missing. Getting

Roger Johnston 13 Nov 23, 2022
A repository of publicly verifiable token Sale contracts

Token-Sale-Plutus-Contract A repository of publicly verifiable token sale and royalty contracts. This will be the storage solution since it is easily

Logical Mechanism 29 Aug 18, 2022
Get an SNS alert for High Severity GuardDuty findings

Automation AWS-GuardDuty findings Get an SNS alert for High Severity GuardDuty findings Problem: Getting notified when there is Red finding in AWS Gua

Giten Mitra 4 Nov 18, 2022
A Telegram Repo For Devs To Controll The Bots Under Maintenance.This Bot Is For Developers, If Your Bot Is Down, Use This Repo To Give Your Dear Subscribers Some Support By Providing Them Response.

Maintenance Bot A Telegram Repo For Devs To Controll The Bots Under Maintenance About This Bot This Bot Is For Developers, If Your Bot Is Down, Use Th

Vɪᴠᴇᴋ 47 Dec 29, 2022
Check and write all account info + Check nitro on account

Discord-Token-Checker Check and write all account info + Check nitro on account Also check https://github.com/GuFFy12/Discord-Token-Parser (Parse disc

36 Jan 01, 2023
Tweet stream in OBS browser source

Tweetron TweetronはOBSブラウザーソースを使用してツイートを画面上に表示するツールソフトです Windowsのみ対応 (Windows10動作確認済) ダウンロード こちらから最新版をダウンロードしてください (現在ベータテスト版を配布しています) Download ver0.0.

Cube 0 Apr 05, 2022
Scratch2py or S2py is a easy to use, versatile tool to communicate with the Scratch API Based of Scratch2py

Scratch2py Scratch2py or S2py is a easy to use, versatile tool to communicate with the Scratch API Based of Scratch2py Installation Run this command i

2 Jan 13, 2022
Paid Udemy Courses with Coupons

Freedemy Paid Udemy Courses with Coupons Steps to run pip3 install -r requirements.txt python3 free-courses.py Then you can click the Enroll Link and

GOKUL A.P 23 Dec 14, 2022
Python library for the DeepL language translation API.

The DeepL API is a language translation API that allows other computer programs to send texts and documents to DeepL's servers and receive high-quality translations. This opens a whole universe of op

DeepL 535 Jan 04, 2023