Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
Python script for download course from platzi.com

Platzi Downloader Tool Esta es una pequeña herramienta que hace mucho y que te ahorra una gran cantidad de trabajo a la hora de descargar cursos de Pl

Devil64-Dev 21 Sep 22, 2022
Google Drive, OneDrive and Youtube as covert-channels - Control systems remotely by uploading files to Google Drive, OneDrive, Youtube or Telegram

covert-control Control systems remotely by uploading files to Google Drive, OneDrive, Youtube or Telegram using Python to create the files and the lis

Ricardo Ruiz 52 Dec 06, 2022
Video Stream: an Advanced Telegram Bot that's allow you to play Video & Music on Telegram Group Video Chat

Video Stream is an Advanced Telegram Bot that's allow you to play Video & Music on Telegram Group Video Chat 🧪 Get SESSION_NAME from below: Pyrogram

Jonathan 6 Feb 08, 2022
ESOLinuxAddonManager - Very simple addon manager for Elder Scrolls Online running on Linux.

ESOLinuxAddonManager Very simple addon manager for Elder Scrolls Online running on Linux. Well, more a downloader for now. Currently it's quite ugly b

Akseli 25 Aug 28, 2022
Open Resource Calculator Module for Python

Calculator Module for Python By White Night Install #

White_Night_awa 4 Aug 22, 2022
alpaca-trade-api-python is a python library for the Alpaca Commission Free Trading API.

alpaca-trade-api-python is a python library for the Alpaca Commission Free Trading API. It allows rapid trading algo development easily, with support for both REST and streaming data interfaces

Alpaca 1.5k Jan 09, 2023
Create custom Vanity URLs for Discord without 30 boosts

CustomVanity - Made by udp#6666 aka Apolo - OpenSource Custom Discord Vanity Creator How To Use Open CustomVanity.py Write your server invite code Wri

apolo 17 Aug 23, 2022
With this program you can work English & Turkish

1 - How Can I Work This? You must have Python compilers in order to run this program. First of all, download the compiler in the link. Compiler 2 - Do

Mustafa Bahadır Doğrusöz 3 Aug 07, 2021
Unlimited Filter Bot

Unlimited Filter Bot The Orginel Owner Of This Repo Is ㅤㅤㅤㅤㅤㅤㅤ ㅤㅤㅤㅤㅤㅤㅤ An advanced Filter Bot with nearly unlimitted filters! Features Nearly unlimite

4 Aug 02, 2022
A bot can play all variants, but standard are abit weak, so if you need strongest you can change fsf instead of stockfish_14_Dev

MAINTAINERS Drdisrespect1 and drrespectable lichess-bot Engine communication code taken from https://github.com/ShailChoksi/lichess-bot by ShailChoksi

RPNS Nimsilu 1 Dec 12, 2021
Stop writing scripts to interact with your APIs. Call them as CLIs instead.

Zum Stop writing scripts to interact with your APIs. Call them as CLIs instead. Zum (German word roughly meaning "to the" or "to" depending on the con

Daniel Leal 84 Nov 17, 2022
Automate HoYoLAB Genshin Daily Check-In Using Github Actions

Genshin Daily Check-In 🤖 Automate HoYoLAB Daily Check-In Using Github Actions KOR, ENG Instructions Fork the repository Go to Settings - Secrets Cli

Leo Kim 41 Jun 24, 2022
An unofficial client library for Google Music.

gmusicapi: an unofficial API for Google Play Music gmusicapi allows control of Google Music with Python. from gmusicapi import Mobileclient api = Mob

Simon Weber 2.5k Dec 15, 2022
An Telegram Bot By @AsmSafone To Stream Videos in Telegram Voice Chat. This is Also The Source Code of The Bot Which is Being Used In @SafoTheBot Group! ❤️

Telegram Video Player Bot (Beta) An Telegram Bot By @AsmSafone To Stream Videos in Telegram Voice Chat. Special Features Supports Live Streaming From

SAF ONE 206 Jan 03, 2023
A file-based quote bot written in Python

Let's Write a Python Quote Bot! This repository will get you started with building a quote bot in Python. It's meant to be used along with the Learnin

A . S . M . RADWAN 2 Apr 03, 2022
A drop-in vanilla discord.py cog to add slash command support with little to no code modifications

discord.py /slash cog A drop-in vanilla discord.py cog that acts as a translation layer to add slash command support with little to no code modificati

marshall 3 Jun 01, 2022
Telegram bot with various Sticker Tools

Sticker Tools Bot @Sticker_Tools_Bot A star ⭐ from you means a lot to us! Telegram bot with various Sticker Tools Usage Deploy to Heroku Tap on above

Stark Bots 20 Dec 08, 2022
Takes upcoming items from a Google Calendar and posts them to Slack.

Google Calendar to Slack by Jason Snell - [email protected] This Python s

6 Aug 21, 2022
pokemon-colorscripts compatible for mac

Pokemon colorscripts some scripts to print out images of pokemons to terminal. Inspired by DT's colorscripts compilation Description Prints out colore

43 Jan 06, 2023
A simple script & container to pull COVID data from covidlive.com.au and post a summary to a slack channel

CovidLive AU Summary Slackbot This bot is a very simple slackbot that pulls data, summarises and posts up to date AU COVID stats to a provided slack c

James 3 Dec 18, 2021