Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
AWS DeepRacer Free Student Workshop: Run faster by using your custom waypoints

AWS DeepRacer Free Student Workshop: Run faster by using your custom waypoints Reward Function Template for waypoints def reward_function(params):

Yuen Cheuk Lam 88 Nov 27, 2022
Tools for use in DeFi. Impermanent Loss calculations, staking and farming strategies, coingecko and pancakeswap API queries, liquidity pools and more

DeFi open source tools Get Started Instalation General Tools Impermanent Loss, simple calculation Compare Buy & Hold with Staking and Farming Complete

Juan Pablo Pisano 467 Jan 08, 2023
This is a music bot for discord written in python

this is a music bot for discord written in python, it is designed for educational use ONLY, I do not take any responsibility for uses outside of educational use

5 Dec 24, 2021
A management system designed for the employees of MIRAS (Art Gallery). It is used to sell/cancel tickets, book/cancel events and keeps track of all upcoming events.

Art-Galleria-Management-System Its a management system designed for the employees of MIRAS (Art Gallery). Backend : Python Frontend : Django Database

Areesha Tahir 8 Nov 30, 2022
Michelle is a Discord Bot coded in Python with Discord.py by Mudit07.

Michelle is a Discord Bot coded in Python with Discord.py by Mudit07.

Michelle 3 Oct 09, 2021
A program that generates discord.py code

discord-py-generator A program that generates discord.py code Setup in cmds.txt file add your user id, client id and bot token you can change the bot

3 Dec 15, 2022
Lumi-Bot - Discord bot that fetches cryptocurrency prices utilizing CoinGeko API

Lumi-Bot Discord bot that fetches and monitors cryptocurrency prices utilizing C

Diego Castro 2 Oct 08, 2022
Nasdaq Cloud Data Service (NCDS) provides a modern and efficient method of delivery for realtime exchange data and other financial information. This repository provides an SDK for developing applications to access the NCDS.

Nasdaq Cloud Data Service (NCDS) Nasdaq Cloud Data Service (NCDS) provides a modern and efficient method of delivery for realtime exchange data and ot

Nasdaq 8 Dec 01, 2022
A Discord bot to allow people to create lists of random characters, with limit reroll options.

Mugen Bot A small bot I made to practice python and allow people to publically select random characters on a discord server. Uses py-cord, as that is

Haley 2 Feb 06, 2022
API Basica per a synologys Active Backup For Buissiness

Synology Active Backup for Business API-NPP Informació Per executar el programa

Nil Pujol 0 May 13, 2022
PaddleOCR推理的pytorch实现和模型转换

PaddleOCR2Pytorch 简介 ”真·白嫖“PaddleOCR 注意 PytorchOCR由PaddleOCR-2.0rc1+动态图版本移植。 特性 高质量推理模型,准确的识别效果 超轻量ptocr_mobile移动端系列 通用ptocr_server系列 支持中英文数字组合识别、竖排文本

519 Jan 08, 2023
“Hey there 👋 I'm szrosebot .A Powerful, Smart And Simple Group Manager with some extra features..

A Powerful, Smart And Simple Group Manager szrose bot This is the clone of DewmiBotit is a Powerful, Smart And Simple Group Manager bot made by hiruna

slgeekshow 36 Oct 30, 2022
This script books automatically a slot on Doctolib in one of the public vaccination centers in Berlin.

BOOKING IN BERLINS VACCINATION CENTERS This python script books automatically a slot on Doctolib in one of the public vaccination centers in Berlin. T

17 Jan 13, 2022
Network simulation tools

Overview I'm building my network simulation environments with Vagrant using libvirt plugin on a Ubuntu 20.04 system... and I always hated how boring i

Ivan Pepelnjak 219 Jan 07, 2023
A discord tool to use bugs and exploits

DiscordTool A discord tool to use bugs and exploits Features: send a buggy messa

6 Aug 19, 2022
Terminal-Trade(Panel API) For Binance

Binance-Trade-Project Binance Futures Terminal-Trade(API) & Binance This Project Worth Around 500 Bucks. **I Did This Project For SomeOne And He

Cyber 9 Dec 30, 2021
A very basic starter bot based on CryptoKKing with a small balance

starterbot A very basic starter bot based on CryptoKKing with a small balance, use at your own risk. I have since upgraded this script significantly a

Danny Kendrick 2 Dec 05, 2021
Discord bot that performs various functions.

rikka-bot A Discord bot that performs various functions. Table of Contents Commands Main Commands Utility Commands Admin Commands Self-Assignable Role

Carlos Saucedo 7 Aug 27, 2021
A Telegram bot written in python.

telegram_bot This bot is currently a beta project. Features A telegram bot which can: Send current COVID-19 cases/stats of Germany Send current worth

HuhnCares 1 Jan 11, 2022
A Telelgram Bot to Extract Text from an Image

Text-Scanner-OCR A Telelgram Bot to Extract Text from an Image Configs Vars API_KEY: Your API_KEY from OCR Space GROUP: Your Group Username without '@

ALBY 8 Feb 20, 2022