Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
Maubot azuracast - A maubot to fetch data from your radio station

Maubot Azuracast A maubot to fetch data from your radio station Setup Configure

3 Mar 14, 2022
Best badge generator API to count visitors of your Repository / Account ๐Ÿฅ‡

github visitors badge A badge generator service to count visitors of your markdown file. Hello every one! In this post, I will tell you the story of m

Sแด‡ษดแดœ Gแด€แดแด‡ส€ Bแดส ใ€ฝ 3 Dec 11, 2021
Add members to unlimited telegram channels and groups

Program Features ๐Ÿ“Œ Coded with Python version 10. ๐Ÿ“Œ without the need for a proxy. ๐Ÿ“Œ without the need for a Telegram ID. ๐Ÿ“Œ Ability to add infinite p

hack4lx 10 Nov 25, 2022
An alternative to OpenFaaS nats-queue-worker for long-running functions

OpenFaas Job Worker OpenFaas Job Worker is a fork of project : OSCAR Worker - https://github.com/grycap/oscar-worker Thanks to Sebรกstian Risco @srisco

Sebastien Aucouturier 1 Jan 07, 2022
This automation protect against subdomain takeover on AWS env which also send alerts on slack.

AWS_Subdomain_Takeover_Detector Purpose The purpose of this automation is to detect misconfigured Route53 entries which are vulnerable to subdomain ta

Puneet Kumar Maurya 8 May 18, 2022
An advanced Filter Bot with nearly unlimitted filters!

Unlimited Filter Bot ใ…คใ…คใ…คใ…คใ…คใ…คใ…ค ใ…คใ…คใ…คใ…คใ…คใ…คใ…ค An advanced Filter Bot with nearly unlimitted filters! Features Nearly unlimited filters Supports all type of fil

TroJanzHEX 445 Jan 03, 2023
Palo Alto Networks PAN-OS SDK for Python

Palo Alto Networks PAN-OS SDK for Python The PAN-OS SDK for Python (pan-os-python) is a package to help interact with Palo Alto Networks devices (incl

Palo Alto Networks 281 Dec 09, 2022
Full-featured Python interface for the Slack API

This repository is archived and will not receive any updates It's time to say goodbye. I'm archiving Slacker. It's been getting harder to find time to

Oktay Sancak 1.6k Dec 13, 2022
A multifunctional bot for Discord

Um bot multifuncional e divertido para Discord Estive desenvolvendo o BotDaora desde o comeรงo de outubro de 2021 e agora ele รฉ open-source! tomei essa

Ruan 4 Dec 28, 2021
Ice-Userbot adalah userbot Telegram modular yang berjalan di Python3 dengan database sqlalchemy

Ice-Userbot Telegram Ice-Userbot adalah userbot Telegram modular yang berjalan di Python3 dengan database sqlalchemy. Berbasis Paperplane dan ProjectB

6 Apr 29, 2022
โ๐“๐ก๐ž ๐Œ๐จ๐ฌ๐ญ ๐๐จ๐ฐ๐ž๐ซ๐Ÿ๐ฎ๐ฅ๐ฅ ๐†๐ซ๐จ๐ฎ๐ฉ ๐Œ๐š๐ง๐š๐ ๐ž๐ฆ๐ž๐ง๐ญ ๐๐จ๐ญโž

โ๐“๐ก๐ž ๐Œ๐จ๐ฌ๐ญ ๐๐จ๐ฐ๐ž๐ซ๐Ÿ๐ฎ๐ฅ๐ฅ ๐†๐ซ๐จ๐ฎ๐ฉ ๐Œ๐š๐ง๐š๐ ๐ž๐ฆ๐ž๐ง๐ญ ๐๐จ๐ญโž

Abdisamad Omar Mohamed 5 Jun 24, 2022
A repo containing toolings and software useful for a DevOps Engineer

DevOps-Tooling A repo containing toolings and software useful for a DevOps Engineer (or if you're setting up your Mac from the beginning) Currently se

Mohamed Abukar 45 Dec 12, 2022
Qbittorrent / Aria2 Mirror & Leech Telegram Bot

This is a Telegram Bot written in Python for mirroring files on the Internet to your Google Drive or Telegram. Based on python-aria-mirror-bot Feature

Hรผzรผnlรผ Artemis [HuzunluArtemis] 81 Jul 15, 2022
This is my Discord-Bot named priamoryki-bot based on python.

This is my Discord-Bot named priamoryki-bot based on python. It's a public repository without private information, so you need to correct some code for everything to be working.

priamoryki 2 Dec 14, 2022
A ideia รฉ fornecer uma base ampla de questรตes do ENEM como uma api REST

base10 "A ideia รฉ fornecer uma base ampla de questรตes do ENEM como uma api REST" TODO Documentar a api com apifairy Criar testes Criar crawler para si

Wadson Garbes 4 Apr 24, 2022
Python tool to Check running WebClient services on multiple targets based on @leechristensen

WebClient Service Scanner Python tool to Check running WebClient services on multiple targets based on @tifkin_ idea. This tool uses impacket project.

Pixis 153 Dec 28, 2022
WhatsApp Multi Device Client

WhatsApp Multi Device Client

23 Nov 18, 2022
This bot will send you an email or notify you via telegram & discord if dolar/lira parity breaks a record.

Dolar Rekor Kฤฑrdฤฑ Mฤฑ? This bot will send you an email or notify you via Telegram & Discord if Dolar/Lira parity breaks a record. Mailgun can be used a

YiฤŸit GรถktuฤŸ Budanur 2 Oct 14, 2021
Clippin n grafting Backend

Clipping' n Grafting Presenting you, ๐ŸŽ‰ Clippin' n Grafting ๐ŸŽ‰ , your very own ecommerce website displaying all your artsy-craftsy stuff. Not only the

Google-Developer-Student-Club-ISquareIT (GDSC IยฒIT) 2 Oct 22, 2021
A Simple Telegram Bot To Download And Upload Files

AquaDLBot โž  I Can Download And Upload files To Telegram DEMO Copyright (C) 2020-2026 by [ema

Asia Argento 8 Feb 15, 2022