Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
unofficial source of the discord bot, “haunting.” created by: vorqz, vert, & Veltz

hauntingSRC unofficial source of the discord bot, “haunting.” created by: vorqz, vert, & Veltz reasoning: creators skidded the most of this bot and do

Vast 11 Nov 04, 2022
pyhakuna is a client to access the API of the time keeping service hakuna.ch.

pyhakuna pyhakuna is a client to access the API of the time keeping service hakuna.ch. The Hakuna API is – unfortunately – personal and currently does

Christian Mäder 1 Feb 15, 2022
McTrade is a bot exploiting Binance API, open source! built in python !

Open Source Crypto Trading Bot using Binance API Report Bug · Request Feature Table of Contents About The Project Built With Getting Started Prerequis

Raphael Cohen 5 Jul 17, 2022
Moderation and Utility Discord bot.

Qrista An advanced Moderation and Utility Discord bot. Features Uses Hikari and Lightbulb as a command handler Advance Logging & Whole new enviroment

Blaze Camp 2 Jan 11, 2022
Social Framework

Social Int Framework Social Int Framework its a Selenium script that scrape the IG photos and do a Reverse search on google and yandex for finding ano

29 Dec 06, 2022
A twitter multi-tool for OSINT on twitter accounts.

TwitterCheckr A twitter multi-tool for OSINT on twitter accounts. Infomation TwitterCheckr also known as TCheckr is multi-tool for OSINT on twitter a

IRIS 16 Dec 23, 2022
Maintained wavelink fork for pycord

Pycord.Wavelink Wavelink is robust and powerful Lavalink wrapper for Pycord! Wavelink features a fully asynchronous API that's intuitive and easy to u

Pycord Development 23 Dec 11, 2022
Balanced API library in python.

Balanced Online Marketplace Payments v1.x requires Balanced API 1.1. Use v0.x for Balanced API 1.0. Installation pip install balanced Usage View Bala

Balanced 70 Oct 04, 2022
A pyrogram simple bot for Educational purpose.

A pyrogram simple bot for Educational purpose. To Learn More check at @PyrogramBot or on Documentation Mandatory variables API_ID - Get It From my.tel

SpamShield 10 Dec 06, 2022
MassReportBot - Discord Mass Report Bot By Dropout

Discord Mass Report Bot By Dropout Discord Report Bot, Just Re-Made The "Admin R

vanis / 1800 0 Jan 20, 2022
A simple python discord bot with commands for moderation and utility.

Discord Bot A simple python discord bot with commands for moderation, utility and fun. Moderation $kick user reason - Kick a user from the server

syn 3 Feb 06, 2022
A file-based quote bot written in Python

Let's Write a Python Quote Bot! This repository will get you started with building a quote bot in Python. It's meant to be used along with the Learnin

Jyoti prakash Rout 1 Jan 08, 2022
Asyncio SDK for Azure Cosmos DB

Asyncio SDK for Azure Cosmos DB. This library is intended to be a very thin asyncio wrapper around the Azure Comsos DB Rest API. It is not intended to have feature parity with the Microsoft Azure SDK

Grant McDonald 4 Dec 04, 2021
Código que verifica se o grafo é Hamiltoniano (Em Python)

Código para encontrar um ciclo de Hamilton em um dado grafo e a partir daí verificar se o grafo é hamiltoniano. Um ciclo hamiltoniano é um ciclo gerad

Hemili Beatriz 1 Jan 08, 2022
With this program you can work English & Turkish

1 - How Can I Work This? You must have Python compilers in order to run this program. First of all, download the compiler in the link. Compiler 2 - Do

Mustafa Bahadır Doğrusöz 3 Aug 07, 2021
The purpose of this bot is to take soundcloud track requests, that are posted in the stream-requests channel, and add them to a playlist, to make the process of scrolling through the requests easier for Root

Discord Song Collector Dont know if anyone is actually going to read this, but the purpose of this bot is to check every message in the stream-request

2 Mar 01, 2022
Telegram bot to trim and download videos from youtube.

Inline-YouTube-Trim-Bot Telegram bot to trim and download youtube videos Deploy You can deploy this bot anywhere. Demo - YouTubeBot Required Variables

SUBIN 56 Dec 11, 2022
Raid ToolBox (RTB) is a big toolkit of Spamming/Raiding/Token management tools for discord.

This code is very out of date and not very good, feel free to make it into something better. (we check the github page every 5 years to pulls your PRs

2 Oct 03, 2021
A GitHub Follower Bot that is a WIP.

GitHub Follower Bot (WIP) Work In Progress This bot is a WIP. There are still many features I plan to add and code I need to improve (I'm still fairly

Christian Deacon 71 Dec 29, 2022
Salmanul Farisx Bot With Python

Salman_Farisx_Bot How To Deploy Video Subscribe YouTube Channel Added Features Imdb posters for autofilter. Imdb rating for autofilter. Custom caption

1 Dec 23, 2021