Starter code for the ICCV 2021 paper, 'Detecting Invisible People'

Overview

Detecting Invisible People

[ICCV 2021 Paper] [Website]

Tarasha Khurana, Achal Dave, Deva Ramanan

Introduction

This repository contains code for Detecting Invisible People. We extend the original DeepSORT algorithm to localize people even while they are completely occluded in a video. See the arXiv preprint for more information.

Dependencies

Create a conda environment with the given environment.yml file.

conda env create -f environment.yml

Preprocessing

The code expects the directory structure of your dataset in the MOT Challenge data format, which is approximately like the following:

MOT17/
-- train/
---- seq_01/
------ img1/
------ img1Depth/
------ gt/
------ det/
...
-- test/
---- seq_02/
------ img1/
------ img1Depth/
------ det/

The folder img1Depth stores the normalized disparity in .npy format. See Note. Originally, the paper runs the method on depth given by the MegaDepth depth estimator.

Given the above folder structure, generate the appearance features for your detections as described in the DeepSORT repository.

Running the method

The script run_forecast_filtering.sh will run the method with hyperparameters used in the paper. It will produce output .txt files in the MOT Challenge submission format. The bashscript has support for computing the metrics, but this has not been verified. Run the bashscript like the following:

bash run_forecast_filtering.sh experimentName

Note that in order to speed up code release, the dataset, preprocessed detections and output file paths are hardcoded in the files and will have to be manually changed.

Citing Detecting Invisible People

If you find this code useful in your research, please consider citing the following paper:

@inproceedings{khurana2021detecting,
  title={{Detecting Invisible People}},
  author={Khurana, Tarasha and Dave, Achal and Ramanan, Deva},
  booktitle={{IEEE/CVF International Conference on Computer Vision (ICCV)}},
  year={2021}
}

Warning

This is only the starter code that has not been cleaned for release. It currently only has verified support for running the method described in Detecting Invisible People, with the output tracks written in the MOT Challenge submission format. Although Top-k metric's code has been provided, this codebase does not guarantee support for the metric yet.

The hope is that you are able to benchmark this method for your CVPR 2022 submission and compute your own metrics on the method's output. If the method code does not work, please open an issue.

Note

Although it is easy to run any monocular depth estimator and store their output (usually given as disparity) in an .npy file, I have added a script in tools/demo_images.py which can save the .npy files for you. Note that this script should be run after setting up the MegaDepth codebase and copying this file to its root directory. I will likely also release my own depth maps for the MOT17 dataset over the Halloween weekend.

If you try to run the metrics, I have given my groundtruth JSON (as expected by pycocotools).

Owner
Tarasha Khurana
Tarasha Khurana
Решения, подсказки, тесты и утилиты для тренировки по алгоритмам от Яндекса.

Решения и подсказки к тренировке по алгоритмам от Яндекса Что есть внутри Решения с подсказками и комментариями; рекомендую сначала смотреть md файл п

Yankovsky Andrey 50 Dec 26, 2022
Translate darknet to tensorflow. Load trained weights, retrain/fine-tune using tensorflow, export constant graph def to mobile devices

Intro Real-time object detection and classification. Paper: version 1, version 2. Read more about YOLO (in darknet) and download weight files here. In

Trieu 6.1k Dec 30, 2022
modelvshuman is a Python library to benchmark the gap between human and machine vision

modelvshuman is a Python library to benchmark the gap between human and machine vision. Using this library, both PyTorch and TensorFlow models can be evaluated on 17 out-of-distribution datasets with

Bethge Lab 244 Jan 03, 2023
Code for "ATISS: Autoregressive Transformers for Indoor Scene Synthesis", NeurIPS 2021

ATISS: Autoregressive Transformers for Indoor Scene Synthesis This repository contains the code that accompanies our paper ATISS: Autoregressive Trans

138 Dec 22, 2022
Generative vs Discriminative: Rethinking The Meta-Continual Learning (NeurIPS 2021)

Generative vs Discriminative: Rethinking The Meta-Continual Learning (NeurIPS 2021) In this repository we provide PyTorch implementations for GeMCL; a

4 Apr 15, 2022
Ansible Automation Example: JSNAPY PRE/POST Upgrade Validation

Ansible Automation Example: JSNAPY PRE/POST Upgrade Validation Overview This example will show how to validate the status of our firewall before and a

Calvin Remsburg 1 Jan 07, 2022
The project page of paper: Architecture disentanglement for deep neural networks [ICCV 2021, oral]

This is the project page for the paper: Architecture Disentanglement for Deep Neural Networks, Jie Hu, Liujuan Cao, Tong Tong, Ye Qixiang, ShengChuan

Jie Hu 15 Aug 30, 2022
PyTorch Implementation of CvT: Introducing Convolutions to Vision Transformers

CvT: Introducing Convolutions to Vision Transformers Pytorch implementation of CvT: Introducing Convolutions to Vision Transformers Usage: img = torch

Rishikesh (ऋषिकेश) 193 Jan 03, 2023
Prososdy Morph: A python library for manipulating pitch and duration in an algorithmic way, for resynthesizing speech.

ProMo (Prosody Morph) Questions? Comments? Feedback? Chat with us on gitter! A library for manipulating pitch and duration in an algorithmic way, for

Tim 71 Jan 02, 2023
existing and custom freqtrade strategies supporting the new hyperstrategy format.

freqtrade-strategies Description Existing and self-developed strategies, rewritten to support the new HyperStrategy format from the freqtrade-develop

39 Aug 20, 2021
1st Solution For NeurIPS 2021 Competition on ML4CO Dual Task

KIDA: Knowledge Inheritance in Data Aggregation This project releases our 1st place solution on NeurIPS2021 ML4CO Dual Task. Slide and model weights a

MEGVII Research 24 Sep 08, 2022
FocusFace: Multi-task Contrastive Learning for Masked Face Recognition

FocusFace This is the official repository of "FocusFace: Multi-task Contrastive Learning for Masked Face Recognition" accepted at IEEE International C

Pedro Neto 21 Nov 17, 2022
A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.

Light Gradient Boosting Machine LightGBM is a gradient boosting framework that uses tree based learning algorithms. It is designed to be distributed a

Microsoft 14.5k Jan 08, 2023
Predict the latency time of the deep learning models

Deep Neural Network Prediction Step 1. Genernate random parameters and Run them sequentially : $ python3 collect_data.py -gp -ep -pp -pl pooling -num

QAQ 1 Nov 12, 2021
Code that accompanies the paper Semi-supervised Deep Kernel Learning: Regression with Unlabeled Data by Minimizing Predictive Variance

Semi-supervised Deep Kernel Learning This is the code that accompanies the paper Semi-supervised Deep Kernel Learning: Regression with Unlabeled Data

58 Oct 26, 2022
A concise but complete implementation of CLIP with various experimental improvements from recent papers

x-clip (wip) A concise but complete implementation of CLIP with various experimental improvements from recent papers Install $ pip install x-clip Usag

Phil Wang 515 Dec 26, 2022
Generative Exploration and Exploitation - This is an improved version of GENE.

GENE This is an improved version of GENE. In the original version, the states are generated from the decoder of VAE. We have to check whether the gere

33 Mar 23, 2022
Framework for abstracting Amiga debuggers and access to AmigaOS libraries and devices.

Framework for abstracting Amiga debuggers. This project provides abstration to control an Amiga remotely using a debugger. The APIs are not yet stable

Roc Vallès 39 Nov 22, 2022
Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shift (ICCV 2021)

Π-NAS This repository provides the evaluation code of our submitted paper: Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training

Jiqi Zhang 18 Aug 18, 2022
A C implementation for creating 2D voronoi diagrams

Branch OSX/Linux Windows master dev jc_voronoi A fast C/C++ header only implementation for creating 2D Voronoi diagrams from a point set Uses Fortune'

Mathias Westerdahl 481 Dec 29, 2022