PyTorch implementations of the paper: "Learning Independent Instance Maps for Crowd Localization"

Overview

IIM - Crowd Localization


This repo is the official implementation of paper: Learning Independent Instance Maps for Crowd Localization. The code is developed based on C3F. framework

Progress

  • Testing Code (2020.12.10)
  • Training Code
    • NWPU (2020.12.14)
    • JHU (2021.01.05)
    • UCF-QNRF (2020.12.30)
    • ShanghaiTech Part A/B (2020.12.29)
    • FDST (2020.12.30)
  • scale information for UCF-QNRF and ShanghaiTech Part A/B (2021.01.07)

Getting Started

Preparation

  • Prerequisites

    • Python 3.7
    • Pytorch 1.6: http://pytorch.org .
    • other libs in requirements.txt, run pip install -r requirements.txt.
  • Code

  • Datasets

    • Download NWPU-Crowd dataset from this link.

    • Unzip *zip files in turns and place images_part* into the same folder (Root/ProcessedData/NWPU/images).

    • Download the processing labels and val gt file from this link. Place them into Root/ProcessedData/NWPU/masks and Root/ProcessedData/NWPU, respectively.

    • If you want to reproduce the results on Shanghai Tech Part A/B , UCF-QNRF, and JHU datasets, you can follow the instructions in DATA.md to setup the datasets.

    • Finally, the folder tree is below:

   -- ProcessedData
   	|-- NWPU
   		|-- images
   		|   |-- 0001.jpg
   		|   |-- 0002.jpg
   		|   |-- ...
   		|   |-- 5109.jpg
   		|-- masks
   		|   |-- 0001.png
   		|   |-- 0002.png
   		|   |-- ...
   		|   |-- 3609.png
   		|-- train.txt
   		|-- val.txt
   		|-- test.txt
   		|-- val_gt_loc.txt
   -- PretrainedModels
     |-- hrnetv2_w48_imagenet_pretrained.pth
   -- IIM
     |-- datasets
     |-- misc
     |-- ...

Training

  • run python train.py.
  • run tensorboard --logdir=exp --port=6006.
  • The validtion records are shown as follows: val_curve
  • The sub images are the input image, GT, prediction map,localization result, and pixel-level threshold, respectively: val_curve

Tips: The training process takes ~50 hours on NWPU datasets with two TITAN RTX (48GB Memeory).

Testing and Submitting

  • Modify some key parameters in test.py:
    • netName.
    • model_path.
  • Run python test.py. Then the output file (*_*_test.txt) will be generated, which can be directly submitted to CrowdBenchmark

Visualization on the val set

  • Modify some key parameters in test.py:
    • test_list = 'val.txt'
    • netName.
    • model_path.
  • Run python test.py. Then the output file (*_*_val.txt) will be generated.
  • Modify some key parameters in vis4val.py:
    • pred_file.
  • Run python vis4val.py.

Performance

The results (F1, Pre., Rec. under the sigma_l) and pre-trained models on NWPU val set, UCF-QNRF, SHT A, SHT B, and FDST:

Method NWPU val UCF-QNRF SHT A
Paper: VGG+FPN [2,3] 77.0/80.2/74.1 68.8/78.2/61.5 72.5/72.6/72.5
This Repo's Reproduction: VGG+FPN [2,3] 77.1/82.5/72.3 67.8/75.7/61.5 71.6/75.9/67.8
Paper: HRNet [1] 80.2/84.1/76.6 72.0/79.3/65.9 73.9/79.8/68.7
This Repo's Reproduction: HRNet [1] 79.8/83.4/76.5 72.0/78.7/66.4 76.1/79.1/73.3
Method SHT B FDST JHU
Paper: VGG+FPN [2,3] 80.2/84.9/76.0 93.1/92.7/93.5 -
This Repo's Reproduction: VGG+FPN [2,3] 81.7/88.5/75.9 93.9/94.7/93.1 61.8/73.2/53.5
Paper: HRNet [1] 86.2/90.7/82.1 95.5/95.3/95.8 62.5/74.0/54.2
This Repo's Reproduction: HRNet [1] 86.0/91.5/81.0 95.7/96.9 /94.4 64.0/73.3/56.8

References

  1. Deep High-Resolution Representation Learning for Visual Recognition, T-PAMI, 2019.
  2. Very Deep Convolutional Networks for Large-scale Image Recognition, arXiv, 2014.
  3. Feature Pyramid Networks for Object Detection, CVPR, 2017.

About the leaderboard on the test set, please visit Crowd benchmark. Our submissions are the IIM(HRNet) and IIM (VGG16).

Video Demo

We test the pretrained HR Net model on the NWPU dataset in a real-world subway scene. Please visit bilibili or YouTube to watch the video demonstration. val_curve

Citation

If you find this project is useful for your research, please cite:

@article{gao2020learning,
  title={Learning Independent Instance Maps for Crowd Localization},
  author={Gao, Junyu and Han, Tao and Yuan, Yuan and Wang, Qi},
  journal={arXiv preprint arXiv:2012.04164},
  year={2020}
}

Our code borrows a lot from the C^3 Framework, and you may cite:

@article{gao2019c,
  title={C$^3$ Framework: An Open-source PyTorch Code for Crowd Counting},
  author={Gao, Junyu and Lin, Wei and Zhao, Bin and Wang, Dong and Gao, Chenyu and Wen, Jun},
  journal={arXiv preprint arXiv:1907.02724},
  year={2019}
}

If you use pre-trained models in this repo (HR Net, VGG, and FPN), please cite them.

Owner
tao han
tao han
Anime Face Detector using mmdet and mmpose

Anime Face Detector This is an anime face detector using mmdetection and mmpose. (To avoid copyright issues, I use generated images by the TADNE model

198 Jan 07, 2023
A PyTorch-Based Framework for Deep Learning in Computer Vision

TorchCV: A PyTorch-Based Framework for Deep Learning in Computer Vision @misc{you2019torchcv, author = {Ansheng You and Xiangtai Li and Zhen Zhu a

Donny You 2.2k Jan 09, 2023
The description of FMFCC-A (audio track of FMFCC) dataset and Challenge resluts.

FMFCC-A This project is the description of FMFCC-A (audio track of FMFCC) dataset and Challenge resluts. The FMFCC-A dataset is shared through BaiduCl

18 Dec 24, 2022
a pytorch implementation of auto-punctuation learned character by character

Learning Auto-Punctuation by Reading Engadget Articles Link to Other of my work 🌟 Deep Learning Notes: A collection of my notes going from basic mult

Ge Yang 137 Nov 09, 2022
Scripts and misc. stuff related to the PortSwigger Web Academy

PortSwigger Web Academy Notes Mostly scripts to automate the exploits. Going in the order of the recomended learning path - starting with SQLi. Commun

pageinsec 17 Dec 30, 2022
[ECCV 2020] Gradient-Induced Co-Saliency Detection

Gradient-Induced Co-Saliency Detection Zhao Zhang*, Wenda Jin*, Jun Xu, Ming-Ming Cheng ⭐ Project Home » The official repo of the ECCV 2020 paper Grad

Zhao Zhang 35 Nov 25, 2022
A pytorch implementation of faster RCNN detection framework (Use detectron2, it's a masterpiece)

Notice(2019.11.2) This repo was built back two years ago when there were no pytorch detection implementation that can achieve reasonable performance.

Ruotian(RT) Luo 1.8k Jan 01, 2023
The Pytorch code of "Joint Distribution Matters: Deep Brownian Distance Covariance for Few-Shot Classification", CVPR 2022 (Oral).

DeepBDC for few-shot learning        Introduction In this repo, we provide the implementation of the following paper: "Joint Distribution Matters: Dee

FeiLong 116 Dec 19, 2022
Discord bot for notifying on github events

Git-Observer Discord bot for notifying on github events ⚠️ This bot is meant to write messages to only one channel (implementing this for multiple pro

ilu_vatar_ 0 Apr 19, 2022
BESS: Balanced Evolutionary Semi-Stacking for Disease Detection via Partially Labeled Imbalanced Tongue Data

Balanced-Evolutionary-Semi-Stacking Code for the paper ''BESS: Balanced Evolutionary Semi-Stacking for Disease Detection via Partially Labeled Imbalan

0 Jan 16, 2022
Official implementation of TMANet.

Temporal Memory Attention for Video Semantic Segmentation, arxiv Introduction We propose a Temporal Memory Attention Network (TMANet) to adaptively in

wanghao 94 Dec 02, 2022
3D-printable hand-strapped keyboard

Note: This repo has not been cleaned up and prepared for general consumption at all. This is just a dump of the project files. If there is any interes

Wojciech Baranowski 41 Dec 31, 2022
A Multi-modal Model Chinese Spell Checker Released on ACL2021.

ReaLiSe ReaLiSe is a multi-modal Chinese spell checking model. This the office code for the paper Read, Listen, and See: Leveraging Multimodal Informa

DaDa 106 Dec 29, 2022
Weakly-Supervised Semantic Segmentation Network with Deep Seeded Region Growing (CVPR 2018).

Weakly-Supervised Semantic Segmentation Network with Deep Seeded Region Growing (CVPR2018) By Zilong Huang, Xinggang Wang, Jiasi Wang, Wenyu Liu and J

Zilong Huang 245 Dec 13, 2022
My course projects for the 2021 Spring Machine Learning course at the National Taiwan University (NTU)

ML2021Spring There are my projects for the 2021 Spring Machine Learning course at the National Taiwan University (NTU) Course Web : https://speech.ee.

Ding-Li Chen 15 Aug 29, 2022
RP-GAN: Stable GAN Training with Random Projections

RP-GAN: Stable GAN Training with Random Projections This repository contains a reference implementation of the algorithm described in the paper: Behna

Ayan Chakrabarti 20 Sep 18, 2021
Tech Resources for Academic Communities

Free tech resources for faculty, students, researchers, life-long learners, and academic community builders for use in tech based courses, workshops, and hackathons.

Microsoft 2.5k Jan 04, 2023
MGFN: Multi-Graph Fusion Networks for Urban Region Embedding was accepted by IJCAI-2022.

Multi-Graph Fusion Networks for Urban Region Embedding (IJCAI-22) This is the implementation of Multi-Graph Fusion Networks for Urban Region Embedding

202 Nov 18, 2022
Keras Image Embeddings using Contrastive Loss

Keras-Image-Embeddings-using-Contrastive-Loss Image to Embedding projection in vector space. Implementation in keras and tensorflow for custom data. B

Shravan Anand K 5 Mar 21, 2022
An Open-Source Toolkit for Prompt-Learning.

An Open-Source Framework for Prompt-learning. Overview • Installation • How To Use • Docs • Paper • Citation • What's New? Nov 2021: Now we have relea

THUNLP 2.3k Jan 07, 2023