Source codes of CenterTrack++ in 2021 ICME Workshop on Big Surveillance Data Processing and Analysis

Overview

MOT Tracked object bounding box association (CenterTrack++)

New association method based on CenterTrack. Two new branches (Tracked Size and IOU) are added onto the original CenterTrack tracker. The proposed method enables the computation of IOU distance matrix for more accurate object association compared to single displacement offset in the original CenterTrack.

Modification to CenterTrack method, image modified from CenterTrack

Abstract

The recent development of multi-object tracking (MOT) on point-based joint detection and tracking methods has attracted much research attention. CenterTrack tracking algorithm is one of such promising methods. It achieves state-of-the-art tracking performance using a simple detection model and single-frame spatial offsets to localize objects and predict their associations in a single network. However, this method still suffers from high identity switches due to the inferior association method. Only point displacement distance matrix is used to associate objects, which is not robust to deal with occlusion scenarios. To reduce the high number of identity switches and improve the tracking accuracy, more effective spatial information should be used in association. In this paper, we propose to incorporate a simple tracked object bounding box and overlapping prediction based on the current frame onto the CenterTrack algorithm. Specifically, we propose a Intersection over Union (IOU) distance cost matrix in the association step instead of point displacement distance. We evaluate our proposed tracker on the MOT17 test dataset, showing that our proposed method can reduce identity switches significantly by 22.6% and obtain a notable improvement of 1.5% in IDF1 compared to the original CenterTrack’s under the same tracklet lifetime.

Main Contributions

  • Proposed two branches (tracked box size and IOU)on top of the existing CenterTrack method for IOU distance metric computation in object association
  • Evaluation the proposed method on MOT17 dataset and obtain significant reduction in IDs and notable improvements in tracking accuracy score

Two new branches

The idea of the proposed method is to enhance the original displacement only association. Inspired by the IOU distance in SORT and IOU-Tracker, IOU distance can be used for more accurate object association across frames. IOU distance is calculated as 1 - IOU(bounding box of detected object in the previous frame and the predicted tracked object bounding box in the previous frame based on the current frame)

Tracked Object Size prediction

In order to obtain the IOU distance, the bounding box of the tracked object in the previous frame should be learnt. In this project, two methods were used to learn the tracked bounding box.

Tracking_wh: Directly learn the width and height of the tracked object bounding box in the previous frame.

Tracking_ltrb: Learn the offsets of the left, top, right and bottom of bounding box from the tracked object center in the previous frame.

The tracking_wh(left) and tracking_ltrb(right) approach illustration.

IOU prediction

To further suppress inaccurate association, the IOU value of the tracked object bounding box in adjacent frames is learnt to provide a threshold to filter unlikely associations. We would set the IOU distance to infinity if IOU distance > IOU.

Association Method

Main results

Comparison with other SOTA tracker on MOT17 test set

Note: S= Spatial features, A=appearance features

Tracker Association Features MOTA IDF1 IDs
TubeTK S 63 58.6 4137
CenterTrack S 67.8 64.7 3039
Ours S 68.1 66.2 2352
SST A 52.4 49.5 8431
CTrackerV1 S+A 66.6 57.4 5529
DEFT S+A 66.6 65.4 2823
FairMOT S+A 73.7 72.3 3303

Ablative studies on tracked size prediction method

Tracking_wh

Association Method IDF1 MOTA IDs FP(%) FN(%)
DIS 69.2 66.2 219 3.9 29.5
IOU 71.1 66.7 204 3.6 29.3
Combined 70.9 66.2 233 3.9 29.6
DIS→IOU 70 66.2 218 3.9 29.5
IOU→DIS 69.8 66.8 185 3.6 29.2

Tracking_ltrb

Association Method IDF1 MOTA IDs FP(%) FN(%)
DIS 69.2 66.2 219 3.9 29.5
IOU 72.4 66.7 191 3.8 29.2
Combined 70.8 66.5 236 3.8 29.3
DIS→IOU 70.5 66.6 202 3.8 29.2
IOU→DIS 71.4 66.7 166 3.8 29.2

Installation

Please refer to INSTALL.md for installation instructions.

Training and Evaluation

  • Download the crowdhuman pretrained model from xinyizhou/CenterTrack MODEL ZOO.md to models
  • prepare the data and convert it into COCO format refer to the original CenterTrack repo.
  • change the dataset root directory data_dir in opt.py
  • ablative studies for tracking_wh and tracking_ltrb approach respectively with five association method (IOU,DIS,Combined, IOU→DIS, DIS→IOU)
sh experiments/mot17val_tracking_wh.sh

sh experiments/mot17val_tracking_ltrb.sh

The trained model on MOT17val dataset using two approach are available in google drive, tracking_ltrb_70val.pth, tracking_wh_70val.pth.

  • Train on full mot17 training set and run model on the test set for evaluation
sh experiments/mot17full.sh

The trained models on full MOT17 dataset using ltrb approach is available in the google drive.

Demo comparison

Occlusion case

Original CenterTrack (left) vs CenterTrack++ (right)

Object exiting the frame

Original CenterTrack (left) vs CenterTrack++ (right)

Acknowledgement

A large part of the code is adapted from xingyizhou/CenterTrack, thanks for their wonderful inspiration.

Citation

If you find this paper and code useful in your research, please cite our papers.

@misc{yang2021multiobject,
      title={Multi-object Tracking with Tracked Object Bounding Box Association}, 
      author={Nanyang Yang and Yi Wang and Lap-Pui Chau},
      year={2021},
      eprint={2105.07901},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}
Owner
Nanyang Technological University Information Engineering and Media Student
YOLOX_AUDIO is an audio event detection model based on YOLOX

YOLOX_AUDIO is an audio event detection model based on YOLOX, an anchor-free version of YOLO. This repo is an implementated by PyTorch. Main goal of YOLOX_AUDIO is to detect and classify pre-defined

intflow Inc. 77 Dec 19, 2022
YOLOV4运行在嵌入式设备上

在嵌入式设备上实现YOLO V4 tiny 在嵌入式设备上实现YOLO V4 tiny 目录结构 目录结构 |-- YOLO V4 tiny |-- .gitignore |-- LICENSE |-- README.md |-- test.txt |-- t

Liu-Wei 6 Sep 09, 2021
Largest list of models for Core ML (for iOS 11+)

Since iOS 11, Apple released Core ML framework to help developers integrate machine learning models into applications. The official documentation We'v

Kedan Li 5.6k Jan 08, 2023
List of papers, code and experiments using deep learning for time series forecasting

Deep Learning Time Series Forecasting List of state of the art papers focus on deep learning and resources, code and experiments using deep learning f

Alexander Robles 2k Jan 06, 2023
An abstraction layer for mathematical optimization solvers.

MathOptInterface Documentation Build Status Social An abstraction layer for mathematical optimization solvers. Replaces MathProgBase. Citing MathOptIn

JuMP-dev 284 Jan 04, 2023
Source code for our paper "Learning to Break Deep Perceptual Hashing: The Use Case NeuralHash"

Learning to Break Deep Perceptual Hashing: The Use Case NeuralHash Abstract: Apple recently revealed its deep perceptual hashing system NeuralHash to

<a href=[email protected]"> 11 Dec 03, 2022
Annotated, understandable, and visually interpretable PyTorch implementations of: VAE, BIRVAE, NSGAN, MMGAN, WGAN, WGANGP, LSGAN, DRAGAN, BEGAN, RaGAN, InfoGAN, fGAN, FisherGAN

Overview PyTorch 0.4.1 | Python 3.6.5 Annotated implementations with comparative introductions for minimax, non-saturating, wasserstein, wasserstein g

Shayne O'Brien 471 Dec 16, 2022
Edge-aware Guidance Fusion Network for RGB-Thermal Scene Parsing

EGFNet Edge-aware Guidance Fusion Network for RGB-Thermal Scene Parsing Dataset and Results Test maps: 百度网盘 提取码:zust Citation @ARTICLE{ author={Zhou,

ShaohuaDong 10 Dec 08, 2022
IRON Kaggle project done while doing IRONHACK Bootcamp where we had to analyze and use a Machine Learning Project to predict future sales

IRON Kaggle project done while doing IRONHACK Bootcamp where we had to analyze and use a Machine Learning Project to predict future sales. In this case, we ended up using XGBoost because it was the o

1 Jan 04, 2022
Breaking the Dilemma of Medical Image-to-image Translation

Breaking the Dilemma of Medical Image-to-image Translation Supervised Pix2Pix and unsupervised Cycle-consistency are two modes that dominate the field

Kid Liet 86 Dec 21, 2022
Unsupervised MRI Reconstruction via Zero-Shot Learned Adversarial Transformers

Official TensorFlow implementation of the unsupervised reconstruction model using zero-Shot Learned Adversarial TransformERs (SLATER). (https://arxiv.

ICON Lab 22 Dec 22, 2022
Open source annotation tool for machine learning practitioners.

doccano doccano is an open source text annotation tool for humans. It provides annotation features for text classification, sequence labeling and sequ

7.1k Jan 01, 2023
A disassembler for the RP2040 Programmable I/O State-machine!

piodisasm A disassembler for the RP2040 Programmable I/O State-machine! Usage Just run piodisasm.py on a file that contains the PIO code as hex! (Such

Ghidra Ninja 29 Dec 06, 2022
MediaPipe Kullanarak İleri Seviye Bilgisayarla Görü

MediaPipe Kullanarak İleri Seviye Bilgisayarla Görü

Burak Bagatarhan 12 Mar 29, 2022
Minimal diffusion models - Minimal code and simple experiments to play with Denoising Diffusion Probabilistic Models (DDPMs)

Minimal code and simple experiments to play with Denoising Diffusion Probabilist

Rithesh Kumar 16 Oct 06, 2022
Subdivision-based Mesh Convolutional Networks

Subdivision-based Mesh Convolutional Networks The official implementation of SubdivNet in our paper, Subdivion-based Mesh Convolutional Networks Requi

Zheng-Ning Liu 181 Dec 28, 2022
Official implementation of "Towards Good Practices for Efficiently Annotating Large-Scale Image Classification Datasets" (CVPR2021)

Towards Good Practices for Efficiently Annotating Large-Scale Image Classification Datasets This is the official implementation of "Towards Good Pract

Sanja Fidler's Lab 52 Nov 22, 2022
Distributionally robust neural networks for group shifts

Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization This code implements the g

151 Dec 25, 2022
An official implementation of "Exploiting a Joint Embedding Space for Generalized Zero-Shot Semantic Segmentation" (ICCV 2021) in PyTorch.

Exploiting a Joint Embedding Space for Generalized Zero-Shot Semantic Segmentation This is an official implementation of the paper "Exploiting a Joint

CV Lab @ Yonsei University 35 Oct 26, 2022
Pytorch code for our paper Beyond ImageNet Attack: Towards Crafting Adversarial Examples for Black-box Domains)

Beyond ImageNet Attack: Towards Crafting Adversarial Examples for Black-box Domains (ICLR'2022) This is the Pytorch code for our paper Beyond ImageNet

Alibaba-AAIG 37 Nov 23, 2022