Source codes of CenterTrack++ in 2021 ICME Workshop on Big Surveillance Data Processing and Analysis

Overview

MOT Tracked object bounding box association (CenterTrack++)

New association method based on CenterTrack. Two new branches (Tracked Size and IOU) are added onto the original CenterTrack tracker. The proposed method enables the computation of IOU distance matrix for more accurate object association compared to single displacement offset in the original CenterTrack.

Modification to CenterTrack method, image modified from CenterTrack

Abstract

The recent development of multi-object tracking (MOT) on point-based joint detection and tracking methods has attracted much research attention. CenterTrack tracking algorithm is one of such promising methods. It achieves state-of-the-art tracking performance using a simple detection model and single-frame spatial offsets to localize objects and predict their associations in a single network. However, this method still suffers from high identity switches due to the inferior association method. Only point displacement distance matrix is used to associate objects, which is not robust to deal with occlusion scenarios. To reduce the high number of identity switches and improve the tracking accuracy, more effective spatial information should be used in association. In this paper, we propose to incorporate a simple tracked object bounding box and overlapping prediction based on the current frame onto the CenterTrack algorithm. Specifically, we propose a Intersection over Union (IOU) distance cost matrix in the association step instead of point displacement distance. We evaluate our proposed tracker on the MOT17 test dataset, showing that our proposed method can reduce identity switches significantly by 22.6% and obtain a notable improvement of 1.5% in IDF1 compared to the original CenterTrack’s under the same tracklet lifetime.

Main Contributions

  • Proposed two branches (tracked box size and IOU)on top of the existing CenterTrack method for IOU distance metric computation in object association
  • Evaluation the proposed method on MOT17 dataset and obtain significant reduction in IDs and notable improvements in tracking accuracy score

Two new branches

The idea of the proposed method is to enhance the original displacement only association. Inspired by the IOU distance in SORT and IOU-Tracker, IOU distance can be used for more accurate object association across frames. IOU distance is calculated as 1 - IOU(bounding box of detected object in the previous frame and the predicted tracked object bounding box in the previous frame based on the current frame)

Tracked Object Size prediction

In order to obtain the IOU distance, the bounding box of the tracked object in the previous frame should be learnt. In this project, two methods were used to learn the tracked bounding box.

Tracking_wh: Directly learn the width and height of the tracked object bounding box in the previous frame.

Tracking_ltrb: Learn the offsets of the left, top, right and bottom of bounding box from the tracked object center in the previous frame.

The tracking_wh(left) and tracking_ltrb(right) approach illustration.

IOU prediction

To further suppress inaccurate association, the IOU value of the tracked object bounding box in adjacent frames is learnt to provide a threshold to filter unlikely associations. We would set the IOU distance to infinity if IOU distance > IOU.

Association Method

Main results

Comparison with other SOTA tracker on MOT17 test set

Note: S= Spatial features, A=appearance features

Tracker Association Features MOTA IDF1 IDs
TubeTK S 63 58.6 4137
CenterTrack S 67.8 64.7 3039
Ours S 68.1 66.2 2352
SST A 52.4 49.5 8431
CTrackerV1 S+A 66.6 57.4 5529
DEFT S+A 66.6 65.4 2823
FairMOT S+A 73.7 72.3 3303

Ablative studies on tracked size prediction method

Tracking_wh

Association Method IDF1 MOTA IDs FP(%) FN(%)
DIS 69.2 66.2 219 3.9 29.5
IOU 71.1 66.7 204 3.6 29.3
Combined 70.9 66.2 233 3.9 29.6
DIS→IOU 70 66.2 218 3.9 29.5
IOU→DIS 69.8 66.8 185 3.6 29.2

Tracking_ltrb

Association Method IDF1 MOTA IDs FP(%) FN(%)
DIS 69.2 66.2 219 3.9 29.5
IOU 72.4 66.7 191 3.8 29.2
Combined 70.8 66.5 236 3.8 29.3
DIS→IOU 70.5 66.6 202 3.8 29.2
IOU→DIS 71.4 66.7 166 3.8 29.2

Installation

Please refer to INSTALL.md for installation instructions.

Training and Evaluation

  • Download the crowdhuman pretrained model from xinyizhou/CenterTrack MODEL ZOO.md to models
  • prepare the data and convert it into COCO format refer to the original CenterTrack repo.
  • change the dataset root directory data_dir in opt.py
  • ablative studies for tracking_wh and tracking_ltrb approach respectively with five association method (IOU,DIS,Combined, IOU→DIS, DIS→IOU)
sh experiments/mot17val_tracking_wh.sh

sh experiments/mot17val_tracking_ltrb.sh

The trained model on MOT17val dataset using two approach are available in google drive, tracking_ltrb_70val.pth, tracking_wh_70val.pth.

  • Train on full mot17 training set and run model on the test set for evaluation
sh experiments/mot17full.sh

The trained models on full MOT17 dataset using ltrb approach is available in the google drive.

Demo comparison

Occlusion case

Original CenterTrack (left) vs CenterTrack++ (right)

Object exiting the frame

Original CenterTrack (left) vs CenterTrack++ (right)

Acknowledgement

A large part of the code is adapted from xingyizhou/CenterTrack, thanks for their wonderful inspiration.

Citation

If you find this paper and code useful in your research, please cite our papers.

@misc{yang2021multiobject,
      title={Multi-object Tracking with Tracked Object Bounding Box Association}, 
      author={Nanyang Yang and Yi Wang and Lap-Pui Chau},
      year={2021},
      eprint={2105.07901},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}
Owner
Nanyang Technological University Information Engineering and Media Student
The Power of Scale for Parameter-Efficient Prompt Tuning

The Power of Scale for Parameter-Efficient Prompt Tuning Implementation of soft embeddings from https://arxiv.org/abs/2104.08691v1 using Pytorch and H

Kip Parker 208 Dec 30, 2022
Perception-aware multi-sensor fusion for 3D LiDAR semantic segmentation (ICCV 2021)

Perception-Aware Multi-Sensor Fusion for 3D LiDAR Semantic Segmentation (ICCV 2021) [中文|EN] 概述 本工作主要探索一种高效的多传感器(激光雷达和摄像头)融合点云语义分割方法。现有的多传感器融合方法主要将点云投影

ICE 126 Dec 30, 2022
PyTorch implementation of PNASNet-5 on ImageNet

PNASNet.pytorch PyTorch implementation of PNASNet-5. Specifically, PyTorch code from this repository is adapted to completely match both my implemetat

Chenxi Liu 314 Nov 25, 2022
This is the code of using DQN to play Sekiro .

Update for using DQN to play sekiro 2021.2.2(English Version) This is the code of using DQN to play Sekiro . I am very glad to tell that I have writen

144 Dec 25, 2022
CVPR2021: Temporal Context Aggregation Network for Temporal Action Proposal Refinement

Temporal Context Aggregation Network - Pytorch This repo holds the pytorch-version codes of paper: "Temporal Context Aggregation Network for Temporal

Zhiwu Qing 63 Sep 27, 2022
CRF-RNN for Semantic Image Segmentation - PyTorch version

This repository contains the official PyTorch implementation of the "CRF-RNN" semantic image segmentation method, published in the ICCV 2015

Sadeep Jayasumana 170 Dec 13, 2022
Codes to pre-train T5 (Text-to-Text Transfer Transformer) models pre-trained on Japanese web texts

t5-japanese Codes to pre-train T5 (Text-to-Text Transfer Transformer) models pre-trained on Japanese web texts. The following is a list of models that

Kimio Kuramitsu 1 Dec 13, 2021
Next-Best-View Estimation based on Deep Reinforcement Learning for Active Object Classification

next_best_view_rl Setup Clone the repository: git clone --recurse-submodules ... In 'third_party/zed-ros-wrapper': git checkout devel Install mujoco `

Christian Korbach 1 Feb 15, 2022
DeepLabv3+:Encoder-Decoder with Atrous Separable Convolution语义分割模型在tensorflow2当中的实现

DeepLabv3+:Encoder-Decoder with Atrous Separable Convolution语义分割模型在tensorflow2当中的实现 目录 性能情况 Performance 所需环境 Environment 注意事项 Attention 文件下载 Download

Bubbliiiing 31 Nov 25, 2022
OpenMMLab Text Detection, Recognition and Understanding Toolbox

Introduction English | 简体中文 MMOCR is an open-source toolbox based on PyTorch and mmdetection for text detection, text recognition, and the correspondi

OpenMMLab 3k Jan 07, 2023
ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detection

ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detection This repository contains implementation of the

Visual Understanding Lab @ Samsung AI Center Moscow 190 Dec 30, 2022
Meandering In Networks of Entities to Reach Verisimilar Answers

MINERVA Meandering In Networks of Entities to Reach Verisimilar Answers Code and models for the paper Go for a Walk and Arrive at the Answer - Reasoni

Shehzaad Dhuliawala 271 Dec 13, 2022
Provide partial dates and retain the date precision through processing

Prefix date parser This is a helper class to parse dates with varied degrees of precision. For example, a data source might state a date as 2001, 2001

Friedrich Lindenberg 13 Dec 14, 2022
Bayesian Optimization using GPflow

Note: This package is for use with GPFlow 1. For Bayesian optimization using GPFlow 2 please see Trieste, a joint effort with Secondmind. GPflowOpt GP

GPflow 257 Dec 26, 2022
Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shift (ICCV 2021)

Π-NAS This repository provides the evaluation code of our submitted paper: Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training

Jiqi Zhang 18 Aug 18, 2022
Pytorch implementation of XRD spectral identification from COD database

XRDidentifier Pytorch implementation of XRD spectral identification from COD database. Details will be explained in the paper to be submitted to NeurI

Masaki Adachi 4 Jan 07, 2023
Bunch of different tools which helps visualizing and annotating images for semantic/instance segmentation tasks

Data Framework for Semantic/Instance Segmentation Bunch of different tools which helps visualizing, transforming and annotating images for semantic/in

Bruno Fernandes Carvalho 5 Dec 21, 2022
Machine Learning Toolkit for Kubernetes

Kubeflow the cloud-native platform for machine learning operations - pipelines, training and deployment. Documentation Please refer to the official do

Kubeflow 12.1k Jan 03, 2023
Self-Learning - Books Papers, Courses & more I have to learn soon

Self-Learning This repository is intended to be used for personal use, all rights reserved to respective owners, please cite original authors and ask

Achint Chaudhary 968 Jan 02, 2022
Functional TensorFlow Implementation of Singular Value Decomposition for paper Fast Graph Learning

tf-fsvd TensorFlow Implementation of Functional Singular Value Decomposition for paper Fast Graph Learning with Unique Optimal Solutions Cite If you f

Sami Abu-El-Haija 14 Nov 25, 2021