Video Visual Relation Detection (VidVRD) tracklets generation. also for ACM MM Visual Relation Understanding Grand Challenge

Overview

VidVRD-tracklets

This repository contains codes for Video Visual Relation Detection (VidVRD) tracklets generation based on MEGA and deepSORT. These tracklets are also suitable for ACM MM Visual Relation Understanding (VRU) Grand Challenge (which is base on the VidOR dataset).

If you are only interested in the generated tracklets, ​you can ignore these codes and download them directly from here

Download generated tracklets directly

We release the object tracklets for VidOR train/validation/test set. You can download the tracklets here, and put them in the following folder as

├── deepSORT
│   ├── ...
│   ├── tracking_results
│   │   ├── VidORtrain_freq1_m60s0.3_part01
│   │   ├── ...
│   │   ├── VidORtrain_freq1_m60s0.3_part14
│   │   ├── VidORval_freq1_m60s0.3
│   │   ├── VidORtest_freq1_m60s0.3
│   │   ├── readme.md
│   │   └── format_demo.py
│   └── ...
├── MEGA
│   ├── ... 
│   └── ...

Please refer to deepSORT/tracking_results/readme.md for more details

Evaluate the tracklets mAP

Run python deepSORT/eval_traj_mAP.py to evaluate the tracklets mAP. (you might need to change some args in deepSORT/eval_traj_mAP.py)

Generate object tracklets by yourself

The object tracklets generation pipeline mainly consists of two parts: MEGA (for video object detection), and deepSORT (for video object tracking).

Quick Start

  1. Install MEGA as the official instructions MEGA/INSTALL.md (Note that the folder path may be different when installing).

    • If you have any trouble when installing MEGA, you can try to clone the official MEGA repository and install it, and then replace the official mega.pytorch/mega_core with our modified MEGA/mega_core. Refer to MEGA/modification_details.md for the details of our modifications.
  2. Download the VidOR dataset and the pre-trained weight of MEGA. Put them in the following folder as

├── deepSORT/
│   ├── ...
├── MEGA/
│   ├── ... 
│   ├── datasets/
│   │   ├── COCOdataset/        # used for MEGA training
│   │   ├── COCOinVidOR/        # used for MEGA training
│   │   ├── vidor-dataset/
│   │   │   ├── annotation/
│   │   │   │   ├── training/
│   │   │   │   └── validation/
│   │   │   ├── img_index/ 
│   │   │   │   ├── VidORval_freq1_0024.txt
│   │   │   │   ├── ...
│   │   │   ├── val_frames/
│   │   │   │   ├── 0001_2793806282/
│   │   │   │   │   ├── 000000.JPEG
│   │   │   │   │   ├── ...
│   │   │   │   ├── ...
│   │   │   ├── val_videos/
│   │   │   │   ├── 0001/
│   │   │   │   │   ├── 2793806282.mp4
│   │   │   │   │   ├── ...
│   │   │   │   ├── ...
│   │   │   ├── train_frames/
│   │   │   ├── train_videos/
│   │   │   ├── test_frames/
│   │   │   ├── test_videos/
│   │   │   └── video2img_vidor.py
│   │   └── construct_img_idx.py
│   ├── training_dir/
│   │   ├── COCO34ORfreq32_4gpu/
│   │   │   ├── inference/
│   │   │   │   ├── VidORval_freq1_0024/
│   │   │   │   │   ├── predictions.pth
│   │   │   │   │   └── result.txt
│   │   │   │   ├── ...
│   │   │   └── model_0180000.pth
│   │   ├── ...
  1. Run python MEGA/datasets/vidor-dataset/video2img_vidor.py (note that you may need to change some args) to extract frames from videos (This causes a lot of data redundancy, but we have to do this, because MEGA takes image data as input).

  2. Run python MEGA/datasets/construct_img_idx.py (note that you may need to change some args) to generate the img_index used in MEGA inference.

    • The generated .txt files will be saved in MEGA/datasets/vidor-dataset/img_index/. You can use VidORval_freq1_0024.txt as a demo for the following commands.
  3. Run the following command to detect frame-level object proposals with bbox features (RoI pooled features).

    CUDA_VISIBLE_DEVICES=0   python  \
        MEGA/tools/test_net.py \
        --config-file MEGA/configs/MEGA/inference/VidORval_freq1_0024.yaml \
        MODEL.WEIGHT MEGA/training_dir/COCO34ORfreq32_4gpu/model_0180000.pth \
        OUTPUT_DIR MEGA/training_dir/COCO34ORfreq32_4gpu/inference
    
    • The above command will generate a predictions.pth file for this VidORval_freq1_0024 demo. We also release this predictions.pth here.

    • the config files for VidOR train set are in MEGA/configs/MEGA/partxx

    • The predictions.pth contains frame-level box positions and features (RoI features) for each object. For RoI features, they can be accessed through roifeats = boxlist.get_field("roi_feats"), if you are familiar with MEGA or maskrcnn-benchmark

  4. Run python MEGA/mega_boxfeatures/cvt_proposal_result.py (note that you may need to change some args) to convert predictions.pth to a .pkl file for the following deepSORT stage.

    • We also provide VidORval_freq1_0024.pkl here
  5. Run python deepSORT/deepSORT_tracking_v2.py (note that you may need to change some args) to perform deepSORT tracking. The results will be saved in deepSORT/tracking_results/

Train MEGA for VidOR by yourself

  1. Download MS-COCO and put them as shown in above.

  2. Run python MEGA/tools/extract_coco.py to extract annotations for COCO in VidOR, which results in COCO_train_34classes.pkl and COCO_valmini_34classes.pkl

  3. train MEGA by the following commands:

    python -m torch.distributed.launch \
        --nproc_per_node=4 \
        tools/train_net.py \
        --master_port=$((RANDOM + 10000)) \
        --config-file MEGA/configs/MEGA/vidor_R_101_C4_MEGA_1x_4gpu.yaml \
        OUTPUT_DIR MEGA/training_dir/COCO34ORfreq32_4gpu

More detailed training instructions will be updated soon...

Django application and library for importing and exporting data with admin integration.

django-import-export django-import-export is a Django application and library for importing and exporting data with included admin integration. Featur

2.6k Jan 07, 2023
Allow foreign key attributes in list_display with '__'

django-related-admin Allow foreign key attributes in Django admin change list list_display with '__' This is based on DjangoSnippet 2996 which was mad

Petr Dlouhý 62 Nov 18, 2022
:honey_pot: A fake Django admin login screen page.

django-admin-honeypot django-admin-honeypot is a fake Django admin login screen to log and notify admins of attempted unauthorized access. This app wa

Derek Payton 907 Dec 31, 2022
Ajenti Core and stock plugins

Ajenti is a Linux & BSD modular server admin panel. Ajenti 2 provides a new interface and a better architecture, developed with Python3 and AngularJS.

Ajenti Project 7k Jan 07, 2023
spider-admin-pro

Spider Admin Pro Github: https://github.com/mouday/spider-admin-pro Gitee: https://gitee.com/mouday/spider-admin-pro Pypi: https://pypi.org/

mouday 289 Jan 06, 2023
Extendable, adaptable rewrite of django.contrib.admin

django-admin2 One of the most useful parts of django.contrib.admin is the ability to configure various views that touch and alter data. django-admin2

Jazzband 1.2k Dec 29, 2022
Extends the Django Admin to include a extensible dashboard and navigation menu

django-admin-tools django-admin-tools is a collection of extensions/tools for the default django administration interface, it includes: a full feature

Django Admin Tools 731 Dec 28, 2022
Drop-in replacement of Django admin comes with lots of goodies, fully extensible with plugin support, pretty UI based on Twitter Bootstrap.

Xadmin Drop-in replacement of Django admin comes with lots of goodies, fully extensible with plugin support, pretty UI based on Twitter Bootstrap. Liv

差沙 4.7k Dec 31, 2022
Python books free to read online or download

Python books free to read online or download

Paolo Amoroso 3.7k Jan 08, 2023
WebVirtCloud is virtualization web interface for admins and users

WebVirtCloud is a virtualization web interface for admins and users. It can delegate Virtual Machine's to users. A noVNC viewer presents a full graphical console to the guest domain. KVM is currently

Anatoliy Guskov 1.3k Dec 29, 2022
Simple and extensible administrative interface framework for Flask

Flask-Admin The project was recently moved into its own organization. Please update your references to Flask-Admin 5.2k Dec 29, 2022

Legacy django jet rebooted , supports only Django 3

Django JET Reboot Rebooting the original project : django-jet. Django Jet is modern template for Django admin interface with improved functionality. W

215 Dec 31, 2022
BitcartCC is a platform for merchants, users and developers which offers easy setup and use.

BitcartCC is a platform for merchants, users and developers which offers easy setup and use.

BitcartCC 270 Jan 07, 2023
Responsive Theme for Django Admin With Sidebar Menu

Responsive Django Admin If you're looking for a version compatible with Django 1.8 just install 0.3.7.1. Features Responsive Sidebar Menu Easy install

Douglas Miranda 852 Dec 02, 2022
Helpers to extend Django Admin with data from external service with minimal hacks

django-admin-data-from-external-service Helpers to extend Django Admin with data from external service with minimal hacks Live demo with sources on He

Evgeniy Tatarkin 7 Apr 27, 2022
AaPanel - Simple but Powerful web-based Control Panel

Introduction: aaPanel is the International version for BAOTA panel(www.bt.cn) There have millions servers had installed BAOTA panel since 2014 in Chin

bt.cn 1.4k Jan 09, 2023
A Django app for easily adding object tools in the Django admin

Django Object Actions If you've ever tried making admin object tools you may have thought, "why can't this be as easy as making Django Admin Actions?"

Chris Chang 524 Dec 26, 2022
Python Crypto Bot

Python Crypto Bot

Michael Whittle 1.6k Jan 06, 2023
Jinja is a fast, expressive, extensible templating engine.

Jinja is a fast, expressive, extensible templating engine. Special placeholders in the template allow writing code similar to Python syntax.

The Pallets Projects 9k Jan 04, 2023
StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery

StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery

3.3k Jan 01, 2023