This is the official implementation of the paper "Object Propagation via Inter-Frame Attentions for Temporally Stable Video Instance Segmentation".

Last update: May 03, 2022

Related tags

Deep Learning ObjProp

Overview

ObjProp

Introduction

This is the official implementation of the paper "Object Propagation via Inter-Frame Attentions for Temporally Stable Video Instance Segmentation".

Installation

This repo is built using mmdetection. To install the dependencies, first clone the repository locally:

git clone https://github.com/anirudh-chakravarthy/objprop.git

Then, install PyTorch 1.1.0, torchvision 0.3.0, mmcv 0.2.12:

conda install pytorch==1.1.0 torchvision==0.3.0 -c pytorch
pip install mmcv==0.2.12

Then, install the CocoAPI for YouTube-VIS

conda install cython scipy
pip install git+https://github.com/youtubevos/cocoapi.git#"egg=pycocotools&subdirectory=PythonAPI"

Training

First, download and prepare the YouTube-VIS dataset using the following instructions.

To train ObjProp, run the following command:

python3 tools/train.py configs/masktrack_rcnn_r50_fpn_1x_youtubevos_objprop.py

In order to change the arguments such as dataset directory, learning rate, number of GPUs, etc, refer to the following configuration file configs/masktrack_rcnn_r50_fpn_1x_youtubevos_objprop.py.

Inference

To perform inference using ObjProp, run the following command:

python3 tools/test_video.py configs/masktrack_rcnn_r50_fpn_1x_youtubevos_objprop.py [MODEL_PATH] --out [OUTPUT_PATH.json] --eval segm

A JSON file with the inference results will be saved at OUTPUT_PATH.json. To evaluate the performance, submit the result file to the evaluation server.

License

ObjProp is released under the Apache 2.0 license.

Citation

@article{Chakravarthy2021ObjProp,
  author = {Anirudh S Chakravarthy and Won-Dong Jang and Zudi Lin and Donglai Wei and Song Bai and Hanspeter Pfister},  
  title = {Object Propagation via Inter-Frame Attentions for Temporally Stable Video Instance Segmentation},
  journal = {CoRR},
  volume = {abs/2111.07529},
  year = {2021},
  url = {https://arxiv.org/abs/2111.07529}
}

This is the official implementation of the paper "Object Propagation via Inter-Frame Attentions for Temporally Stable Video Instance Segmentation".

Related tags

Overview

ObjProp

Introduction

Installation

Training

Inference

License

Citation

Owner

Anirudh S Chakravarthy

Oriented Object Detection: Oriented RepPoints + Swin Transformer/ReResNet

Code accompanying "Evolving spiking neuron cellular automata and networks to emulate in vitro neuronal activity," accepted to IEEE SSCI ICES 2021

How to Leverage Multimodal EHR Data for Better Medical Predictions?

Custom TensorFlow2 implementations of forward and backward computation of soft-DTW algorithm in batch mode.

CvT2DistilGPT2 is an encoder-to-decoder model that was developed for chest X-ray report generation.

Sequence Modeling with Structured State Spaces

mPose3D, a mmWave-based 3D human pose estimation model.

A PyTorch implementation of Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks

CNNs for Sentence Classification in PyTorch

Image-Stitching - Panorama composition using SIFT Features and a custom implementaion of RANSAC algorithm

Scrutinizing XAI with linear ground-truth data

Le dataset des images du projet d'IA de 2021

《A-CNN: Annularly Convolutional Neural Networks on Point Clouds》(2019)

Use your Philips Hue lights as Racing Flags. Works with Assetto Corsa, Assetto Corsa Competizione and iRacing.

This repository contains the implementation of the following paper: Cross-Descriptor Visual Localization and Mapping

[ICCV 2021] FaPN: Feature-aligned Pyramid Network for Dense Image Prediction

RNG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answering

ZeroVL - The official implementation of ZeroVL

Scripts for training an AI to play the endless runner Subway Surfers using a supervised machine learning approach by imitation and a convolutional neural network (CNN) for image classification

Match SafeGraph POIs with Data collected through a cultural resource survey in Washington DC.