The modify PyTorch version of Siam-trackers which are speed-up by TensorRT.

Overview

SiamTracker-with-TensorRT

The modify PyTorch version of Siam-trackers which are speed-up by TensorRT or ONNX. [Updating...]
Examples demonstrating how to optimize PyTorch models with ONNX or TensorRT and do inference on NVIDIA Jetson platforms.

  • All reported speeds contain pre-process process.
Dataset Tracker Platform Origin Success(%)/Speed(fps) TensorRT-FP32 Success(%)/Speed(fps) TensorRT-FP16 Success(%)/Speed(fps) ONNX Success(%)/Speed(fps)
UAV123 LightTrack Jetson AGX Xavier 62.6/15~ 61.7/38~ 61.3/42~ 61.6/36~
  • For fixed image size of 256x256, the performance of ONNX and TensorRT model is not equal to PyTorch model which input size is either of 288x288 or 256x256. If the input size is same, they are equal.

Installation

Prerequisites

The code in this repository was tested on Jetson AGX Xavier DevKits. In order to go futher, first make sure you have the proper version of image (JetPack) installed on the target Jetson system. If not, you can refer to this: Set up Jetson Xavier family , we use JetPack4.6 in this repo.

  • More specifically, the target Jetson system must have ONNX and TensorRT libraries installed. Check which version of TensorRT has been installed on your Jetson system:
    sudo -H pip3 install jetson-stats
    jetson_release -v
  • Install ONNX
    sudo apt-get install protobuf-compiler libprotoc-dev
    pip3 install onnx
  • Install ONNX Runtime. Jetson_Zoo has provided pre-build version on Jetson systems, you can download proper version on this website, more details refer to Set up Jetson Xavier family.
    # Download pip wheel from location mentioned above
    $ wget https://nvidia.box.com/s/bfs688apyvor4eo8sf3y1oqtnarwafww -O onnxruntime_gpu-1.6.0-cp36-cp36m-linux_aarch64.whl
    # Install pip wheel
    $ pip3 install onnxruntime_gpu-1.6.0-cp36-cp36m-linux_aarch64.whl
  • Install pycuda pip3 install pycuda

Convert PyTorch Model to ONNX

  • In LightTrack repo, first modify the loaded PyTorch model path in LightTrack/tracking/lighttrack_onnx.py. To obtain ONNX model:
    python3 tracking/lighttrack_onnx.py

Build engine

  • If the TensorRT libraries has been installed, trtexec exists in /usr/src/tensorrt/bin, then run:
    /usr/src/tensorrt/bin/trtexec --onnx=/path/to/your/onnx/model --saveEngine=LightTrack.trt --fp16

Test TensorRT model

  • In LightTrack repo, modify engine path in lib/models/lighttrack_speed_trt.py, then run:
    python3 tracking/test_lighttrack_trt_uav.py --arch LightTrackM_Subnet --dataset UAV123 --stride 16 --even 0 --path_name back_04502514044521042540+cls_211000022+reg_100000111_ops_32

Evaluate performance

  • Put the results folder under pysot, evaluation please refer to pysot.
Accuracy Aligned. Concise Implementation of Swin Transformer

Accuracy Aligned. Concise Implementation of Swin Transformer This repository contains the implementation of Swin Transformer, and the training codes o

FengWang 77 Dec 16, 2022
Yolo object detection - Yolo object detection with python

How to run download required files make build_image make download Docker versio

3 Jan 26, 2022
MARS: Learning Modality-Agnostic Representation for Scalable Cross-media Retrieva

Introduction This is the source code of our TCSVT 2021 paper "MARS: Learning Modality-Agnostic Representation for Scalable Cross-media Retrieval". Ple

7 Aug 24, 2022
A blender add-on that automatically re-aligns wrong axis objects.

Auto Align A blender add-on that automatically re-aligns wrong axis objects. Usage There are three options available in the 3D Viewport Sidebar It

29 Nov 25, 2022
Mapping Conditional Distributions for Domain Adaptation Under Generalized Target Shift

This repository contains the official code of OSTAR in "Mapping Conditional Distributions for Domain Adaptation Under Generalized Target Shift" (ICLR 2022).

Matthieu Kirchmeyer 5 Dec 06, 2022
Improving Calibration for Long-Tailed Recognition (CVPR2021)

MiSLAS Improving Calibration for Long-Tailed Recognition Authors: Zhisheng Zhong, Jiequan Cui, Shu Liu, Jiaya Jia [arXiv] [slide] [BibTeX] Introductio

DV Lab 116 Dec 20, 2022
FairyTailor: Multimodal Generative Framework for Storytelling

FairyTailor: Multimodal Generative Framework for Storytelling

Eden Bens 172 Dec 30, 2022
This repository contains the implementation of the paper Contrastive Instance Association for 4D Panoptic Segmentation using Sequences of 3D LiDAR Scans

Contrastive Instance Association for 4D Panoptic Segmentation using Sequences of 3D LiDAR Scans This repository contains the implementation of the pap

Photogrammetry & Robotics Bonn 40 Dec 01, 2022
OMNIVORE is a single vision model for many different visual modalities

Omnivore: A Single Model for Many Visual Modalities [paper][website] OMNIVORE is a single vision model for many different visual modalities. It learns

Meta Research 451 Dec 27, 2022
UltraPose: Synthesizing Dense Pose with 1 Billion Points by Human-body Decoupling 3D Model

UltraPose: Synthesizing Dense Pose with 1 Billion Points by Human-body Decoupling 3D Model Official repository for the ICCV 2021 paper: UltraPose: Syn

MomoAILab 92 Dec 21, 2022
Safe Local Motion Planning with Self-Supervised Freespace Forecasting, CVPR 2021

Safe Local Motion Planning with Self-Supervised Freespace Forecasting By Peiyun Hu, Aaron Huang, John Dolan, David Held, and Deva Ramanan Citing us Yo

Peiyun Hu 90 Dec 01, 2022
This project demonstrates the use of neural networks and computer vision to create a classifier that interprets the Brazilian Sign Language.

LIBRAS-Image-Classifier This project demonstrates the use of neural networks and computer vision to create a classifier that interprets the Brazilian

Aryclenio Xavier Barros 26 Oct 14, 2022
PGPortfolio: Policy Gradient Portfolio, the source code of "A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem"(https://arxiv.org/pdf/1706.10059.pdf).

This is the original implementation of our paper, A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem (arXiv:1706.1

Zhengyao Jiang 1.5k Dec 29, 2022
Self-Adaptable Point Processes with Nonparametric Time Decays

NPPDecay This is our implementation for the paper Self-Adaptable Point Processes with Nonparametric Time Decays, by Zhimeng Pan, Zheng Wang, Jeff M. P

zpan 2 Sep 24, 2022
View model summaries in PyTorch!

torchinfo (formerly torch-summary) Torchinfo provides information complementary to what is provided by print(your_model) in PyTorch, similar to Tensor

Tyler Yep 1.5k Jan 05, 2023
EgoNN: Egocentric Neural Network for Point Cloud Based 6DoF Relocalization at the City Scale

EgonNN: Egocentric Neural Network for Point Cloud Based 6DoF Relocalization at the City Scale Paper: EgoNN: Egocentric Neural Network for Point Cloud

19 Sep 20, 2022
CasualHealthcare's Pneumonia detection with Artificial Intelligence (Convolutional Neural Network)

CasualHealthcare's Pneumonia detection with Artificial Intelligence (Convolutional Neural Network) This is PneumoniaDiagnose, an artificially intellig

Azhaan 2 Jan 03, 2022
Powerful and efficient Computer Vision Annotation Tool (CVAT)

Computer Vision Annotation Tool (CVAT) CVAT is free, online, interactive video and image annotation tool for computer vision. It is being used by our

OpenVINO Toolkit 8.6k Jan 01, 2023
Make a Turtlebot3 follow a figure 8 trajectory and create a robot arm and make it follow a trajectory

HW2 - ME 495 Overview Part 1: Makes the robot move in a figure 8 shape. The robot starts moving when launched on a real turtlebot3 and can be paused a

Devesh Bhura 0 Oct 21, 2022
Emulation and Feedback Fuzzing of Firmware with Memory Sanitization

BaseSAFE This repository contains the BaseSAFE Rust APIs, introduced by "BaseSAFE: Baseband SAnitized Fuzzing through Emulation". The example/ directo

Security in Telecommunications 138 Dec 16, 2022