PyTorch 1.0 inference in C++ on Windows10 platforms

Related tags

Deep Learningpytorch-inference
Overview

Serving PyTorch Models in C++ on Windows10 platforms

Dynamic graph

How to use

Prepare Data

examples/data/train/

	- 0
	- 1
	  .
	  .
	  .
	- n

examples/data/test/

	- 0
	- 1
	  .
	  .
	  .
	- n

Train Model

cd examples && python train.py

Transform Model

cd examples && python transform_model.py

Test Model

cd makefile/pytorch
mkdir build && cd build && cmake -A x64 ..

or

mkdir build && cd build && cmake -G "Visual Studio 15 2017 Win64" ..

set Command Arguments -> ..\..\..\examples\checkpoint ..\..\..\examples\images
set Environment -> path=%path%;../../../thirdparty/libtorch/lib;../../../thirdparty/opencv/build/x64/vc15/bin;

Test CUDA Softmax

cd makefile/cuda
mkdir build && cd build && cmake -A x64 ..

or

mkdir build && cd build && cmake -G "Visual Studio 15 2017 Win64" ..

Inference onnx model

cd makefile/tensorRT/classification
mkdir build && cd build && cmake -A x64 ..

or

mkdir build && cd build && cmake -G "Visual Studio 15 2017 Win64" ..
set Environment -> path=%path%;../../../../thirdparty/TensorRT/lib;

Inference caffe model for faster-rcnn

cd makefile/tensorRT/detection
mkdir build && cd build && cmake -A x64 ..

or

mkdir build && cd build && cmake -G "Visual Studio 15 2017 Win64" ..
set Environment -> path=%path%;../../../../thirdparty/TensorRT/lib;

download VGG16_faster_rcnn_final.caffemodel

Thirdparty

thirdparty/
	- libtorch  
	- opencv 
	- CUDA
	- TensorRT

download thirdparty from here.

Docker

docker pull zccyman/deepframe
nvidia-docker run -it --name=mydocker zccyman/deepframe /bin/bash
cd workspace && git clone https://github.com/zccyman/pytorch-inference.git

Environment

  • Windows10
  • VS2017
  • CMake3.13
  • CUDA10.0
  • CUDNN7.3
  • Pyton3.5
  • ONNX1.1.2
  • TensorRT5.0.1
  • Pytorch1.0
  • Libtorch
  • OpenCV4.0.1

Todo List

  • train and transform pytorch model

  • multi-batch inference pytorch model in C++

  • cpu and gpu softmax

  • transform pytorch model to ONNX model, and inference onnx model using tensorRT

  • inference caffe model for faster-rcnn using tensorRT

  • build classification network

  • compress pytorch model

  • object detection pytorch inference using C++ on Window platforms

Notes

  • "torch.jit.trace" doesn’t support nn.DataParallel so far.

  • TensorRT doesn’t supports opset 7 above so far, but pyTorch ONNX exporter seems to export opset 9.

Acknowledgement

Owner
Henson
Henson
GitHub Repository
VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioning

VisualGPT Our Paper VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioning Main Architecture of Our VisualGPT Downloa

Vision CAIR Research Group, KAUST 140 Dec 28, 2022
Simulation of Self Driving Car

In this repository, the code to use Udacity's self driving car simulator as a testbed for training an autonomous car are provided.

Shyam Das Shrestha 1 Nov 21, 2021
Evaluation framework for testing segmentation networks in PyTorch

Evaluation framework for testing segmentation networks in PyTorch. What segmentation network to choose for next Kaggle competition? This benchmark knows the answer!

Eugene Khvedchenya 37 Apr 27, 2022
SAAVN - Sound Adversarial Audio-Visual Navigation,ICLR2022 (In PyTorch)

SAAVN SAAVN Code release for paper "Sound Adversarial Audio-Visual Navigation,IC

YinfengYu 10 Aug 30, 2022
Official code of our work, AVATAR: A Parallel Corpus for Java-Python Program Translation.

AVATAR Official code of our work, AVATAR: A Parallel Corpus for Java-Python Program Translation. AVATAR stands for jAVA-pyThon progrAm tRanslation. AV

Wasi Ahmad 26 Dec 03, 2022
pytorch implementation for Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network arXiv:1609.04802

PyTorch SRResNet Implementation of Paper: "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network"(https://arxiv.org/abs

Jiu XU 436 Jan 09, 2023
Cancer Drug Response Prediction via a Hybrid Graph Convolutional Network

DeepCDR Cancer Drug Response Prediction via a Hybrid Graph Convolutional Network This work has been accepted to ECCB2020 and was also published in the

Qiao Liu 50 Dec 18, 2022
Tensor-based approaches for fMRI classification

tensor-fmri Using tensor-based approaches to classify fMRI data from StarPLUS. Citation If you use any code in this repository, please cite the follow

4 Sep 07, 2022
Source code of the paper "Deep Learning of Latent Variable Models for Industrial Process Monitoring".

Source code of the paper "Deep Learning of Latent Variable Models for Industrial Process Monitoring".

Xiangyin Kong 7 Nov 08, 2022
This is the pytorch implementation for the paper: Generalizable Mixed-Precision Quantization via Attribution Rank Preservation, which is accepted to ICCV2021.

GMPQ: Generalizable Mixed-Precision Quantization via Attribution Rank Preservation This is the pytorch implementation for the paper: Generalizable Mix

18 Sep 02, 2022
Pytorch implementation of forward and inverse Haar Wavelets 2D

Pytorch implementation of forward and inverse Haar Wavelets 2D

Sergei Belousov 9 Oct 30, 2022
Graph Representation Learning via Graphical Mutual Information Maximization

GMI (Graphical Mutual Information) Graph Representation Learning via Graphical Mutual Information Maximization (Peng Z, Huang W, Luo M, et al., WWW 20

93 Dec 29, 2022
City-Scale Multi-Camera Vehicle Tracking Guided by Crossroad Zones Code

City-Scale Multi-Camera Vehicle Tracking Guided by Crossroad Zones Requirements Python 3.8 or later with all requirements.txt dependencies installed,

88 Dec 12, 2022
PyTorch Implementation of Backbone of PicoDet

PicoDet-Backbone PyTorch Implementation of Backbone of PicoDet Original Implementation is implemented on PaddlePaddle. Example picodet_l_backbone = ES

Yonghye Kwon 7 Jul 12, 2022
Equivariant Imaging: Learning Beyond the Range Space

[Project] Equivariant Imaging: Learning Beyond the Range Space Project about the

Georges Le Bellier 3 Feb 06, 2022
TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.

TensorFlowOnSpark TensorFlowOnSpark brings scalable deep learning to Apache Hadoop and Apache Spark clusters. By combining salient features from the T

Yahoo 3.8k Jan 04, 2023
A hue shift helper for OBS

obs-hue-shift A hue shift helper for OBS This is a repo based on the really nice script Hegemege made. The original script can be found https://gist.g

Alexis Tyler 1 Jan 10, 2022
This is the pytorch implementation for the paper: *Learning Accurate Performance Predictors for Ultrafast Automated Model Compression*, which is in submission to TPAMI

SeerNet This is the pytorch implementation for the paper: Learning Accurate Performance Predictors for Ultrafast Automated Model Compression, which is

3 May 01, 2022
Unofficial PyTorch implementation of TokenLearner by Google AI

tokenlearner-pytorch Unofficial PyTorch implementation of TokenLearner by Ryoo et al. from Google AI (abs, pdf) Installation You can install TokenLear

Rishabh Anand 46 Dec 20, 2022
Source-to-Source Debuggable Derivatives in Pure Python

Tangent Tangent is a new, free, and open-source Python library for automatic differentiation. Existing libraries implement automatic differentiation b

Google 2.2k Jan 01, 2023