Here I will explain the flow to deploy your custom deep learning models on Ultra96V2.

Overview

Xilinx_Vitis_AI

This repo will help you to Deploy your Deep Learning Model on Ultra96v2 Board.


Prerequisites

  1. Vitis Core Development Kit 2019.2

This could be downloaded from here: Link to the websire

  1. Vitis-AI GitHub Repository v1.1

Here is the link to the repository v1.1

  1. Vitis-Ai Docker Container

The command to pull the container: docker pull xilinx/vitis-ai:1.1.56

  1. XRT 2019.2

GitHub Repo Link 2019.2

  1. Avnet Vitis Platform 2019.2

Here is the link to download the zip file Avnet Website

  1. Ubuntu OS 18.04

Once the tools have been setup, there are five (5) main steps to targeting an AI applications to Ultra96V2 Platform:

  1. Build the Hardware Design
  2. Compile Your Custom Model
  3. Build the AI Applications
  4. Create the SD Card Content
  5. Execute the AI Applications on hardware

Supposed that you have trained your model previously in one of the Tensorflow (.Pb), Caffe(.Caffemodel and .Prototxt) and Darknet(.Weights and .Cfg) Frameworks.

Build the Hardware Design

Clone Xilinx’s Vitis-AI github repository:

$ git clone --branch v1.1 https://github.com/Xilinx/Vitis-AI
$ cd Vitis-AI
$ export VITIS_AI_HOME = "$PWD"

Install the Avnet Vitis platform:>

Download this and extract to the hard drive of your linux machine. Then, specify the location of the Vitis platform, by creating the SDX_PLATFORM environment variable that specified to the location of the.xpfm file.

$ export SDX_PLATFORM=/home/Avnet/vitis/platform_repo/ULTRA96V2/ULTRA96V2.xpfm

Build the Hardware Project (SD Card Image)

I suggest you to download the Pre-Built from here

Compile the Trained Models

Remember that you should have pulled the docker container first.

Caffe Models:

$ cd $VITIS_AI_HOME
$ mkdir project
$ cp PATH/TO/TRAINED/MODELS  $VITIS_AI_HOME/project
$ ./docker_run.sh xilinx/vitis-ai:1.1.56
$ cd project
$ conda activate vitis-ai-caffe
$ vai_q_caffe quantize -model float.prototxt -weights float.caffemodel -calib_iter 5
$ vai_c_caffe -p .PROTOTXT -c .CAFFEMODEL -a ARCH.JSON -o OUTPUT_DIR -n NET_NAME 

Tensorflow Models:

$ cd $VITIS_AI_HOME
$ mkdir project
$ cp PATH/TO/TRAINED/MODELS  $VITIS_AI_HOME/project
$ ./docker_run.sh xilinx/vitis-ai:1.1.56
$ cd project
$ conda activate vitis-ai-tensorflow
$ vai_q_tensorflow quantize --input_frozen_graph FROZEN_PB --input_nodes xxx --output_nodes yyy --input_shapes zzz --input_fn module.calib_input --calib_iter 5
$ vai_c_tensorflow -f FROZEN_PB -a ARCH.JSON -o OUTPUT_DIR -n NET_NAME 

Compile the AI Application Using DNNDK APIs

The DNNDK API is the low-level API used to communicate with the AI engine (DPU). This API is the recommended API for users that will be creating their own custom neural networks.

Download and install the SDK for cross-compilation, specifying a unique and meaningful installation destination (knowing that this SDK will be specific to the Vitis-AI 1.1 DNNDK samples):

$ wget -O sdk.sh https://www.xilinx.com/bin/public/openDownload?filename=sdk.sh
$ chmod +x sdk.sh
$ ./sdk.sh -d ~/petalinux_sdk_vai_1_1_dnndk 

Setup the environment for cross-compilation:

$ unset LD_LIBRARY_PATH
$ source ~/petalinux_sdk_vai_1_1_dnndk/environment-setup-aarch64-xilinx-linux

Download and extract the DNNDK runtime examples and Install the additional DNNDK runtime content:

$ wget -O vitis-ai_v1.1_dnndk.tar.gz  https://www.xilinx.com/bin/public/openDownload?filename=vitis-ai_v1.1_dnndk.tar.gz
$ tar -xvzf vitis-ai-v1.1_dnndk.tar.gz
$ cd vitis-ai-v1.1_dnndk
$ ./install.sh $SDKTARGETSYSROOT

Copy the Compiled project:

$ cp -r ../project/ .

Download and extract the additional content (images and video files) for the DNNDK examples:

$ wget -O vitis-ai_v1.1_dnndk_sample_img.tar.gz https://www.xilinx.com/bin/public/openDownload?filename=vitis-ai_v1.1_dnndk_sample_img.tar.gz
$ tar -xvzf vitis-ai_v1.1_dnndk_sample_img.tar.gz

For the custom application (project folder), create a model directory and copy the dpu_*.elf model files you previously built:

$ cd $VITIS_AI_HOME/project
$ mkdir model_for_ultra96v2
$ cp -r model_for_ultra96v2 model
$ make

NOTE: You could also edit the build.sh script to add support for the new Platforms like Ultra96V2.

Execute the AI Application on ULTRA96V2

  1. Boot the Ultra96V2 with the pre-build sd-card image you dowloaded. For Learning How to Do This, Click HERE!
  2. $ cd /run/media/mmcblk0p1
  3. $ cp dpu.xclbin /usr/lib/.
  4. Install the Vitis-AI embedded package:
$ cd runtime/vitis-ai_v1.1_dnndk 
$ source ./install.sh
  1. Define the DISPLAY environment variable:
$ export DISPLAY=:0.0
$ xrandr --output DP-1 --mode 640x480
  1. Run the Custom Application:
 $ cd vitis_ai_dnndk_samples
 $ ./App 
Owner
Amin Mamandipoor
Currently, Studying Master of Computer Systems Architecture at the University of Tabriz.
Amin Mamandipoor
Pytorch implementation of NEGEV method. Paper: "Negative Evidence Matters in Interpretable Histology Image Classification".

Pytorch 1.10.0 code for: Negative Evidence Matters in Interpretable Histology Image Classification (https://arxiv. org/abs/xxxx.xxxxx) Citation: @arti

Soufiane Belharbi 4 Dec 01, 2022
Adversarial vulnerability of powerful near out-of-distribution detection

Adversarial vulnerability of powerful near out-of-distribution detection by Stanislav Fort In this repository we're collecting replications for the ke

Stanislav Fort 9 Aug 30, 2022
ECCV18 Workshops - Enhanced SRGAN. Champion PIRM Challenge on Perceptual Super-Resolution. The training codes are in BasicSR.

ESRGAN (Enhanced SRGAN) [ 🚀 BasicSR] [Real-ESRGAN] ✨ New Updates. We have extended ESRGAN to Real-ESRGAN, which is a more practical algorithm for rea

Xintao 4.7k Jan 02, 2023
Dogs classification with Deep Metric Learning using some popular losses

Tsinghua Dogs classification with Deep Metric Learning 1. Introduction Tsinghua Dogs dataset Tsinghua Dogs is a fine-grained classification dataset fo

QuocThangNguyen 45 Nov 09, 2022
Source code for our CVPR 2019 paper - PPGNet: Learning Point-Pair Graph for Line Segment Detection

PPGNet: Learning Point-Pair Graph for Line Segment Detection PyTorch implementation of our CVPR 2019 paper: PPGNet: Learning Point-Pair Graph for Line

SVIP Lab 170 Oct 25, 2022
Official repository for the paper F, B, Alpha Matting

FBA Matting Official repository for the paper F, B, Alpha Matting. This paper and project is under heavy revision for peer reviewed publication, and s

Marco Forte 404 Jan 05, 2023
A High-Performance Distributed Library for Large-Scale Bundle Adjustment

MegBA: A High-Performance and Distributed Library for Large-Scale Bundle Adjustment This repo contains an official implementation of MegBA. MegBA is a

旷视研究院 3D 组 336 Dec 27, 2022
GAN-based Matrix Factorization for Recommender Systems

GAN-based Matrix Factorization for Recommender Systems This repository contains the datasets' splits, the source code of the experiments and their res

Ervin Dervishaj 9 Nov 06, 2022
Контрольная работа по математическим методам машинного обучения

ML-MathMethods-Test Контрольная работа по математическим методам машинного обучения. Вычисление основных статистик, диаграмм и графиков, проверка разл

Stas Ivanovskii 1 Jan 06, 2022
This repository gives an example on how to preprocess the data of the HECKTOR challenge

HECKTOR 2021 challenge This repository gives an example on how to preprocess the data of the HECKTOR challenge. Any other preprocessing is welcomed an

56 Dec 01, 2022
[CVPR 2021] Released code for Counterfactual Zero-Shot and Open-Set Visual Recognition

Counterfactual Zero-Shot and Open-Set Visual Recognition This project provides implementations for our CVPR 2021 paper Counterfactual Zero-S

144 Dec 24, 2022
BOVText: A Large-Scale, Multidimensional Multilingual Dataset for Video Text Spotting

BOVText: A Large-Scale, Bilingual Open World Dataset for Video Text Spotting Updated on December 10, 2021 (Release all dataset(2021 videos)) Updated o

weijiawu 47 Dec 26, 2022
Official release of MSHT: Multi-stage Hybrid Transformer for the ROSE Image Analysis of Pancreatic Cancer axriv: http://arxiv.org/abs/2112.13513

MSHT: Multi-stage Hybrid Transformer for the ROSE Image Analysis This is the official page of the MSHT with its experimental script and records. We de

Tianyi Zhang 53 Dec 27, 2022
AirPose: Multi-View Fusion Network for Aerial 3D Human Pose and Shape Estimation

AirPose AirPose: Multi-View Fusion Network for Aerial 3D Human Pose and Shape Estimation Check the teaser video This repository contains the code of A

Robot Perception Group 41 Dec 05, 2022
Tutorial repo for an end-to-end Data Science project

End-to-end Data Science project This is the repo with the notebooks, code, and additional material used in the ITI's workshop. The goal of the session

Deena Gergis 127 Dec 30, 2022
A-ESRGAN aims to provide better super-resolution images by using multi-scale attention U-net discriminators.

A-ESRGAN: Training Real-World Blind Super-Resolution with Attention-based U-net Discriminators The authors are hidden for the purpose of double blind

77 Dec 16, 2022
The code of Zero-shot learning for low-light image enhancement based on dual iteration

Zero-shot-dual-iter-LLE The code of Zero-shot learning for low-light image enhancement based on dual iteration. You can get the real night image tests

1 Mar 18, 2022
Fine-Tune EleutherAI GPT-Neo to Generate Netflix Movie Descriptions in Only 47 Lines of Code Using Hugginface And DeepSpeed

GPT-Neo-2.7B Fine-Tuning Example Using HuggingFace & DeepSpeed Installation cd venv/bin ./pip install -r ../../requirements.txt ./pip install deepspe

Nikita 180 Jan 05, 2023
Benchmarks for Model-Based Optimization

Design-Bench Design-Bench is a benchmarking framework for solving automatic design problems that involve choosing an input that maximizes a black-box

Brandon Trabucco 43 Dec 20, 2022
Semi-supervised learning for object detection

Source code for STAC: A Simple Semi-Supervised Learning Framework for Object Detection STAC is a simple yet effective SSL framework for visual object

Google Research 348 Dec 25, 2022