Here I will explain the flow to deploy your custom deep learning models on Ultra96V2.

Overview

Xilinx_Vitis_AI

This repo will help you to Deploy your Deep Learning Model on Ultra96v2 Board.


Prerequisites

  1. Vitis Core Development Kit 2019.2

This could be downloaded from here: Link to the websire

  1. Vitis-AI GitHub Repository v1.1

Here is the link to the repository v1.1

  1. Vitis-Ai Docker Container

The command to pull the container: docker pull xilinx/vitis-ai:1.1.56

  1. XRT 2019.2

GitHub Repo Link 2019.2

  1. Avnet Vitis Platform 2019.2

Here is the link to download the zip file Avnet Website

  1. Ubuntu OS 18.04

Once the tools have been setup, there are five (5) main steps to targeting an AI applications to Ultra96V2 Platform:

  1. Build the Hardware Design
  2. Compile Your Custom Model
  3. Build the AI Applications
  4. Create the SD Card Content
  5. Execute the AI Applications on hardware

Supposed that you have trained your model previously in one of the Tensorflow (.Pb), Caffe(.Caffemodel and .Prototxt) and Darknet(.Weights and .Cfg) Frameworks.

Build the Hardware Design

Clone Xilinx’s Vitis-AI github repository:

$ git clone --branch v1.1 https://github.com/Xilinx/Vitis-AI
$ cd Vitis-AI
$ export VITIS_AI_HOME = "$PWD"

Install the Avnet Vitis platform:>

Download this and extract to the hard drive of your linux machine. Then, specify the location of the Vitis platform, by creating the SDX_PLATFORM environment variable that specified to the location of the.xpfm file.

$ export SDX_PLATFORM=/home/Avnet/vitis/platform_repo/ULTRA96V2/ULTRA96V2.xpfm

Build the Hardware Project (SD Card Image)

I suggest you to download the Pre-Built from here

Compile the Trained Models

Remember that you should have pulled the docker container first.

Caffe Models:

$ cd $VITIS_AI_HOME
$ mkdir project
$ cp PATH/TO/TRAINED/MODELS  $VITIS_AI_HOME/project
$ ./docker_run.sh xilinx/vitis-ai:1.1.56
$ cd project
$ conda activate vitis-ai-caffe
$ vai_q_caffe quantize -model float.prototxt -weights float.caffemodel -calib_iter 5
$ vai_c_caffe -p .PROTOTXT -c .CAFFEMODEL -a ARCH.JSON -o OUTPUT_DIR -n NET_NAME 

Tensorflow Models:

$ cd $VITIS_AI_HOME
$ mkdir project
$ cp PATH/TO/TRAINED/MODELS  $VITIS_AI_HOME/project
$ ./docker_run.sh xilinx/vitis-ai:1.1.56
$ cd project
$ conda activate vitis-ai-tensorflow
$ vai_q_tensorflow quantize --input_frozen_graph FROZEN_PB --input_nodes xxx --output_nodes yyy --input_shapes zzz --input_fn module.calib_input --calib_iter 5
$ vai_c_tensorflow -f FROZEN_PB -a ARCH.JSON -o OUTPUT_DIR -n NET_NAME 

Compile the AI Application Using DNNDK APIs

The DNNDK API is the low-level API used to communicate with the AI engine (DPU). This API is the recommended API for users that will be creating their own custom neural networks.

Download and install the SDK for cross-compilation, specifying a unique and meaningful installation destination (knowing that this SDK will be specific to the Vitis-AI 1.1 DNNDK samples):

$ wget -O sdk.sh https://www.xilinx.com/bin/public/openDownload?filename=sdk.sh
$ chmod +x sdk.sh
$ ./sdk.sh -d ~/petalinux_sdk_vai_1_1_dnndk 

Setup the environment for cross-compilation:

$ unset LD_LIBRARY_PATH
$ source ~/petalinux_sdk_vai_1_1_dnndk/environment-setup-aarch64-xilinx-linux

Download and extract the DNNDK runtime examples and Install the additional DNNDK runtime content:

$ wget -O vitis-ai_v1.1_dnndk.tar.gz  https://www.xilinx.com/bin/public/openDownload?filename=vitis-ai_v1.1_dnndk.tar.gz
$ tar -xvzf vitis-ai-v1.1_dnndk.tar.gz
$ cd vitis-ai-v1.1_dnndk
$ ./install.sh $SDKTARGETSYSROOT

Copy the Compiled project:

$ cp -r ../project/ .

Download and extract the additional content (images and video files) for the DNNDK examples:

$ wget -O vitis-ai_v1.1_dnndk_sample_img.tar.gz https://www.xilinx.com/bin/public/openDownload?filename=vitis-ai_v1.1_dnndk_sample_img.tar.gz
$ tar -xvzf vitis-ai_v1.1_dnndk_sample_img.tar.gz

For the custom application (project folder), create a model directory and copy the dpu_*.elf model files you previously built:

$ cd $VITIS_AI_HOME/project
$ mkdir model_for_ultra96v2
$ cp -r model_for_ultra96v2 model
$ make

NOTE: You could also edit the build.sh script to add support for the new Platforms like Ultra96V2.

Execute the AI Application on ULTRA96V2

  1. Boot the Ultra96V2 with the pre-build sd-card image you dowloaded. For Learning How to Do This, Click HERE!
  2. $ cd /run/media/mmcblk0p1
  3. $ cp dpu.xclbin /usr/lib/.
  4. Install the Vitis-AI embedded package:
$ cd runtime/vitis-ai_v1.1_dnndk 
$ source ./install.sh
  1. Define the DISPLAY environment variable:
$ export DISPLAY=:0.0
$ xrandr --output DP-1 --mode 640x480
  1. Run the Custom Application:
 $ cd vitis_ai_dnndk_samples
 $ ./App 
Owner
Amin Mamandipoor
Currently, Studying Master of Computer Systems Architecture at the University of Tabriz.
Amin Mamandipoor
This is the code for "HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields".

HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields This is the code for "HyperNeRF: A Higher-Dimensional

Google 702 Jan 02, 2023
Monitor your ML jobs on mobile devices📱, especially for Google Colab / Kaggle

TF Watcher TF Watcher is a simple to use Python package and web app which allows you to monitor 👀 your Machine Learning training or testing process o

Rishit Dagli 54 Nov 01, 2022
Find the Heart simple Python Game

This is a simple Python game for finding a heart emoji. There is a 3 x 3 matrix in which a heart emoji resides. The location of the heart is randomized and is not revealed. The player must guess the

p.katekomol 1 Jan 24, 2022
🌊 Online machine learning in Python

In a nutshell River is a Python library for online machine learning. It is the result of a merger between creme and scikit-multiflow. River's ambition

OnlineML 4k Jan 02, 2023
This repository contains the accompanying code for Deep Virtual Markers for Articulated 3D Shapes, ICCV'21

Deep Virtual Markers This repository contains the accompanying code for Deep Virtual Markers for Articulated 3D Shapes, ICCV'21 Getting Started Get sa

KimHyomin 45 Oct 07, 2022
Cl datasets - PyTorch image dataloaders and utility functions to load datasets for supervised continual learning

Continual learning datasets Introduction This repository contains PyTorch image

berjaoui 5 Aug 28, 2022
Automatic library of congress classification, using word embeddings from book titles and synopses.

Automatic Library of Congress Classification The Library of Congress Classification (LCC) is a comprehensive classification system that was first deve

Ahmad Pourihosseini 3 Oct 01, 2022
Official code for "End-to-End Optimization of Scene Layout" -- including VAE, Diff Render, SPADE for colorization (CVPR 2020 Oral)

End-to-End Optimization of Scene Layout Code release for: End-to-End Optimization of Scene Layout CVPR 2020 (Oral) Project site, Bibtex For help conta

Andrew Luo 41 Dec 09, 2022
Simple, efficient and flexible vision toolbox for mxnet framework.

MXbox: Simple, efficient and flexible vision toolbox for mxnet framework. MXbox is a toolbox aiming to provide a general and simple interface for visi

Ligeng Zhu 31 Oct 19, 2019
Code for the paper "Graph Attention Tracking". (CVPR2021)

SiamGAT 1. Environment setup This code has been tested on Ubuntu 16.04, Python 3.5, Pytorch 1.2.0, CUDA 9.0. Please install related libraries before r

122 Dec 24, 2022
Auto White-Balance Correction for Mixed-Illuminant Scenes

Auto White-Balance Correction for Mixed-Illuminant Scenes Mahmoud Afifi, Marcus A. Brubaker, and Michael S. Brown York University Video Reference code

Mahmoud Afifi 47 Nov 26, 2022
A Python Reconnection Tool for alt:V

altv-reconnect What? It invokes a reconnect in the altV Client Dev Console. You get to determine when your local client should reconnect when developi

8 Jun 30, 2022
Source for the paper "Universal Activation Function for machine learning"

Universal Activation Function Tensorflow and Pytorch source code for the paper Yuen, Brosnan, Minh Tu Hoang, Xiaodai Dong, and Tao Lu. "Universal acti

4 Dec 03, 2022
GAN example for Keras. Cuz MNIST is too small and there should be something more realistic.

Keras-GAN-Animeface-Character GAN example for Keras. Cuz MNIST is too small and there should an example on something more realistic. Some results Trai

160 Sep 20, 2022
CyTran: Cycle-Consistent Transformers for Non-Contrast to Contrast CT Translation

CyTran: Cycle-Consistent Transformers for Non-Contrast to Contrast CT Translation We propose a novel approach to translate unpaired contrast computed

Nicolae Catalin Ristea 13 Jan 02, 2023
A U-Net combined with a variational auto-encoder that is able to learn conditional distributions over semantic segmentations.

Probabilistic U-Net + **Update** + An improved Model (the Hierarchical Probabilistic U-Net) + LIDC crops is now available. See below. Re-implementatio

Simon Kohl 498 Dec 26, 2022
Part-Aware Data Augmentation for 3D Object Detection in Point Cloud

Part-Aware Data Augmentation for 3D Object Detection in Point Cloud This repository contains a reference implementation of our Part-Aware Data Augment

Jaeseok Choi 62 Jan 03, 2023
Wordle-solver - Wordle answer generation program in python

🟨 Wordle Solver 🟩 Wordle answer generation program in python ✔️ Requirements U

Dahyun Kang 4 May 28, 2022
Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields.

This repository contains the code release for Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields. This implementation is written in JAX, and is a fork of Google's JaxNeRF

Google 625 Dec 30, 2022
Raster Vision is an open source Python framework for building computer vision models on satellite, aerial, and other large imagery sets

Raster Vision is an open source Python framework for building computer vision models on satellite, aerial, and other large imagery sets (including obl

Azavea 1.7k Dec 22, 2022