Python wrapper class for OpenVINO Model Server. User can submit inference request to OVMS with just a few lines of code

Last update: Jul 27, 2022

Overview

OpenVINO Model Server wrapper API for Python

Description

This project provides a Python wrapper class for OpenVINO Model Server.
User can submit DL inference request to OVMS with just a few lines of code.

Sample code

from ovms_wrapper.ovms_wrapper import OpenVINO_Model_Server

import cv2
import numpy as np

ovms = OpenVINO_Model_Server()
ovms.connect('127.0.0.1', 9000)
#print(ovms.get_model_status('resnet_50'))
model = ovms.open_model('resnet_50')
print(model.inputs, model.outputs)

image_file  = 'ovms/lib/python3.8/site-packages/skimage/data/rocket.jpg'
img = cv2.imread(image_file)                # Read an image
res = model.single_image_infer(img)         # Infer
result = res[model.outputs[0]['name']]

# display result
nu = np.array(result)
ma = np.argmax(nu)
print("Result:", ma)

How to setup OpenVINO Model Server (Ubuntu)

Note: OVMS can run on Windows too. Please refer to the official OVMS document for details.

sudo apt update && sudo apt install -y python3-venv
python -m pip install tensorflow tensorflow-serving-api

Create Python virtual env, install OpenVINO, and prepare an IR model
Installing OpenVINO just for downloading a DL model and converting it into OpenVINO IR model. This is not required if you already have the IR models.

python3 -m venv ovms
. ovms/bin/activate
python -m pip install openvino-dev
omz_downloader --name resnet-50-tf
omz_converter --name resnet-50-tf --precisions FP16
deactivate

Start OpenVINO Model Server as Docker container

docker run -d --rm \
  -v $PWD/public/resnet-50-tf/FP16:/models/resnet50/1 \
  -p 9000:9000 \
  openvino/model_server:latest \
  --model_path /models/resnet50 \
  --model_name resnet_50 \
  --port 9000

OVMS will start serving the Resnet-50 model as model-name='resnet_50', model-version=1, and gRPC-port=9000.

Python wrapper class for OpenVINO Model Server. User can submit inference request to OVMS with just a few lines of code

Related tags

Overview

OpenVINO Model Server wrapper API for Python

Description

Sample code

How to setup OpenVINO Model Server (Ubuntu)

Owner

Yasunori Shimura

An official source code for "Augmentation-Free Self-Supervised Learning on Graphs"

Revisiting Temporal Alignment for Video Restoration

Dilated Convolution for Semantic Image Segmentation

This repository contains the code for designing risk bounded motion plans for car-like robot using Carla Simulator.

REBEL: Relation Extraction By End-to-end Language generation

A Python library that provides a simplified alternative to DBAPI 2

EASY - Ensemble Augmented-Shot Y-shaped Learning: State-Of-The-Art Few-Shot Classification with Simple Ingredients.

kullanışlı ve işinizi kolaylaştıracak bir araç

Code for "Single-view robot pose and joint angle estimation via render & compare", CVPR 2021 (Oral).

Mask2Former: Masked-attention Mask Transformer for Universal Image Segmentation in TensorFlow 2

Open-source code for Generic Grouping Network (GGN, CVPR 2022)

JORLDY an open-source Reinforcement Learning (RL) framework provided by KakaoEnterprise

CyTran: Cycle-Consistent Transformers for Non-Contrast to Contrast CT Translation

Robust fine-tuning of zero-shot models

PyTorch-Geometric Implementation of MarkovGNN: Graph Neural Networks on Markov Diffusion

Implementation of TimeSformer, a pure attention-based solution for video classification

Physics-Aware Training (PAT) is a method to train real physical systems with backpropagation.

code for "AttentiveNAS Improving Neural Architecture Search via Attentive Sampling"

Original Implementation of Prompt Tuning from Lester, et al, 2021

Code for "MetaMorph: Learning Universal Controllers with Transformers", Gupta et al, ICLR 2022