3D Human Pose Estimation with OpenVINO

This demo contains 3D multi-person pose estimation demo. Intel OpenVINO™ backend can be used for fast inference on CPU. It is based on Lightweight OpenPose and Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB papers.
The implementation of this demo starts with the ideas I originally wrote about in my blog There are two options involved in this pull request. One is to use WebGL, which interacts with the browser, and the other is to use the less dependent OpenCV, which implements a basic 3D visual library.

406-human-pose-estimation-3d

This demo allows you to use the mouse to change the angle of view from which you view an object.

406-opencv-human-pose-estimation-3d

This example allows you to use the keyboard to move your camera and press ESC to exit.(You need to set use_popup=True firstly)

I fixed a lot of things, added documentation and removed big files from the git history. Hence, I create a new PR.

The picture has the proper licence, as it comes from COCO - https://cocodataset.org/#explore?id=166392

This PR adds demo notebook for grayscale image colorization using colorization-v2 model from Open Model Zoo

Pending Tasks:

• [x] - ~~Handle video input to colorize~~
• [x] - Add explanation (markdown) to the notebook cells
• [x] - Complete README.md
• [x] - Follow up with suggestions and reviews

Added the following features to PyTorch and Tensorflow quantization aware training notebooks

• fine-tuning of float32 model the same way as int8 model is finetuned
• accuracy comparison between fine-tuned int8 and fine-tuned float32 models

Note: nbval fails, however it also seems to fail for the main branch

AnimeGAN notebook with model from https://github.com/PaddlePaddle/PaddleGAN

Convert PaddleGAN model to ONNX and then to IR, and show inference results.

PaddlePaddle requirements are installed in the notebook, with !pip. This requires that users activated the openvino_env environment and kernel - which they do if they follow our instructions.

Converting this model was not completely straightforward. I added some steps to the notebook that show how to go about this, for example to do predictor.run?? to show the source of the function, to see how to preprocess and postprocess the model output.

This is a Draft PR - a README should be added and the descriptions in the notebook should be updated before merging.

The notebook currently fails in the CI for Windows. I'll look into that - it seems to be a resource issue. It works on my Windows laptop.

Describe the bug I followed notebook 401-object-detection and it works. Then I wanted to reuse the converted model within a python script with the same command line : ie_core = Core() model = ie_core.read_model(model=root + converted_model_path) where "root" is path to openvino_notebooks

But I get openvino_notebooks/notebooks/401-object-detection-webcam/model/public/ssdlite_mobilenet_v2/FP16/ssdlite_mobilenet_v2.xml cannot be opened!

Expected behavior I hope I can reuse the converted model from my script

Screenshots If applicable, add screenshots to help explain your problem.

Installation instructions (Please mark the checkbox) [ x ] I followed the installation guide at https://github.com/openvinotoolkit/openvino_notebooks#-installation-guide to install the notebooks. I did it twice !

** Environment information ** Pip version: 22.1 OpenVINO source: /home/fenaux/openvino_env/lib/python3.9/site-packages/openvino OpenVINO IE version: 2022.1.0-7019-cdb9bec7210-releases/2022/1 OpenVINO environment activated: OK Jupyter kernel installed for openvino_env: NOT OK Python version: 3.9 OK OpenVINO pip package installed: OK OpenVINO import succeeds: OK OpenVINO development tools installed: OK OpenVINO not installed globally: OK No broken requirements: OK

Thanks for your help

Here is a webcam version of hello world. It uses the same model as the 001-hello-world but we use a webcam feed as the input. The main issue is how we can do CI with this at all, and that's why I'm thinking we have to put this under a 4xx series as it will have hardware dependencies.

However, I will push a pull request here and see what we think about it and so at least I have this somewhere. :)

Interactive Text Prediction with OpenVINO

This is a demo for text prediction using gpt-2 model The complete pipeline of this demo's notebook is shown below.

A notebook that implements Yolo-v4-tiny-tf and yolo-v4-tf. Compared to the 401 object detection notebook, changes had to be made to the output processing to find bounding boxes and to resize the image while preserving the aspect ratio for improved performance. yolov4 model

What's left: Some documentation / explanation.

Something I found that needs to be confirmed is that Cx, Cy (cell index) and w, h (bounding box width/height), from the documentation, needs to have order changed to Cy, Cx and h, w respectively. Converted model input documentation should also become B, H, W, C, instead of B, C, H, W which gives an error. Whether or not the input is BGR or RGB isn't too clear yet considering the model goes by original input for dimensions.

Edit: New Documentation is consistent with the current inputs I have. So BHWC is correct (using BGR).

Here is a list of known issues for using OpenVINO 2022.3 and OpenVINO Notebooks. You can compile and obtain the 2022.3 from here.

https://github.com/openvinotoolkit/openvino/wiki

Known issues (Ubuntu 22.04 + Python 3.10):

1. Python 3.10 and torch 1.8.1 dependencies is conflicting. (ERROR: Could not find a version that satisfies the requirement torch==1.8.1+cpu)
2. PaddlePaddle 2.2 is also conflicting/missing. (ERROR: Could not find a version that satisfies the requirement paddlepaddle==2.2.*)
3. Tensorflow 2.5.3 (ERROR: Could not find a version that satisfies the requirement tensorflow==2.5.3)

• waveplot is deprecated in favor of waveshow since librosa v0.8.1, and removed in v0.9.0.

  This was mentioned in the original pull request in Dec. 2021, but not fixed (for unknown reason).

  Note that max_sr in waveplot is removed in waveshow. max_sr controls how signals are down-sampled if they’re too long. (waveshow uses “a downsampled amplitude envelope” instead)

In the committed version, Imports are at the top of the notebook.

• librosa.filters.mel in audio_to_mel

  FutureWarning: Pass sr=16000, n_fft=512 as keyword args. From version 0.10 passing these as positional arguments will result in an error.

Question

The original pull requeset did NOT add librosa (the audio analysis package used here) into requirements.txt or .docker/Pipfile. Is it on purpose? Should I tell how to install it in the notebook?

im using yolov7 226 yolo optimisation notebook,

i trained my model using yolov7x.cfg which has 40 classes, I did all respectivate changes according to model .

i am able to generate -onnx & .XML file , its inference is also working but when I was planning for converting it into int8 format . I'm not able to load it

from openvino.runtime import Core
core = Core()
# read converted model
model = core.read_model('model/best_veh_withbgnew.xml')
# load model on CPU device
compiled_model = core.compile_model(model, 'CPU')

I'm getting this error

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-10-db161e3ad74f> in <module>
      2 core = Core()
      3 # read converted model
----> 4 model = core.read_model('model/best_veh_withbgnew.xml')
      5 # load model on CPU device
      6 compiled_model = core.compile_model(model, 'CPU')

RuntimeError: Check 'false' failed at C:\Jenkins\workspace\private-ci\ie\build-windows-vs2019\b\repos\openvino\src\frontends\common\src\frontend.cpp:54:
Converting input model
Incorrect weights in bin file!

When I run this notebook on Ubuntu with a successful setup of the virtual env and requirements.txt install...the kernel dies on my machine half way through every time...would you have tips to try?

Its this block of code towards the end...where it does run I can see the process go from 0 to 100% but after a 100% is met the Kernel dies and I cant make it any further.

mp, mr, map50, map, maps, num_images, labels = test(data=data, model=compiled_model, dataloader=dataloader, names=NAMES)
# Print results
s = ('%20s' + '%12s' * 6) % ('Class', 'Images', 'Labels', 'Precision', 'Recall', '[email protected]', '[email protected]:.95')
print(s)
pf = '%20s' + '%12i' * 2 + '%12.3g' * 4  # print format
print(pf % ('all', num_images, labels, mp, mr, map50, map))

Any options to try greatly appreciated.

I found there are many duplicates in the repository e.g coco.jpg. It increases cloning time and space usage. It would be good to create a "central directory" with images and videos to use across all notebooks.

I propose:

1. Create the "data" dir in the root dir
2. Move all images and videos from specific notebooks, remove duplicates
3. Update links to media in all notebooks
4. Update contributing guide