ONNX Command-Line Toolbox

Overview

ONNX Command Line Toolbox

Build and Test CodeQL Sanity Coverage

  • Aims to improve your experience of investigating ONNX models.
  • Use it like onnx infershape /path/to/model.onnx. (See the usage section for more.)

Installation

Recommand to install via GitHub repo for the latest functionality.

pip install git+https://github.com/jackwish/onnxcli.git

Two alternative ways are:

  1. Install via pypi package pip install onnxcli
  2. Download and add the code tree to your $PYTHONPATH. This is for development purpose since the command line is different.
    git clone https://github.com/jackwish/onnxcli.git
    export PYTHONPATH=$(pwd)/onnxcli:${PYTHONPATH}
    python onnxcli/cli/dispatcher.py <more args>
    

The onnx draw requires dot command (graphviz) to be avaiable on your machine - which can be installed by command as below on Ubuntu/Debian.

sudo apt install -y graphviz

Usage

Once installed, the onnx and onnxcli commands are avaiable on your machine. You can play with commands such as onnx infershape /path/to/model.onnx. The general format is onnx <sub command> <dedicated arguments ...>. The sub commands are as sections below.

Check the online help with onnx --help and onnx <subcmd> --help for latest usage.

infershape

onnx infershape performs shape inference of the ONNX model. It's an CLI wrapper of onnx.shape_inference. You will find it useful to generate shape information for the models that are extracted by onnx extract.

extract

onnx extract extracts the sub model that is determined by the names of the input and output tensor of the subgraph from the original model. It's a CLI wrapper of onnx.utils.extract_model (which I authorized in the ONNX repo).

inspect

onnx inspect gives you quick view of the information of the given model. It's inspired by the tf-onnx tool.

When working on deep learning, you may like to take a look at what's inside the model. Netron is powerful but doesn't provide fine-grain view.

With onnx inspect, you no longer need to scroll the Netron window to look for nodes or tensors. Instead, you can dump the node attributes and tensor values with a single command.

Click here to see a node example

$ onnx inspect ./assets/tests/conv.float32.onnx --node --indices 0 --detail

Inpect of model ./assets/tests/conv.float32.onnx Graph name: 9 Graph inputs: 1 Graph outputs: 1 Nodes in total: 1 ValueInfo in total: 2 Initializers in total: 2 Sparse Initializers in total: 0 Quantization in total: 0

Node information: Node "output": type "Conv", inputs "['input', 'Variable/read', 'Conv2D_bias']", outputs "['output']" attributes: [name: "dilations" ints: 1 ints: 1 type: INTS , name: "group" i: 1 type: INT , name: "kernel_shape" ints: 3 ints: 3 type: INTS , name: "pads" ints: 1 ints: 1 ints: 1 ints: 1 type: INTS , name: "strides" ints: 1 ints: 1 type: INTS ]

Click here to see a tensor example

$ onnx inspect ./assets/tests/conv.float32.onnx --tensor --names Conv2D_bias --detail

Inpect of model ./assets/tests/conv.float32.onnx Graph name: 9 Graph inputs: 1 Graph outputs: 1 Nodes in total: 1 ValueInfo in total: 2 Initializers in total: 2 Sparse Initializers in total: 0 Quantization in total: 0

Tensor information: Initializer "Conv2D_bias": type FLOAT, shape [16], float data: [0.4517577290534973, -0.014192663133144379, 0.2946248948574066, -0.9742919206619263, -1.2975586652755737, 0.7223454117774963, 0.7835700511932373, 1.7674627304077148, 1.7242872714996338, 1.1230682134628296, -0.2902531623840332, 0.2627834975719452, 1.0175092220306396, 0.5643373131752014, -0.8244842290878296, 1.2169424295425415]

draw

onnx draw draws the graph in dot, svg, png formats. It gives you quick view of the type and shape of the tensors that are fed to a specific node. You can view the model topology in image viewer of browser without waiting for the model to load, which I found is really helpful for large models.

If you are viewing svg in browser, you can even quick search for the nodes and tensors. Together with onnx inspect, it will be very efficient to understand the issue you are looking into.

The node are in ellipses and tensors are in rectangles where the rounded ones are initializers. The node type of the node and the data type and shape of the tenors are also rendered. Here is a Convolution node example.

conv

Contributing

Welcome to contribute new commands or enhance them. Let's make our life easier together.

The workflow is pretty simple:

  1. Starting with GitHub Codespace or clone locally.
  • make setup to config the dependencies (or pip install -r ./requirements.txt if you prefer).
  1. Create a new subcommand
  • Starting by copying and modifying infershape.
  • Register the command in the dispatcher
  • Create a new command line test
  • make test to build and test.
  • make check and make format to fix any code style issues.
  1. Try out, debug, commit, push, and open pull request.
  • The code has been protected by CI. You need to get a pass before merging.
  • Ask if any questions.

License

Apache License Version 2.0.

Comments
  • Some ONNX models don't list activation tensors in GraphProto.value_info

    Some ONNX models don't list activation tensors in GraphProto.value_info

    They should, but they don't. I am not sure why such models behave like this - they cannot pass the ONNX model checker.

    There should be something wrong with the exporter. I can try to figure out which exporter has such issues.

    For onnxcli, any functionality depending on walking GraphProto.value_info may not show the real model. This is not our defect, but the models'. To workaround, you can firstly run shape inference on the model, and the GraphProto.value_info listing issue will be fixed.

    onnx infershape /path/to/input/model /path/to/output/model
    
    documentation 
    opened by zhenhuaw-me 2
  • Integrate the onnx dumper

    Integrate the onnx dumper

    src: https://github.com/onnx/tensorflow-onnx/blob/master/tools/dump-onnx.py

    most of them need to be renamed.

    • [x] inspect to check the model
    • [x] dump dot has high priotiry
    • [ ] print to std if no file specified
    opened by zhenhuaw-me 0
  • Optimizer reports

    Optimizer reports "Unresolved value references" since v0.3.0

    Via pipeline https://github.com/zhenhuaw-me/onnxcli/actions/runs/3453474851/jobs/5764096907.

    A simple model works no issue till optimizer v0.2.7 (verified locally), but starts to fail with optimizer v0.3.0 (verified locally) and still fail with v0.3.2 (the pipeline).

    It's onnx optimize ./assets/tests/conv.float32.onnx optimized.onnx.

    opened by zhenhuaw-me 2
  • Overwrite weights (initializers) with fixed data or random data

    Overwrite weights (initializers) with fixed data or random data

    Bert series ONNX models are very large (x GB) thus not easy to share the real file. We can improve this process by overwriting the weights (initializers)

    • It can be fixed data (e.g. all 0.1 or other value specified), thus the model can be compressed.
    • After sharing, we can recover with numpy style random numbers.

    This can only be used as a sharing method, the generated model are not useful when evaluate accuracy.

    For better usage:

    • Annotation will be added when writing fixed data, thus when re-random we can detect automatically.
    • The tensors can be specified with names or size.
    • Only works for FP32/FP16.
    • 0 removed.
    enhancement 
    opened by zhenhuaw-me 0
  • [draw] show tensor information on the edges

    [draw] show tensor information on the edges

    We currently draw tensors as boxes and operators as circles.

    image

    The graph will be complex if large model. We draw the tensor information on the edges and keep only operators as nodes.

    enhancement 
    opened by zhenhuaw-me 0
  • [infershape] should be able to set tensor shapes - inputs and others

    [infershape] should be able to set tensor shapes - inputs and others

    infershape is not very useful if the input shapes are symbolics (dynamic shapes). If the user can set input shapes, it's more powerful:

    • If set to static shapes, the shape of the model will be known.
    • Even for symbolics, the user can update the input shapes.

    The setup should be optional, and can extend to all the tensors in the model (excluding shape op related).

    Interface should be something like below.

    onnx infershape path/to/input/model.onnx path/to/output/model.onnx --tensor-shape t1:[d0,d1] t2:[d0,d1,d3]
    
    enhancement 
    opened by zhenhuaw-me 0
  • Extract should be able to skip the input tensor names

    Extract should be able to skip the input tensor names

    We should be able to walk the graph starting with the output tensor names and auto infer the input names if not given.

    It would be interesting to figure out if the user provided input tensor names and output tensor names don't cut a subgraph.

    enhancement 
    opened by zhenhuaw-me 0
Releases(v0.2.1)
  • v0.2.1(Nov 13, 2022)

    What's Changed

    • Ping onnxoptimizer to 0.2.7 due to "Unresolved value references" issue. See more in https://github.com/zhenhuaw-me/onnxcli/issues/28
    • convert: enable onnx to json by @zhenhuaw-me in https://github.com/zhenhuaw-me/onnxcli/pull/10
    • inspect: print input and output tensor too by @zhenhuaw-me in https://github.com/zhenhuaw-me/onnxcli/pull/12
    • inspect: dump input output tensor by @zhenhuaw-me in https://github.com/zhenhuaw-me/onnxcli/pull/14
    • inspect: show dimension name instead of value if has any by @zhenhuaw-me in https://github.com/zhenhuaw-me/onnxcli/pull/17
    • draw: gen tensor info for tensors that only have name by @zhenhuaw-me in https://github.com/zhenhuaw-me/onnxcli/pull/18
    • setup: install the dependent python packages by @zhenhuaw-me in https://github.com/zhenhuaw-me/onnxcli/pull/19
    • Check command by @zhenhuaw-me in https://github.com/zhenhuaw-me/onnxcli/pull/21

    Full Changelog: https://github.com/zhenhuaw-me/onnxcli/compare/v0.2.0...v0.2.1

    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Jan 8, 2022)

  • v0.1.0(Dec 24, 2021)

Owner
黎明灰烬 (王振华 Zhenhua WANG)
A b[i|y]te of ML.sys|Arch|VM.
黎明灰烬 (王振华 Zhenhua WANG)
BERTMap: A BERT-Based Ontology Alignment System

BERTMap: A BERT-based Ontology Alignment System Important Notices The relevant paper was accepted in AAAI-2022. Arxiv version is available at: https:/

KRR 36 Dec 24, 2022
Mesh Graphormer is a new transformer-based method for human pose and mesh reconsruction from an input image

MeshGraphormer ✨ ✨ This is our research code of Mesh Graphormer. Mesh Graphormer is a new transformer-based method for human pose and mesh reconsructi

Microsoft 251 Jan 08, 2023
An auto discord account and token generator. Automatically verifies the phone number. Works without proxy. Bypasses captcha.

JOIN DISCORD SERVER https://discord.gg/uAc3agBY FREE HCAPTCHA SOLVING API Discord-Token-Gen An auto discord token generator. Auto verifies phone numbe

3kp 271 Jan 01, 2023
Public repo for the ICCV2021-CVAMD paper "Is it Time to Replace CNNs with Transformers for Medical Images?"

Is it Time to Replace CNNs with Transformers for Medical Images? Accepted at ICCV-2021: Workshop on Computer Vision for Automated Medical Diagnosis (C

Christos Matsoukas 80 Dec 27, 2022
Learning the Beauty in Songs: Neural Singing Voice Beautifier; ACL 2022 (Main conference); Official code

Learning the Beauty in Songs: Neural Singing Voice Beautifier Jinglin Liu, Chengxi Li, Yi Ren, Zhiying Zhu, Zhou Zhao Zhejiang University ACL 2022 Mai

Jinglin Liu 257 Dec 30, 2022
This is a Deep Leaning API for classifying emotions from human face and human audios.

Emotion AI This is a Deep Leaning API for classifying emotions from human face and human audios. Starting the server To start the server first you nee

crispengari 5 Oct 02, 2022
EmoTag helps you train emotion detection model for Chinese audios

emoTag emoTag helps you train emotion detection model for Chinese audios. Environment pip install -r requirement.txt Data We used Emotional Speech Dat

_zza 4 Sep 07, 2022
Library for implementing reservoir computing models (echo state networks) for multivariate time series classification and clustering.

Framework overview This library allows to quickly implement different architectures based on Reservoir Computing (the family of approaches popularized

Filippo Bianchi 249 Dec 21, 2022
Confidence Propagation Cluster aims to replace NMS-based methods as a better box fusion framework in 2D/3D Object detection

CP-Cluster Confidence Propagation Cluster aims to replace NMS-based methods as a better box fusion framework in 2D/3D Object detection, Instance Segme

Yichun Shen 41 Dec 08, 2022
Official implementation of "StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation" (SIGGRAPH 2021)

StyleCariGAN in PyTorch Official implementation of StyleCariGAN:Caricature Generation via StyleGAN Feature Map Modulation in PyTorch Requirements PyTo

PeterZhouSZ 49 Oct 31, 2022
"Learning and Analyzing Generation Order for Undirected Sequence Models" in Findings of EMNLP, 2021

undirected-generation-dev This repo contains the source code of the models described in the following paper "Learning and Analyzing Generation Order f

Yichen Jiang 0 Mar 25, 2022
[CVPR2021 Oral] End-to-End Video Instance Segmentation with Transformers

VisTR: End-to-End Video Instance Segmentation with Transformers This is the official implementation of the VisTR paper: Installation We provide instru

Yuqing Wang 687 Jan 07, 2023
Interpolation-based reduced-order models

Interpolation-reduced-order-models Interpolation-based reduced-order models High-fidelity computational fluid dynamics (CFD) solutions are time consum

Donovan Blais 1 Jan 10, 2022
Research - dataset and code for 2016 paper Learning a Driving Simulator

the people's comma the paper Learning a Driving Simulator the comma.ai driving dataset 7 and a quarter hours of largely highway driving. Enough to tra

comma.ai 4.1k Jan 02, 2023
SwinTrack: A Simple and Strong Baseline for Transformer Tracking

SwinTrack This is the official repo for SwinTrack. A Simple and Strong Baseline Prerequisites Environment conda (recommended) conda create -y -n SwinT

LitingLin 196 Jan 04, 2023
Command-line tool for downloading and extending the RedCaps dataset.

RedCaps Downloader This repository provides the official command-line tool for downloading and extending the RedCaps dataset. Users can seamlessly dow

RedCaps dataset 33 Dec 14, 2022
A fast Protein Chain / Ligand Extractor and organizer.

Are you tired of using visualization software, or full blown suites just to separate protein chains / ligands ? Are you tired of organizing the mess o

Amine Abdz 9 Nov 06, 2022
Luminous is a framework for testing the performance of Embodied AI (EAI) models in indoor tasks.

Luminous is a framework for testing the performance of Embodied AI (EAI) models in indoor tasks. Generally, we intergrete different kind of functional

28 Jan 08, 2023
Implementation for our ICCV2021 paper: Internal Video Inpainting by Implicit Long-range Propagation

Implicit Internal Video Inpainting Implementation for our ICCV2021 paper: Internal Video Inpainting by Implicit Long-range Propagation paper | project

202 Dec 30, 2022
This repo provides the base code for pytorch-lightning and weight and biases simultaneous integration.

Write your model faster with pytorch-lightning-wadb-code-backbone This repository provides the base code for pytorch-lightning and weight and biases s

9 Mar 29, 2022