Pre-Holiday Season 2022 Release

• Restructured the repo (re-organized folders and files)
• Updated model tables in README to add model task categories and grouped similar models together
• NO new models were added in this release

December 2022 First Release

1. Reformatted README file to enhance readability
2. Added detailed model performance evaluation results in their respective Markdown document files
3. Added the following PyTorch models: QuickSRNet, Bert, DistilBert, MiniLM, MobileBert, MobileViT, Roberta, ViT

This releases PyTorch W8A8 optimized weight and quantization config file for vision transformer model (ViT) for image classification tasks using ImageNet dataset

This releases PyTorch Roberta for Natural Language Classifier models' optimized checkpoints for GLUE dataset.

This releases PyTorch W8A8 optimized weights for mobile friendly vision transformer model (MobileViT) for image classification tasks using ImageNet dataset

This releases PyTorch MobileBERT W8A8 optimized checkpoints for GLUE dataset and SQuAD dataset

This releases PyTorch MiniLM optimized W8A8 checkpoints for GLUE dataset and SQuAD dataset

November 2022 Release Added W4A8 quantization support and updated code, documentation and model checkpoint artifacts for the following existing models:

1. Pytorch Classification (regnet_x_3_2gf, resnet18, resnet50)
2. PyTorch HRNet Posenet
3. PyTorch HRNet
4. PyTorch EfficientNet Lite0
5. PyTorch DeeplabV3-MobileNetV2

This releases PyTorch DistilBERT optimized W8A8 checkpoints for GLUE dataset and SQuAD dataset.

This releases PyTorch Bert optimized W8A8 checkpoints for GLUE dataset and SQuAD dataset.

The release provides the model checkpoint tarballs for different variations of the PyTorch-based QuickSRNet. Each model tarball corresponds to a given num_channels and scaling_factor (release_quicksrnet__<scaling_factor>x.tar.gz). Each tarball contains the following:

• checkpoint_float32.pth.tar - full-precision model with the highest validation accuracy on the DIV2k dataset
• checkpoint_int8.pth - quantized model with the highest validation accuracy obtained with Quantization-aware Training using AIMET
• checkpoint_int8.encodings - Encodings for the quantized models
• checkpoint_int8_param_only.encodings - Encodings for the quantized models, with only parameters, without activation encodings

There are 5 models in total. The first three models are for high resolution (10242048) input, and the last two models for low resolution (5121024) input.

1. segmentation_ffnet78S_dBBB_mobile
2. segmentation_ffnet54S_dBBB_mobile
3. segmentation_ffnet40S_dBBB_mobile
4. segmentation_ffnet78S_BCC_mobile_pre_down
5. segmentation_ffnet122NS_CCC_mobile_pre_down

The following assets are provided for each model:

• Prepared FP32 checkpoint
• Optimized INT8 checkpoint
• Optimized INT8 encodings

Optimized W8A8 checkpoints and encodings for InverseForm OCRNet-48-IF and HRNet-16-Slim-IF models.

1. Cross Layer Equalization and AdaRound in per channel mode has been applied on the original model
2. Adaround has been optimized with 8-bit width and "tf_enhanced" quant scheme
3. Quantization evaluated with "tf_enhanced" quant scheme in 8 bit width weight & activation quantization

This is the TensorFlow MobileDet-EdgeTPU (FP32 pretrained model is from mlcommons) optimized checkpoint

1. Batch Norm folding and Adaround has been applied on the original model
2. Adaround has been optimized with 8-bit width and "tf" quant scheme in the per-tensor mode
3. Quantization evaluated with "tf" quant scheme in 8 bit width weight & activation quantization

September 2022 Release

• Updated quantization evaluation code and documentation that is compatible with the latest AIMET OS release 1.22
• Increased clarity and ease of use
• More standardized and consistent procedure across all models
• Better automation and streamlining of setup tasks
• Code and documentation cleanup and fixes.
• NO new models were added and NO new binaries or artifacts were published as part of this release

Optimized W4A8 and W8A8 checkpoints and encodings for Pytorch Torchvision instances of Resnet18, Resnet50 and Regnet_x_3_2gf.

1. Cross-Layer Equalization and AdaRound have been applied in per channel mode on the original model for both INT4 and INT8 optimization
2. "TF_enhanced" quantscheme was used for quantsim

Optimized W4A8 and W8A8 checkpoints, encodings and FP32 checkpoint for Pytorch HRNET-Posenet model.

For INT8 optimization:

1. Batchnorm folding has been applied on the original FP32 instance of HRNET-Posenet model.
2. TF quantscheme was used in per channel mode for quantsim.

For INT4 optimization:

1. Batchnorm folding followed by AdaRound in per channel mode have been applied on the original FP32 instance of HRNET-Posenet model.
2. TF quantscheme was used in per channel mode for quantsim.

This are the PyTorch HRNet W8A8 and W4A8 per channel quantsim model checkpoint

1. Cross Layer Equalization and Adaround has been applied on the original model
2. Adaround has been optimized with 8-bit width OR 4-bit width and "tf_enhanced" quant scheme
3. Quantization evaluated with "tf_enhanced" quant scheme in 8 bit width activation quantization

These are the PyTorch EfficientNet Lite W8A8 and W4A8 per channel quantsim model checkpoints:

1. Original FP32 model is transformed to a new FP32 model by model validate->model preparer -> model validate
2. Batch Norm folding and Adaround has been applied on the transformed model
3. Adaround has been optimized with 4-bit width OR 8-bit width and "tf_enhanced" quant scheme
4. Quantization evaluated with "tf" quant scheme in 8 bit width activation quantization

This is the PyTorch Deeplab V3+ W8A8 OR W4A8 per channel QuantSim model weights file and encodings file

1. Cross Layer Equalization and Adaround has been applied on the original model
2. Adaround has been optimized with 8-bit OR 4-bit width width, and "tf_enhanced" quant scheme
3. Quantization evaluated with "tf_enhanced" quant scheme in 8 bit width activation quantization

The release provides the model checkpoint tarballs for different variations of the PyTorch-based Extremely Lightweight Quantization Robust Real-Time Single-Image Super Resolution (XLSR) model. Each model tarball corresponds to a given scaling_factor (release_xlsr_<scaling_factor>x.tar.gz). Each tarball contains the following:

• checkpoint_float32.pth.tar - full-precision model with the highest validation accuracy on the DIV2k dataset
• checkpoint_int8.pth - quantized model with the highest validation accuracy obtained with Quantization-aware Training using AIMET
• checkpoint_int8.encodings - Encodings for the quantized models

The release provides the model checkpoint tarballs for different variations of the PyTorch-based PyTorch Super-Efficient Super Resolution (SESR) model. Each model tarball corresponds to a given scaling_factor (release_sesr_<config_name>_<scaling_factor>x.tar.gz). Each tarball contains the following:

• checkpoint_float32.pth.tar - full-precision model with the highest validation accuracy on the DIV2k dataset
• checkpoint_int8.pth - quantized model with the highest validation accuracy obtained with Quantization-aware Training using AIMET
• checkpoint_int8.encodings - Encodings for the quantized models

The release provides the model checkpoint tarballs for different variations of the PyTorch-based Anchor-based PlainNet Model (ABPN). Each model tarball corresponds to a given num_channels and scaling_factor (release_abpn_<num_channels>_<scaling_factor>x.tar.gz). Each tarball contains the following:

• checkpoint_float32.pth.tar - full-precision model with the highest validation accuracy on the DIV2k dataset
• checkpoint_int8.pth - quantized model with the highest validation accuracy obtained with Quantization-aware Training using AIMET
• checkpoint_int8.encodings - Encodings for the quantized models

This is the PyTorch EfficientNet Lite optimized checkpoint

1. Batch Norm folding and Adaround has been applied on the original model
2. Adaround has been optimized with 8-bit width and "tf_enhanced" quant scheme
3. Quantization evaluated with "tf_enhanced" quant scheme in 8 bit width weight & activation quantization

fake quant removed ssd mobilenet_v2 quantized 300x300 coco 2019 01 03

The model was from an older version of mmediting which was called mmsr. It is not publicly available anymore so we release the model here.

srgan_mmsr_MSRGANx4.gz

A forked version is found here. No model file available just the script.

The tarball file contains a compressed PyTorch pose estimation model.

This contains the model checkpoint for optimized Tensorflow pose estimation model.

• Model has been optimized with Data Free Quantization + Quantization Aware Training • Batch norm folding and Data Free Quantization has been applied, this checkpoint does not contain batch norm layers and ReLU6 in model definition MUST be replaced with RELU • This checkpoint has been evaluated in INT8 - 71.14%

General

Post QAT checkpoint for mobilenetv2-1.4. Quantization was done after Batch Norm folding, with tf quant scheme encodings, and the default configuration file. Note that this checkpoint has Batch Norms folded.

Quantized Accuracy: 74.11%

Quantizer Op Assumptions

In the evaluation script included, we have used the default config file, which configures the quantizer ops with the following assumptions:

• Weight quantization: 8 bits, asymmetric quantization
• Bias parameters are not quantized
• Activation quantization: 8 bits, asymmetric quantization
• Model inputs are not quantized
• Operations which shuffle data such as reshape or transpose do not require additional quantizers

Contents

The tarball contains the following files: checkpoint – Text file for TensorFlow to find latest checkpoint model.data-00000-of-00001 model.index model.meta - Model checkpoint and meta files