This is project is the implementation of the DeepShift: Towards Multiplication-Less Neural Networks paper

Overview

DeepShift

This is project is the implementation of the DeepShift: Towards Multiplication-Less Neural Networks paper, that aims to replace multiplications in a neural networks with bitwise shift (and sign change).

[Paper] - [arXiv] - [Video] - [Presentation]

This research project was done at Huawei Technologies.

If you find this code useful, please cite our paper:

@InProceedings{Elhoushi_2021_CVPR,
    author    = {Elhoushi, Mostafa and Chen, Zihao and Shafiq, Farhan and Tian, Ye Henry and Li, Joey Yiwei},
    title     = {DeepShift: Towards Multiplication-Less Neural Networks},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
    month     = {June},
    year      = {2021},
    pages     = {2359-2368}
}
Table of Contents

Overview

The main idea of DeepShift is to test the ability to train and infer using bitwise shifts. Main Concept of DeepShift

We present 2 approaches:

  • DeepShift-Q: the parameters are floating point weights just like regular networks, but the weights are rounded to powers of 2 during the forward and backward passes
  • DeepShift-PS: the parameters are signs and shift values DeepShift-Q DeepShift-PS

Important Notes

  • To train from scratch, the learning rate --lr option should be set to 0.01. To train from pre-trained model, it should be set to 0.001 and lr-step-size should be set to 5
  • To use DeepShift-PS, the --optimizer must be set to radam in order to obtain good results.

Getting Started

  1. Clone the repo:
git clone https://github.com/mostafaelhoushi/DeepShift.git
  1. Change directory
cd DeepShift
  1. Create virtual environment:
virtualenv venv --prompt="(DeepShift) " --python=/usr/bin/python3.6
  1. (Needs to be done every time you run code) Source the environment:
source venv/bin/activate
  1. Install required packages and build the spfpm package for fixed point
pip install -r requirements.txt
  1. cd into pytorch directroy:
cd pytorch
  1. To list all the available options, you may run:
python .py --help

where can be either mnist, cifar10, imagenet.

When you run any training or evaluation script, you will have the model binary file as well as the training log in ./models/// where:

  • is either shift_q if you pass --shift-type Q, shift_ps if you pass --shift-type PS, or shift_0 if you are running the default FP32 baseline
  • is the number of layers from the start of the model that have been converted to DeepShift. This is determined by the shift-depth argument
  1. Now you can run the different scripts with different options, e.g., a) Train a DeepShift simple fully-connected model on the MNIST dataset, using the PS apprach:
    python mnist.py --shift-depth 3 --shift-type PS --optimizer radam
    
    b) Train a DeepShift simple convolutional model on the MNIST dataset, using the Q approach:
    python mnist.py --type conv --shift-depth 3 --shift-type Q 
    
    c) Train a DeepShift ResNet20 on the CIFAR10 dataset from scratch:
    python cifar10.py --arch resnet20 --pretrained False --shift-depth 1000 --shift-type Q 
    
    d) Train a DeepShift ResNet18 model on the Imagenet dataset using converted pretrained weights for 5 epochs with learning rate 0.001:
    python imagenet.py  --arch resnet18 --pretrained True --shift-depth 1000 --shift-type Q --epochs 5 --lr 0.001
    
    e) Train a DeepShift ResNet18 model on the Imagenet dataset from scratch with an initial learning rate of 0.01:
    python imagenet.py  --arch resnet18 --pretrained False --shift-depth 1000 --shift-type PS --optimizer radam --lr 0.01
    
    f) Train a DeepShift ResNet18 model on the CIFAR10 dataset from scratch with 8-bit fixed point activation (3-bits for integers and 5-bits for fractions):
    python cifar10.py --arch resnet18 --pretrained False --shift-depth 1000 --shift-type PS --optimizer radam --lr 0.01 -ab 3 5
    

Running the Bitwise Shift CUDA & CPU Kernels

  1. cd into DeepShift/pytorch directroy:
cd DeepShift/pytorch
  1. Run the installation script to install our CPU and CUDA kernels that perform matrix multiplication and convolution using bit-wise shifts:
sh install_kernels.sh
  1. Now you can run a model with acutal bit-wise shift kernels in CUDA using the --use-kernel True option. Remember that the kernel only works for inference not training, so you need to add the -e option as well:
python imagenet.py --arch resnet18 -e --shift-depth 1000 --pretrained True --use-kernel True
  1. To compare the latency with a naive regular convolution kernel that does not include cuDNN's other optimizations:
python imagenet.py --arch resnet18 -e --pretrained True --use-kernel True

Results

MNIST

Train from Scratch

Model Original DeepShift-Q DeepShift-PS
Simple FC Model 96.92% [1] 97.03% [2] 98.26% [3]
Simple Conv Model 98.75% [4] 98.81% [5] 99.12% [6]

Commands to reproduce results:

  1. python mnist.py
  2. python mnist.py --shift-depth 1000 --shift-type Q
  3. python mnist.py --shift-depth 1000 --shift-type PS --opt radam
  4. python mnist.py --type conv
  5. python mnist.py --type conv --shift-depth 1000 --shift-type Q
  6. python mnist.py --type conv --shift-depth 1000 --shift-type PS --opt radam

Train from Pre-Trained

Model Original DeepShift-Q DeepShift-PS
Simple FC Model 96.92% [1] 97.85% [7] 98.26% [8]
Simple Conv Model 98.75% [4] 99.15% [9] 99.16% [10]

Commands to reproduce results (assumes you have run commands [1] and [2] in order to have the baseline pretrained weights):

  1. python mnist.py --weights ./models/mnist/simple_linear/shift_0/weights.pth --shift-depth 1000 --shift-type Q --desc from_pretrained
  2. python mnist.py --weights ./models/mnist/simple_linear/shift_0/weights.pth --shift-depth 1000 --shift-type PS --opt radam --desc from_pretrained
  3. python mnist.py --type conv --weights ./models/mnist/simple_conv/shift_0/weights.pth --shift-depth 1000 --shift-type Q --desc from_pretrained
  4. python mnist.py --type conv --weights ./models/mnist/simple_conv/shift_0/weights.pth --shift-depth 1000 --shift-type PS --opt radam --desc from_pretrained

CIFAR10

Train from Scratch

Model Original [11] DeepShift-Q [12] DeepShift-PS [13]
resnet18 94.45% 94.42% 93.20%
mobilenetv2 93.57% 93.63% 92.64%
resnet20 91.79% 89.85% 88.84%
resnet32 92.39% 91.13% 89.97%
resnet44 92.84% 91.29% 90.92%
resnet56 93.46% 91.52% 91.11%

Commands to reproduce results:

  1. python cifar10.py --arch
  2. python cifar10.py --arch --shift-depth 1000 --shift-type Q
  3. python cifar10.py --arch --shift-depth 1000 --shift-type PS --opt radam

Train from Pre-Trained

Model Original [11] DeepShift-Q [14] DeepShift-PS [15]
resnet18 94.45% 94.25% 94.12%
mobilenetv2 93.57% 93.04% 92.78%

Commands to reproduce results (assumes you have run command [11] for the corresponding architecture in order to have the baseline pretrained weights):

  1. python cifar10.py --arch --weights ./models/cifar10//shift_0/checkpoint.pth.tar --shift-depth 1000 --shift-type Q --desc from_pretrained --lr 1e-3 --lr-step 5 --epochs 15
  2. python cifar10.py --arch --weights ./models/cifar10//shift_0/checkpoint.pth.tar --shift-depth 1000 --shift-type PS --opt radam --desc from_pretrained --lr 1e-3 --lr-step 5 --epochs 15

Using Fewer Bits

Model Type Weight Bits Train from Scratch Train from Pre-Trained
resnet18 Original 32 94.45% [11] -
resnet18 DeepShift-PS 5 93.20% [13] 94.12% [15]
resnet18 DeepShift-PS 4 94.12% [16] 94.13% [17]
resnet18 DeepShift-PS 3 92.85% [18] 91.16% [19]
resnet18 DeepShift-PS 2 92.80% [20] 90.68% [21]

Commands to reproduce results (assumes you have run command [11] for the corresponding architecture in order to have the baseline pretrained weights):

  1. python cifar10.py --arch --shift-depth 1000 --shift-type PS -wb 4 --opt radam
  2. python cifar10.py --arch --weights ./models/cifar10//shift_0/checkpoint.pth.tar --shift-depth 1000 --shift-type PS -wb 4 --opt radam --desc from_pretrained --lr 1e-3 --lr-step 5 --epochs 15
  3. python cifar10.py --arch --shift-depth 1000 --shift-type PS -wb 3 --opt radam
  4. python cifar10.py --arch --weights ./models/cifar10//shift_0/checkpoint.pth.tar --shift-depth 1000 --shift-type PS -wb 3 --opt radam --desc from_pretrained --lr 1e-3 --lr-step 5 --epochs 15
  5. python cifar10.py --arch --shift-depth 1000 --shift-type PS -wb 2 --opt radam
  6. python cifar10.py --arch --weights ./models/cifar10//shift_0/checkpoint.pth.tar --shift-depth 1000 --shift-type PS -wb 2 --opt radam --desc from_pretrained --lr 1e-3 --lr-step 5 --epochs 15

ImageNet

Accuracies shown are Top1 / Top5.

Train from Scratch

Model Original [22] DeepShift-Q [23] DeepShift-PS [24]
resnet18 69.76% / 89.08% 65.32% / 86.29% 65.34% / 86.05%
resnet50 76.13% / 92.86% 70.70% / 90.20% 71.90% / 90.20%
vgg16 71.59% / 90.38% 70.87% / 90.09% TBD

Commands to reproduce results:

  1. a) To evaluate PyTorch pretrained models: python imagenet.py --arch --pretrained True -e OR b) To train from scratch: python imagenet.py --arch --pretrained False
  2. python imagenet.py --arch --pretrained False --shift-depth 1000 --shift-type Q --desc from_scratch --lr 0.01
  3. python imagenet.py --arch --pretrained False --shift-depth 1000 --shift-type PS --desc from_scratch --lr 0.01 --opt radam

Train from Pre-Trained

Model Original [22] DeepShift-Q [25] DeepShift-PS [26]
resnet18 69.76% / 89.08% 69.56% / 89.17% 69.27% / 89.00%
resnet50 76.13% / 92.86% 76.33% / 93.05% 75.93% / 92.90%
googlenet 69.78% / 89.53% 70.73% / 90.17% 69.87% / 89.62%
vgg16 71.59% / 90.38% 71.56% / 90.48% 71.39% / 90.33%
alexnet 56.52% / 79.07% 55.81% / 78.79% 55.90% / 78.73%
densenet121 74.43% / 91.97% 74.52% / 92.06% TBD
  1. python imagenet.py --arch --pretrained True --shift-depth 1000 --shift-type Q --desc from_pretrained --lr 1e-3 --lr-step 5 --epochs 15
  2. python imagenet.py --arch --pretrained True --shift-depth 1000 --shift-type PS --desc from_pretrained --lr 1e-3 --lr-step 5 --epochs 15 --opt radam

Using Fewer Bits

Model Type Weight Bits Train from Scratch Train from Pre-Trained
resNet18 Original 32 69.76% / 89.08% -
resNet18 DeepShift-Q 5 65.34% / 86.05% 69.56% / 89.17%
resNet18 DeepShift-PS 5 65.34% / 86.05% 69.27% / 89.00%
resNet18 DeepShift-Q 4 TBD 69.56% / 89.14%
resNet18 DeepShift-PS 4 67.07% / 87.36% 69.02% / 88.73%
resNet18 DeepShift-PS 3 63.11% / 84.45% TBD
resNet18 DeepShift-PS 2 60.80% / 83.01% TBD

Binary Files of Trained Models

Code WalkThrough

  • pytorch: directory containing implementation, tests, and saved models using PyTorch
    • deepshift: directory containing the PyTorch models as well as the CUDA and CPU kernels of LinearShift and Conv2dShift ops
    • unoptimized: directory containing the PyTorch models as well as the CUDA and CPU kernels of the naive implementations of Linear and Conv2d ops
    • mnist.py: example script to train and infer on MNIST dataset using simple models in both their original forms and DeepShift version.
    • cifar10.py: example script to train and infer on CIFAR10 dataset using various models in both their original forms and DeepShift version.
    • imagenet.py: example script to train and infer on Imagenet dataset using various models in both their original forms and DeepShift version.
    • optim: directory containing definition of RAdam and Ranger optimizers. RAdam optimizer is crucial to get DeepShift-PS obtain the accuracies shown here
Comments
  • About trained model

    About trained model

    Thank you for providing your code :) I tried to train your code because there is no trained models. However the accuracy is not high as much as you reported. I really want to inference about DeepShift. Could you provide your files of trained models? Thank you

    opened by SoyeonUm 10
  • Does this code only implement the forward propagation shift operation?

    Does this code only implement the forward propagation shift operation?

    Great job!

    There are some questions. Does this code only implement the forward propagation shift operation? The back-propagation shift operation code is not found? Can you explain where the back propagation code is?

    thank you! @mostafaelhoushi

    opened by mengjingyouling 7
  • We used shift type of PS to train resnet-18 from source but it did not converge.

    We used shift type of PS to train resnet-18 from source but it did not converge.

    The training strictly followed README and converged on MNIST. However, we used shift type of PS to train resnet-18 but it did not converge, and we got top-1 acc of 5.13% on val dataset after 90 epochs training. We ensure no problem in dataset. Could you provide some guides for this?

    opened by PoonKinWang 5
  • round_to_fixed  unsigned tensors

    round_to_fixed unsigned tensors

    hi, @mostafaelhoushi

    The input x is converted to 32bit fixed point in you paper,as follows: `def round_to_fixed(input, integer_bits=16, fraction_bits=16): assert integer_bits >= 1, integer_bits # TODO: Deal with unsigned tensors where there is no sign bit # which is the case with activations to convolution that # are usually the output of a Relu layer if integer_bits == 1: return torch.sign(input) - 1 delta = math.pow(2.0, -(fraction_bits)) bound = math.pow(2.0, integer_bits-1) min_val = - bound max_val = bound - 1 rounded = torch.floor(input / delta) * delta

    clipped_value = torch.clamp(rounded, min_val, max_val)
    return clipped_value`
    

    In the annotation of this function, it is said that this function is about unsigned tensor. But we think it is about signed tensor. For example: signed int8=[-128,127]

    round_to_fixed(-128, integer_bits=8, fraction_bits=8) =-128

    Are we right? Thank you.

    opened by mengjingyouling 4
  • round_to_fixed

    round_to_fixed

    Hi, thanks for the awesome work!

    I am very interested in this work. However, I am new to the area of quantization and have some questions about the round_to_fixed function in deepshift/utils.py.

    This function aims to convert the input from FP32 to fixed-point format (e.g., fix16), to mimic the shift operation and precion of fixed-point input.

    While the range of FP32 is very large, I didn't get how this round_to_fixed function can convert input to merely 16bits. In my opinion the delta should be considered together with the range of input. If the input is in [-1,1], this function works fine (although the bound here should be also 1), so is there any implication that input is in [-1,1]? Or how should I set the default parameters (fractions and intergers) if I want to convert input to fix16?

    Could you give me some comments about the difference of these two implementations? Thanks!!

    opened by msxiaojin 4
  • The Number of Shifted Bits

    The Number of Shifted Bits

    The weight value is cliped to [- 1, 1] in the code: self.shift_range = (-1 * (2(weight_bits - 1) - 2), 0)**

    But the weight should have a small number of values exceeding 1. For example 2 , it is easily obtained by shifting,<<1. Is it better to rewrite the code as follows: self.shift_range = (-1 * (2(weight_bits - 1) - 2), 2**(weight_bits)**

    Thank you. @mostafaelhoushi

    opened by mengjingyouling 3
  • Bump tensorflow from 1.13.1 to 1.15.0

    Bump tensorflow from 1.13.1 to 1.15.0

    Bumps tensorflow from 1.13.1 to 1.15.0.

    Release notes

    Sourced from tensorflow's releases.

    TensorFlow 1.15.0

    Release 1.15.0

    This is the last 1.x release for TensorFlow. We do not expect to update the 1.x branch with features, although we will issue patch releases to fix vulnerabilities for at least one year.

    Major Features and Improvements

    • As announced, tensorflow pip package will by default include GPU support (same as tensorflow-gpu now) for the platforms we currently have GPU support (Linux and Windows). It will work on machines with and without Nvidia GPUs. tensorflow-gpu will still be available, and CPU-only packages can be downloaded at tensorflow-cpu for users who are concerned about package size.
    • TensorFlow 1.15 contains a complete implementation of the 2.0 API in its compat.v2 module. It contains a copy of the 1.15 main module (without contrib) in the compat.v1 module. TensorFlow 1.15 is able to emulate 2.0 behavior using the enable_v2_behavior() function. This enables writing forward compatible code: by explicitly importing either tensorflow.compat.v1 or tensorflow.compat.v2, you can ensure that your code works without modifications against an installation of 1.15 or 2.0.
    • EagerTensor now supports numpy buffer interface for tensors.
    • Add toggles tf.enable_control_flow_v2() and tf.disable_control_flow_v2() for enabling/disabling v2 control flow.
    • Enable v2 control flow as part of tf.enable_v2_behavior() and TF2_BEHAVIOR=1.
    • AutoGraph translates Python control flow into TensorFlow expressions, allowing users to write regular Python inside tf.function-decorated functions. AutoGraph is also applied in functions used with tf.data, tf.distribute and tf.keras APIS.
    • Adds enable_tensor_equality(), which switches the behavior such that:
      • Tensors are no longer hashable.
      • Tensors can be compared with == and !=, yielding a Boolean Tensor with element-wise comparison results. This will be the default behavior in 2.0.
    • Auto Mixed-Precision graph optimizer simplifies converting models to float16 for acceleration on Volta and Turing Tensor Cores. This feature can be enabled by wrapping an optimizer class with tf.train.experimental.enable_mixed_precision_graph_rewrite().
    • Add environment variable TF_CUDNN_DETERMINISTIC. Setting to "true" or "1" forces the selection of deterministic cuDNN convolution and max-pooling algorithms. When this is enabled, the algorithm selection procedure itself is also deterministic.
    • TensorRT
      • Migrate TensorRT conversion sources from contrib to compiler directory in preparation for TF 2.0.
      • Add additional, user friendly TrtGraphConverter API for TensorRT conversion.
      • Expand support for TensorFlow operators in TensorRT conversion (e.g. Gather, Slice, Pack, Unpack, ArgMin, ArgMax,DepthSpaceShuffle).
      • Support TensorFlow operator CombinedNonMaxSuppression in TensorRT conversion which significantly accelerates object detection models.

    Breaking Changes

    • Tensorflow code now produces 2 different pip packages: tensorflow_core containing all the code (in the future it will contain only the private implementation) and tensorflow which is a virtual pip package doing forwarding to tensorflow_core (and in the future will contain only the public API of tensorflow). We don't expect this to be breaking, unless you were importing directly from the implementation.
    • TensorFlow 1.15 is built using devtoolset7 (GCC7) on Ubuntu 16. This may lead to ABI incompatibilities with extensions built against earlier versions of TensorFlow.
    • Deprecated the use of constraint= and .constraint with ResourceVariable.
    • tf.keras:
      • OMP_NUM_THREADS is no longer used by the default Keras config. To configure the number of threads, use tf.config.threading APIs.
      • tf.keras.model.save_model and model.save now defaults to saving a TensorFlow SavedModel.
      • keras.backend.resize_images (and consequently, keras.layers.Upsampling2D) behavior has changed, a bug in the resizing implementation was fixed.
      • Layers now default to float32, and automatically cast their inputs to the layer's dtype. If you had a model that used float64, it will probably silently use float32 in TensorFlow2, and a warning will be issued that starts with Layer "layer-name" is casting an input tensor from dtype float64 to the layer's dtype of float32. To fix, either set the default dtype to float64 with tf.keras.backend.set_floatx('float64'), or pass dtype='float64' to each of the Layer constructors. See tf.keras.layers.Layer for more information.
      • Some tf.assert_* methods now raise assertions at operation creation time (i.e. when this Python line executes) if the input tensors' values are known at that time, not during the session.run(). When this happens, a noop is returned and the input tensors are marked non-feedable. In other words, if they are used as keys in feed_dict argument to session.run(), an error will be raised. Also, because some assert ops don't make it into the graph, the graph structure changes. A different graph can result in different per-op random seeds when they are not given explicitly (most often).

    Bug Fixes and Other Changes

    • tf.estimator:
      • tf.keras.estimator.model_to_estimator now supports exporting to tf.train.Checkpoint format, which allows the saved checkpoints to be compatible with model.load_weights.
      • Fix tests in canned estimators.
      • Expose Head as public API.
      • Fixes critical bugs that help with DenseFeatures usability in TF2
    • tf.data:
      • Promoting unbatch from experimental to core API.
      • Adding support for datasets as inputs to from_tensors and from_tensor_slices and batching and unbatching of nested datasets.
    • tf.keras:
      • tf.keras.estimator.model_to_estimator now supports exporting to tf.train.Checkpoint format, which allows the saved checkpoints to be compatible with model.load_weights.
      • Saving a Keras Model using tf.saved_model.save now saves the list of variables, trainable variables, regularization losses, and the call function.
      • Deprecated tf.keras.experimental.export_saved_model and tf.keras.experimental.function. Please use tf.keras.models.save_model(..., save_format='tf') and tf.keras.models.load_model instead.
      • Add an implementation=3 mode for tf.keras.layers.LocallyConnected2D and tf.keras.layers.LocallyConnected1D layers using tf.SparseTensor to store weights, allowing a dramatic speedup for large sparse models.
    ... (truncated)
    Changelog

    Sourced from tensorflow's changelog.

    Release 1.15.0

    This is the last 1.x release for TensorFlow. We do not expect to update the 1.x branch with features, although we will issue patch releases to fix vulnerabilities for at least one year.

    Major Features and Improvements

    • As announced, tensorflow pip package will by default include GPU support (same as tensorflow-gpu now) for the platforms we currently have GPU support (Linux and Windows). It will work on machines with and without Nvidia GPUs. tensorflow-gpu will still be available, and CPU-only packages can be downloaded at tensorflow-cpu for users who are concerned about package size.
    • TensorFlow 1.15 contains a complete implementation of the 2.0 API in its compat.v2 module. It contains a copy of the 1.15 main module (without contrib) in the compat.v1 module. TensorFlow 1.15 is able to emulate 2.0 behavior using the enable_v2_behavior() function. This enables writing forward compatible code: by explicitly importing either tensorflow.compat.v1 or tensorflow.compat.v2, you can ensure that your code works without modifications against an installation of 1.15 or 2.0.
    • EagerTensor now supports numpy buffer interface for tensors.
    • Add toggles tf.enable_control_flow_v2() and tf.disable_control_flow_v2() for enabling/disabling v2 control flow.
    • Enable v2 control flow as part of tf.enable_v2_behavior() and TF2_BEHAVIOR=1.
    • AutoGraph translates Python control flow into TensorFlow expressions, allowing users to write regular Python inside tf.function-decorated functions. AutoGraph is also applied in functions used with tf.data, tf.distribute and tf.keras APIS.
    • Adds enable_tensor_equality(), which switches the behavior such that:
      • Tensors are no longer hashable.
      • Tensors can be compared with == and !=, yielding a Boolean Tensor with element-wise comparison results. This will be the default behavior in 2.0.

    Breaking Changes

    • Tensorflow code now produces 2 different pip packages: tensorflow_core containing all the code (in the future it will contain only the private implementation) and tensorflow which is a virtual pip package doing forwarding to tensorflow_core (and in the future will contain only the public API of tensorflow). We don't expect this to be breaking, unless you were importing directly from the implementation.
    • TensorFlow 1.15 is built using devtoolset7 (GCC7) on Ubuntu 16. This may lead to ABI incompatibilities with extensions built against earlier versions of TensorFlow.
    • Deprecated the use of constraint= and .constraint with ResourceVariable.
    • tf.keras:
      • OMP_NUM_THREADS is no longer used by the default Keras config. To configure the number of threads, use tf.config.threading APIs.
      • tf.keras.model.save_model and model.save now defaults to saving a TensorFlow SavedModel.
      • keras.backend.resize_images (and consequently, keras.layers.Upsampling2D) behavior has changed, a bug in the resizing implementation was fixed.
      • Layers now default to float32, and automatically cast their inputs to the layer's dtype. If you had a model that used float64, it will probably silently use float32 in TensorFlow2, and a warning will be issued that starts with Layer "layer-name" is casting an input tensor from dtype float64 to the layer's dtype of float32. To fix, either set the default dtype to float64 with tf.keras.backend.set_floatx('float64'), or pass dtype='float64' to each of the Layer constructors. See tf.keras.layers.Layer for more information.
      • Some tf.assert_* methods now raise assertions at operation creation time (i.e. when this Python line executes) if the input tensors' values are known at that time, not during the session.run(). When this happens, a noop is returned and the input tensors are marked non-feedable. In other words, if they are used as keys in feed_dict argument to session.run(), an error will be raised. Also, because some assert ops don't make it into the graph, the graph structure changes. A different graph can result in different per-op random seeds when they are not given explicitly (most often).

    Bug Fixes and Other Changes

    • tf.estimator:
      • tf.keras.estimator.model_to_estimator now supports exporting to tf.train.Checkpoint format, which allows the saved checkpoints to be compatible with model.load_weights.
      • Fix tests in canned estimators.
      • Expose Head as public API.
      • Fixes critical bugs that help with DenseFeatures usability in TF2
    • tf.data:
      • Promoting unbatch from experimental to core API.
      • Adding support for datasets as inputs to from_tensors and from_tensor_slices and batching and unbatching of nested datasets.
    • tf.keras:
      • tf.keras.estimator.model_to_estimator now supports exporting to tf.train.Checkpoint format, which allows the saved checkpoints to be compatible with model.load_weights.
      • Saving a Keras Model using tf.saved_model.save now saves the list of variables, trainable variables, regularization losses, and the call function.
      • Deprecated tf.keras.experimental.export_saved_model and tf.keras.experimental.function. Please use tf.keras.models.save_model(..., save_format='tf') and tf.keras.models.load_model instead.
      • Add an implementation=3 mode for tf.keras.layers.LocallyConnected2D and tf.keras.layers.LocallyConnected1D layers using tf.SparseTensor to store weights, allowing a dramatic speedup for large sparse models.
      • Enable the Keras compile API experimental_run_tf_function flag by default. This flag enables single training/eval/predict execution path. With this 1. All input types are converted to Dataset. 2. When distribution strategy is not specified this goes through the no-op distribution strategy path. 3. Execution is wrapped in tf.function unless run_eagerly=True is set in compile.
      • Raise error if batch_size argument is used when input is dataset/generator/keras sequence.
    • tf.lite
      • Add GATHER support to NN API delegate.
      • tflite object detection script has a debug mode.
      • Add delegate support for QUANTIZE.
      • Added evaluation script for COCO minival.
      • Add delegate support for QUANTIZED_16BIT_LSTM.
      • Converts hardswish subgraphs into atomic ops.
    • Add support for defaulting the value of cycle_length argument of tf.data.Dataset.interleave to the number of schedulable CPU cores.
    ... (truncated)
    Commits
    • 590d6ee Merge pull request #31861 from tensorflow-jenkins/relnotes-1.15.0rc0-16184
    • b27ac43 Update RELEASE.md
    • 07bf663 Merge pull request #33213 from Intel-tensorflow/mkl-dnn-0.20.6
    • 46f50ff Merge pull request #33262 from tensorflow/ggadde-1-15-cp2
    • 49c154e Merge pull request #33263 from tensorflow/ggadde-1-15-final-version
    • a16adeb Update TensorFlow version to 1.15.0 in preparation for final relase.
    • 8d71a87 Add saving of loaded/trained compatibility models in test and fix a compatibi...
    • 8c48aff [Intel Mkl] Upgrading MKL-DNN to 0.20.6 to fix SGEMM regression
    • 38ea9bb Merge pull request #33120 from tensorflow/perf
    • a8ef0f5 Automated rollback of commit db7e43192d405973c6c50f6e60e831a198bb4a49
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot ignore this [patch|minor|major] version will close this PR and stop Dependabot creating any more for this minor/major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Bump pillow from 6.0.0 to 6.2.0

    Bump pillow from 6.0.0 to 6.2.0

    Bumps pillow from 6.0.0 to 6.2.0.

    Release notes

    Sourced from pillow's releases.

    6.2.0

    https://pillow.readthedocs.io/en/stable/releasenotes/6.2.0.html

    6.1.0

    https://pillow.readthedocs.io/en/stable/releasenotes/6.1.0.html

    Changelog

    Sourced from pillow's changelog.

    6.2.0 (2019-10-01)

    • Catch buffer overruns #4104 [radarhere]

    • Initialize rows_per_strip when RowsPerStrip tag is missing #4034 [cgohlke, radarhere]

    • Raise error if TIFF dimension is a string #4103 [radarhere]

    • Added decompression bomb checks #4102 [radarhere]

    • Fix ImageGrab.grab DPI scaling on Windows 10 version 1607+ #4000 [nulano, radarhere]

    • Corrected negative seeks #4101 [radarhere]

    • Added argument to capture all screens on Windows #3950 [nulano, radarhere]

    • Updated warning to specify when Image.frombuffer defaults will change #4086 [radarhere]

    • Changed WindowsViewer format to PNG #4080 [radarhere]

    • Use TIFF orientation #4063 [radarhere]

    • Raise the same error if a truncated image is loaded a second time #3965 [radarhere]

    • Lazily use ImageFileDirectory_v1 values from Exif #4031 [radarhere]

    • Improved HSV conversion #4004 [radarhere]

    • Added text stroking #3978 [radarhere, hugovk]

    • No more deprecated bdist_wininst .exe installers #4029 [hugovk]

    • Do not allow floodfill to extend into negative coordinates #4017 [radarhere]

    ... (truncated)
    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot ignore this [patch|minor|major] version will close this PR and stop Dependabot creating any more for this minor/major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Bump urllib3 from 1.24.1 to 1.24.2

    Bump urllib3 from 1.24.1 to 1.24.2

    Bumps urllib3 from 1.24.1 to 1.24.2.

    Changelog

    Sourced from urllib3's changelog.

    1.24.2 (2019-04-17)

    • Don't load system certificates by default when any other ca_certs, ca_certs_dir or ssl_context parameters are specified.

    • Remove Authorization header regardless of case when redirecting to cross-site. (Issue #1510)

    • Add support for IPv6 addresses in subjectAltName section of certificates. (Issue #1269)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot ignore this [patch|minor|major] version will close this PR and stop Dependabot creating any more for this minor/major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Bump tensorflow-gpu from 1.13.1 to 1.15.0

    Bump tensorflow-gpu from 1.13.1 to 1.15.0

    Bumps tensorflow-gpu from 1.13.1 to 1.15.0.

    Release notes

    Sourced from tensorflow-gpu's releases.

    TensorFlow 1.15.0

    Release 1.15.0

    This is the last 1.x release for TensorFlow. We do not expect to update the 1.x branch with features, although we will issue patch releases to fix vulnerabilities for at least one year.

    Major Features and Improvements

    • As announced, tensorflow pip package will by default include GPU support (same as tensorflow-gpu now) for the platforms we currently have GPU support (Linux and Windows). It will work on machines with and without Nvidia GPUs. tensorflow-gpu will still be available, and CPU-only packages can be downloaded at tensorflow-cpu for users who are concerned about package size.
    • TensorFlow 1.15 contains a complete implementation of the 2.0 API in its compat.v2 module. It contains a copy of the 1.15 main module (without contrib) in the compat.v1 module. TensorFlow 1.15 is able to emulate 2.0 behavior using the enable_v2_behavior() function. This enables writing forward compatible code: by explicitly importing either tensorflow.compat.v1 or tensorflow.compat.v2, you can ensure that your code works without modifications against an installation of 1.15 or 2.0.
    • EagerTensor now supports numpy buffer interface for tensors.
    • Add toggles tf.enable_control_flow_v2() and tf.disable_control_flow_v2() for enabling/disabling v2 control flow.
    • Enable v2 control flow as part of tf.enable_v2_behavior() and TF2_BEHAVIOR=1.
    • AutoGraph translates Python control flow into TensorFlow expressions, allowing users to write regular Python inside tf.function-decorated functions. AutoGraph is also applied in functions used with tf.data, tf.distribute and tf.keras APIS.
    • Adds enable_tensor_equality(), which switches the behavior such that:
      • Tensors are no longer hashable.
      • Tensors can be compared with == and !=, yielding a Boolean Tensor with element-wise comparison results. This will be the default behavior in 2.0.
    • Auto Mixed-Precision graph optimizer simplifies converting models to float16 for acceleration on Volta and Turing Tensor Cores. This feature can be enabled by wrapping an optimizer class with tf.train.experimental.enable_mixed_precision_graph_rewrite().
    • Add environment variable TF_CUDNN_DETERMINISTIC. Setting to "true" or "1" forces the selection of deterministic cuDNN convolution and max-pooling algorithms. When this is enabled, the algorithm selection procedure itself is also deterministic.
    • TensorRT
      • Migrate TensorRT conversion sources from contrib to compiler directory in preparation for TF 2.0.
      • Add additional, user friendly TrtGraphConverter API for TensorRT conversion.
      • Expand support for TensorFlow operators in TensorRT conversion (e.g. Gather, Slice, Pack, Unpack, ArgMin, ArgMax,DepthSpaceShuffle).
      • Support TensorFlow operator CombinedNonMaxSuppression in TensorRT conversion which significantly accelerates object detection models.

    Breaking Changes

    • Tensorflow code now produces 2 different pip packages: tensorflow_core containing all the code (in the future it will contain only the private implementation) and tensorflow which is a virtual pip package doing forwarding to tensorflow_core (and in the future will contain only the public API of tensorflow). We don't expect this to be breaking, unless you were importing directly from the implementation.
    • TensorFlow 1.15 is built using devtoolset7 (GCC7) on Ubuntu 16. This may lead to ABI incompatibilities with extensions built against earlier versions of TensorFlow.
    • Deprecated the use of constraint= and .constraint with ResourceVariable.
    • tf.keras:
      • OMP_NUM_THREADS is no longer used by the default Keras config. To configure the number of threads, use tf.config.threading APIs.
      • tf.keras.model.save_model and model.save now defaults to saving a TensorFlow SavedModel.
      • keras.backend.resize_images (and consequently, keras.layers.Upsampling2D) behavior has changed, a bug in the resizing implementation was fixed.
      • Layers now default to float32, and automatically cast their inputs to the layer's dtype. If you had a model that used float64, it will probably silently use float32 in TensorFlow2, and a warning will be issued that starts with Layer "layer-name" is casting an input tensor from dtype float64 to the layer's dtype of float32. To fix, either set the default dtype to float64 with tf.keras.backend.set_floatx('float64'), or pass dtype='float64' to each of the Layer constructors. See tf.keras.layers.Layer for more information.
      • Some tf.assert_* methods now raise assertions at operation creation time (i.e. when this Python line executes) if the input tensors' values are known at that time, not during the session.run(). When this happens, a noop is returned and the input tensors are marked non-feedable. In other words, if they are used as keys in feed_dict argument to session.run(), an error will be raised. Also, because some assert ops don't make it into the graph, the graph structure changes. A different graph can result in different per-op random seeds when they are not given explicitly (most often).

    Bug Fixes and Other Changes

    • tf.estimator:
      • tf.keras.estimator.model_to_estimator now supports exporting to tf.train.Checkpoint format, which allows the saved checkpoints to be compatible with model.load_weights.
      • Fix tests in canned estimators.
      • Expose Head as public API.
      • Fixes critical bugs that help with DenseFeatures usability in TF2
    • tf.data:
      • Promoting unbatch from experimental to core API.
      • Adding support for datasets as inputs to from_tensors and from_tensor_slices and batching and unbatching of nested datasets.
    • tf.keras:
      • tf.keras.estimator.model_to_estimator now supports exporting to tf.train.Checkpoint format, which allows the saved checkpoints to be compatible with model.load_weights.
      • Saving a Keras Model using tf.saved_model.save now saves the list of variables, trainable variables, regularization losses, and the call function.
      • Deprecated tf.keras.experimental.export_saved_model and tf.keras.experimental.function. Please use tf.keras.models.save_model(..., save_format='tf') and tf.keras.models.load_model instead.
      • Add an implementation=3 mode for tf.keras.layers.LocallyConnected2D and tf.keras.layers.LocallyConnected1D layers using tf.SparseTensor to store weights, allowing a dramatic speedup for large sparse models.
    ... (truncated)
    Changelog

    Sourced from tensorflow-gpu's changelog.

    Release 1.15.0

    This is the last 1.x release for TensorFlow. We do not expect to update the 1.x branch with features, although we will issue patch releases to fix vulnerabilities for at least one year.

    Major Features and Improvements

    • As announced, tensorflow pip package will by default include GPU support (same as tensorflow-gpu now) for the platforms we currently have GPU support (Linux and Windows). It will work on machines with and without Nvidia GPUs. tensorflow-gpu will still be available, and CPU-only packages can be downloaded at tensorflow-cpu for users who are concerned about package size.
    • TensorFlow 1.15 contains a complete implementation of the 2.0 API in its compat.v2 module. It contains a copy of the 1.15 main module (without contrib) in the compat.v1 module. TensorFlow 1.15 is able to emulate 2.0 behavior using the enable_v2_behavior() function. This enables writing forward compatible code: by explicitly importing either tensorflow.compat.v1 or tensorflow.compat.v2, you can ensure that your code works without modifications against an installation of 1.15 or 2.0.
    • EagerTensor now supports numpy buffer interface for tensors.
    • Add toggles tf.enable_control_flow_v2() and tf.disable_control_flow_v2() for enabling/disabling v2 control flow.
    • Enable v2 control flow as part of tf.enable_v2_behavior() and TF2_BEHAVIOR=1.
    • AutoGraph translates Python control flow into TensorFlow expressions, allowing users to write regular Python inside tf.function-decorated functions. AutoGraph is also applied in functions used with tf.data, tf.distribute and tf.keras APIS.
    • Adds enable_tensor_equality(), which switches the behavior such that:
      • Tensors are no longer hashable.
      • Tensors can be compared with == and !=, yielding a Boolean Tensor with element-wise comparison results. This will be the default behavior in 2.0.

    Breaking Changes

    • Tensorflow code now produces 2 different pip packages: tensorflow_core containing all the code (in the future it will contain only the private implementation) and tensorflow which is a virtual pip package doing forwarding to tensorflow_core (and in the future will contain only the public API of tensorflow). We don't expect this to be breaking, unless you were importing directly from the implementation.
    • TensorFlow 1.15 is built using devtoolset7 (GCC7) on Ubuntu 16. This may lead to ABI incompatibilities with extensions built against earlier versions of TensorFlow.
    • Deprecated the use of constraint= and .constraint with ResourceVariable.
    • tf.keras:
      • OMP_NUM_THREADS is no longer used by the default Keras config. To configure the number of threads, use tf.config.threading APIs.
      • tf.keras.model.save_model and model.save now defaults to saving a TensorFlow SavedModel.
      • keras.backend.resize_images (and consequently, keras.layers.Upsampling2D) behavior has changed, a bug in the resizing implementation was fixed.
      • Layers now default to float32, and automatically cast their inputs to the layer's dtype. If you had a model that used float64, it will probably silently use float32 in TensorFlow2, and a warning will be issued that starts with Layer "layer-name" is casting an input tensor from dtype float64 to the layer's dtype of float32. To fix, either set the default dtype to float64 with tf.keras.backend.set_floatx('float64'), or pass dtype='float64' to each of the Layer constructors. See tf.keras.layers.Layer for more information.
      • Some tf.assert_* methods now raise assertions at operation creation time (i.e. when this Python line executes) if the input tensors' values are known at that time, not during the session.run(). When this happens, a noop is returned and the input tensors are marked non-feedable. In other words, if they are used as keys in feed_dict argument to session.run(), an error will be raised. Also, because some assert ops don't make it into the graph, the graph structure changes. A different graph can result in different per-op random seeds when they are not given explicitly (most often).

    Bug Fixes and Other Changes

    • tf.estimator:
      • tf.keras.estimator.model_to_estimator now supports exporting to tf.train.Checkpoint format, which allows the saved checkpoints to be compatible with model.load_weights.
      • Fix tests in canned estimators.
      • Expose Head as public API.
      • Fixes critical bugs that help with DenseFeatures usability in TF2
    • tf.data:
      • Promoting unbatch from experimental to core API.
      • Adding support for datasets as inputs to from_tensors and from_tensor_slices and batching and unbatching of nested datasets.
    • tf.keras:
      • tf.keras.estimator.model_to_estimator now supports exporting to tf.train.Checkpoint format, which allows the saved checkpoints to be compatible with model.load_weights.
      • Saving a Keras Model using tf.saved_model.save now saves the list of variables, trainable variables, regularization losses, and the call function.
      • Deprecated tf.keras.experimental.export_saved_model and tf.keras.experimental.function. Please use tf.keras.models.save_model(..., save_format='tf') and tf.keras.models.load_model instead.
      • Add an implementation=3 mode for tf.keras.layers.LocallyConnected2D and tf.keras.layers.LocallyConnected1D layers using tf.SparseTensor to store weights, allowing a dramatic speedup for large sparse models.
      • Enable the Keras compile API experimental_run_tf_function flag by default. This flag enables single training/eval/predict execution path. With this 1. All input types are converted to Dataset. 2. When distribution strategy is not specified this goes through the no-op distribution strategy path. 3. Execution is wrapped in tf.function unless run_eagerly=True is set in compile.
      • Raise error if batch_size argument is used when input is dataset/generator/keras sequence.
    • tf.lite
      • Add GATHER support to NN API delegate.
      • tflite object detection script has a debug mode.
      • Add delegate support for QUANTIZE.
      • Added evaluation script for COCO minival.
      • Add delegate support for QUANTIZED_16BIT_LSTM.
      • Converts hardswish subgraphs into atomic ops.
    • Add support for defaulting the value of cycle_length argument of tf.data.Dataset.interleave to the number of schedulable CPU cores.
    ... (truncated)
    Commits
    • 590d6ee Merge pull request #31861 from tensorflow-jenkins/relnotes-1.15.0rc0-16184
    • b27ac43 Update RELEASE.md
    • 07bf663 Merge pull request #33213 from Intel-tensorflow/mkl-dnn-0.20.6
    • 46f50ff Merge pull request #33262 from tensorflow/ggadde-1-15-cp2
    • 49c154e Merge pull request #33263 from tensorflow/ggadde-1-15-final-version
    • a16adeb Update TensorFlow version to 1.15.0 in preparation for final relase.
    • 8d71a87 Add saving of loaded/trained compatibility models in test and fix a compatibi...
    • 8c48aff [Intel Mkl] Upgrading MKL-DNN to 0.20.6 to fix SGEMM regression
    • 38ea9bb Merge pull request #33120 from tensorflow/perf
    • a8ef0f5 Automated rollback of commit db7e43192d405973c6c50f6e60e831a198bb4a49
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot ignore this [patch|minor|major] version will close this PR and stop Dependabot creating any more for this minor/major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Bump werkzeug from 0.15.2 to 0.15.3

    Bump werkzeug from 0.15.2 to 0.15.3

    Bumps werkzeug from 0.15.2 to 0.15.3.

    Release notes

    Sourced from werkzeug's releases.

    0.15.3

    • Blog: https://palletsprojects.com/blog/werkzeug-0-15-3-released/
    • Changes: https://werkzeug.palletsprojects.com/en/0.15.x/changes/#version-0-15-3
    Changelog

    Sourced from werkzeug's changelog.

    Version 0.15.3

    Released 2019-05-14

    • Properly handle multi-line header folding in development server in Python 2.7. (:issue:1080)
    • Restore the response argument to :exc:~exceptions.Unauthorized. (:pr:1527)
    • :exc:~exceptions.Unauthorized doesn't add the WWW-Authenticate header if www_authenticate is not given. (:issue:1516)
    • The default URL converter correctly encodes bytes to string rather than representing them with b''. (:issue:1502)
    • Fix the filename format string in :class:~middleware.profiler.ProfilerMiddleware to correctly handle float values. (:issue:1511)
    • Update :class:~middleware.lint.LintMiddleware to work on Python 3. (:issue:1510)
    • The debugger detects cycles in chained exceptions and does not time out in that case. (:issue:1536)
    • When running the development server in Docker, the debugger security pin is now unique per container.
    Commits
    • 9b1123a release version 0.15.3
    • 00bc43b unique debugger pin in Docker containers
    • 2cbdf2b Merge pull request #1542 from asottile/exceptions_arent_always_hashable
    • 0e669f6 Fix unhashable exception types
    • bdc17e4 Merge pull request #1540 from pallets/break-tb-cycle
    • 44e38c2 break cycle in chained exceptions
    • 777500b Merge pull request #1518 from NiklasMM/fix/1510_lint-middleware-python3-compa...
    • e00c7c2 Make LintMiddleware Python 3 compatible and add tests
    • d590cc7 Merge pull request #1539 from pallets/profiler-format
    • 0388fc9 update filename_format for ProfilerMiddleware.
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot ignore this [patch|minor|major] version will close this PR and stop Dependabot creating any more for this minor/major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • test mAP

    test mAP

    The minist.py code implements the training process of the Deepshift method. As it is a complete process, the model is trained and then tested. The test model is generated using model. eval (). After training, save the model weight file (. pth).This training process can achieve high accuracy(train_log.csv).

    However, we found that loading the generated weight file(weights.pth) for inference(test fuction) will reduce the accuracy.

    Do you have any suggestions?

    Thanks!

    opened by mengjingyouling 2
  • Error When Shifting Twice

    Error When Shifting Twice

    Copying the question by @mengjingyouling from this issue to create a new issue:

    We also want to discuss a problem with you. In your paper, the shift network is applied in classification network, not target detection. What do you think? Is there a decline in the accuracy?

    Because the shift 1 bit will lead to some accuracy loss. We want to shift twice to solve it. For example: 10 = 8 + 2( shift 3 bits + shift 1 bit). Therefore, we modify the code as follows:

    def get_shift_and_sign(x, rounding='deterministic'):
      sign = torch.sign(x)
      x_abs = torch.abs(x)
      shift1 = round(torch.log(x_abs) / np.log(2), rounding)
      wr1 = 2 ** shift1
      w1 = x_abs-wr1
      shift2 = round(torch.log(w1) / np.log(2), rounding)
      return shift1,shift2, sign
    
    def round_power_of_2(x, rounding='deterministic'):
    
      shift1,shift2,sign = get_shift_and_sign(x, rounding)
      x_rounded = (2.0 ** shift1+2.0 ** shift2) * sign
      return x_rounded
    

    However, the input in class Conv2dShiftQ(_ConvNdShiftQ): function will become Nan, which should be caused by data overflow:

    class Conv2dShiftQ(_ConvNdShiftQ):
    ... ....
    ... ...
    
      #@weak_script_method
      def forward(self, input):
        print("--------------------------------------forward---------------------------------------------------")
        print("input======",input)
    

    Can you give some suggestions to solve it? Thank you very much.

    opened by mostafaelhoushi 7
  • Loading Weights from `weights.pth` Not Working

    Loading Weights from `weights.pth` Not Working

    Passing a weights.pth file to the --weights is not working properly. It probably doesn't load the weights.

    While passing a checkpoint.pth.tar file to the --weights is working properly

    opened by mostafaelhoushi 0
  • Round to Fixed to Deal with Unsigned Tensors

    Round to Fixed to Deal with Unsigned Tensors

    We need to deal with unsigned tensors where there is no sign bit which is the case with activations to convolution that are usually the output of a Relu layer. This will save us one bit when we want to quantize activations to lower bitwidths.

    image

    opened by mostafaelhoushi 0
  • shift_kernel & shift_cuda_kernel compiled but can not import

    shift_kernel & shift_cuda_kernel compiled but can not import

    Successfully setup everything, and compiled shift_kernel, but when import shift_kernel, error message appeared:

    ImportError: /home/grant/venv/lib/python3.6/site-packages/shift_kernel-0.0.0-py3.6-linux-x86_64.egg/shift_kernel.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN6caffe26detail36_typeMetaDataInstance_preallocated_4E

    For shift_cuda_kernel, the error message is: Segmentation fault (core dumped)

    I am working on Ubuntu 18.04, others are as required.

    opened by Grant-Tao 8
Releases(v1.0)
Owner
Mostafa Elhoushi
BSc from Ain Shams University, Egypt and PhD from Queen's University, Canada
Mostafa Elhoushi
The official repo of the CVPR2021 oral paper: Representative Batch Normalization with Feature Calibration

Representative Batch Normalization (RBN) with Feature Calibration The official implementation of the CVPR2021 oral paper: Representative Batch Normali

Open source projects of ShangHua-Gao 76 Nov 09, 2022
Neural Radiance Fields Using PyTorch

This project is a PyTorch implementation of Neural Radiance Fields (NeRF) for reproduction of results whilst running at a faster speed.

Vedant Ghodke 1 Feb 11, 2022
Fastshap: A fast, approximate shap kernel

fastshap: A fast, approximate shap kernel fastshap was designed to be: Fast Calculating shap values can take an extremely long time. fastshap utilizes

Samuel Wilson 22 Sep 24, 2022
Multi-task Learning of Order-Consistent Causal Graphs (NeuRIPs 2021)

Multi-task Learning of Order-Consistent Causal Graphs (NeuRIPs 2021) Authors: Xinshi Chen, Haoran Sun, Caleb Ellington, Eric Xing, Le Song Link to pap

Xinshi Chen 2 Dec 20, 2021
The AugNet Python module contains functions for the fast computation of image similarity.

AugNet AugNet: End-to-End Unsupervised Visual Representation Learning with Image Augmentation arxiv link In our work, we propose AugNet, a new deep le

Ming 74 Dec 28, 2022
Occlusion robust 3D face reconstruction model in CFR-GAN (WACV 2022)

Occlusion Robust 3D face Reconstruction Yeong-Joon Ju, Gun-Hee Lee, Jung-Ho Hong, and Seong-Whan Lee Code for Occlusion Robust 3D Face Reconstruction

Yeongjoon 31 Dec 19, 2022
Model Serving Made Easy

The easiest way to build Machine Learning APIs BentoML makes moving trained ML models to production easy: Package models trained with any ML framework

BentoML 4.4k Jan 08, 2023
Deep Learning as a Cloud API Service.

Deep API Deep Learning as Cloud APIs. This project provides pre-trained deep learning models as a cloud API service. A web interface is available as w

Wu Han 4 Jan 06, 2023
Measuring Coding Challenge Competence With APPS

Measuring Coding Challenge Competence With APPS This is the repository for Measuring Coding Challenge Competence With APPS by Dan Hendrycks*, Steven B

Dan Hendrycks 218 Dec 27, 2022
PyTorch implementations of deep reinforcement learning algorithms and environments

Deep Reinforcement Learning Algorithms with PyTorch This repository contains PyTorch implementations of deep reinforcement learning algorithms and env

Petros Christodoulou 4.7k Jan 04, 2023
3D2Unet: 3D Deformable Unet for Low-Light Video Enhancement (PRCV2021)

3DDUNET This is the code for 3D2Unet: 3D Deformable Unet for Low-Light Video Enhancement (PRCV2021) Conference Paper Link Dataset We use SMOID dataset

1 Jan 07, 2022
MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Images

Main repo for ECCV 2020 paper MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Images. visual.cs.brown.edu/matryodshka

Brown University Visual Computing Group 75 Dec 13, 2022
[WACV21] Code for our paper: Samuel, Atzmon and Chechik, "From Generalized zero-shot learning to long-tail with class descriptors"

DRAGON: From Generalized zero-shot learning to long-tail with class descriptors Paper Project Website Video Overview DRAGON learns to correct the bias

Dvir Samuel 25 Dec 06, 2022
TUPÃ was developed to analyze electric field properties in molecular simulations

TUPÃ: Electric field analyses for molecular simulations What is TUPÃ? TUPÃ (pronounced as tu-pan) is a python algorithm that employs MDAnalysis engine

Marcelo D. Polêto 10 Jul 17, 2022
Semi-Supervised Signed Clustering Graph Neural Network (and Implementation of Some Spectral Methods)

SSSNET SSSNET: Semi-Supervised Signed Network Clustering For details, please read our paper. Environment Setup Overview The project has been tested on

Yixuan He 9 Nov 24, 2022
NaturalProofs: Mathematical Theorem Proving in Natural Language

NaturalProofs: Mathematical Theorem Proving in Natural Language NaturalProofs: Mathematical Theorem Proving in Natural Language Sean Welleck, Jiacheng

Sean Welleck 83 Jan 05, 2023
Code for the paper Learning the Predictability of the Future

Learning the Predictability of the Future Code from the paper Learning the Predictability of the Future. Website of the project in hyperfuture.cs.colu

Computer Vision Lab at Columbia University 139 Nov 18, 2022
Pytorch implementation for "Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasets" (ECCV 2020 Spotlight)

Distribution-Balanced Loss [Paper] The implementation of our paper Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasets (

Tong WU 304 Dec 22, 2022
Official implementation of VaxNeRF (Voxel-Accelearated NeRF).

VaxNeRF Paper | Google Colab This is the official implementation of VaxNeRF (Voxel-Accelearated NeRF). VaxNeRF provides very fast training and slightl

naruya 132 Nov 21, 2022
Image marine sea litter prediction Shiny

MARLITE Shiny app for floating marine litter detection in aerial images. This directory contains the instructions and software needed to install the S

19 Dec 22, 2022