Streaming over lightweight data transformations

Overview

slide

PyPI version Build Status Codecoverage Codacy Badge License DOI

Description

Data augmentation libarary for Deep Learning, which supports images, segmentation masks, labels and keypoints. Furthermore, SOLT is fast and has OpenCV in its backend. Full auto-generated docs and examples are available here: https://mipt-oulu.github.io/solt/.

Features

  • Support of Images, masks and keypoints for all the transforms (including multiple items at the time)
  • Fast and PyTorch-integrated
  • Convenient and flexible serialization API
  • Excellent documentation
  • Easy to extend
  • 100% Code coverage

Examples

Images: Cats Images + Keypoints: Cats Medical Images + Binary Masks: Brain MRI Medical Images + Multiclass Masks Knee MRI

E.g. the last row is generated using the following transforms stream.

stream = solt.Stream([
    slt.Rotate(angle_range=(-20, 20), p=1, padding='r'),
    slt.Crop((256, 256)),
    solt.SelectiveStream([
        slt.GammaCorrection(gamma_range=0.5, p=1),
        slt.Noise(gain_range=0.1, p=1),
        slt.Blur()    
    ], n=3)
])

img_aug, mask_aug = stream({'image': img, 'mask': mask})

If you want to visualize the results, you need to modify the execution of the transforms:

img_aug, mask_aug = stream({'image': img, 'mask': mask}, return_torch=False).data

Installation

The most recent version is available in pip:

pip install solt

You can fetch the most fresh changes from this repository:

pip install git+https://github.com/MIPT-Oulu/solt

Benchmark

We propose a fair benchmark based on the refactored version of the one proposed by albumentations team, but here, we also convert the results into a PyTorch tensor and do the ImageNet normalization. The following numbers support a realistic and honest comparison between the libraries (number of images per second, the higher - the better):

albumentations
0.4.3
torchvision (Pillow-SIMD backend)
0.5.0
augmentor
0.2.8
solt
0.1.9
HorizontalFlip 2253 2549 2561 3530
VerticalFlip 2380 2557 2572 3740
RotateAny 1479 1389 670 2070
Crop224 2566 1966 1981 4281
Crop128 5467 5738 5720 7186
Crop64 9285 9112 9049 10345
Crop32 11979 10550 10607 12348
Pad300 1642 109 - 2631
VHFlipRotateCrop 1574 1334 616 1889
HFlipCrop 2391 1943 1917 3572

Python and library versions: Python 3.7.0 (default, Oct 9 2018, 10:31:47) [GCC 7.3.0], numpy 1.18.1, pillow-simd 7.0.0.post3, opencv-python 4.2.0.32, scikit-image 0.16.2, scipy 1.4.1.

The code was run on AMD Threadripper 1900. Please find the details about the benchmark here.

How to contribute

Follow the guidelines described here.

Author

Aleksei Tiulpin, Research Unit of Medical Imaging, Physics and Technology, University of Oulu, Finalnd.

How to cite

If you use SOLT and cite it in your research, please, don't hesitate to sent an email to Aleksei Tiulpin. All the papers that use SOLT are listed here.

@misc{solt2019,
  author       = {Aleksei Tiulpin},
  title        = {SOLT: Streaming over Lightweight Transformations},
  month        = jul,
  year         = 2019,
  version      = {v0.1.9},
  doi          = {10.5281/zenodo.3702819},
  url          = {https://doi.org/10.5281/zenodo.3702819}
}
Comments
  • Crashed without clear error message when tranforming multiple images with different shapes

    Crashed without clear error message when tranforming multiple images with different shapes

    Code to reproduce:

    import solt
    import solt.transforms as slt
    import numpy as np
    
    
    if __name__ == "__main__":
        trf = solt.Stream([slt.Resize(resize_to=(50, 50)),
                           slt.Flip()])
        img1 = np.ones((100, 100, 3), dtype=np.int32)
        img2 = np.ones((110, 110, 3), dtype=np.int32)
        trf_img = trf({'images': (img1, img2)})
        print('Done')
    

    Adding a resize transformation at the beginning doesn't help. A clear error message is needed for this case.

    opened by hoanghng 4
  • NameError: name 'ktps' is not defined

    NameError: name 'ktps' is not defined

    While testing one of the notebook algorithm, this error occured: kpts = None for annotation_fname in glob.glob(os.path.join('Data', 'helen_annotations', '*.txt')): with open(annotation_fname) as f: if f.readline()[:-1] == fname.split('.')[0]: ktps = [] for l in f: tmp = l.split() ktps.append([float(tmp[0]), float(tmp[2])]) break kpts = np.array(ktps)

    Specific Error:

    NameError Traceback (most recent call last) in 8 ktps.append([float(tmp[0]), float(tmp[2])]) 9 break ---> 10 kpts = np.array(ktps)

    NameError: name 'ktps' is not defined

    opened by tobimichigan 3
  • Use pytorch for 3D+ data

    Use pytorch for 3D+ data

    Proof of concept.

    • Switched to pytorch for 3D+ in Flip, Crop and Pad
    • Flip now accepts multiple axes, but not axis=-1. Also, the axis order has been switched from opencv to numpy
    • Minor cleanup to the tests
    opened by soupault 2
  • nd follow-up

    nd follow-up

    • Removed height and width concepts where possible
    • Switched from shape checking decorator to function
    • Added shape validation also for masks and keypoints
    • Splitted utils into checks and serial
    opened by soupault 2
  • Bump opencv-python-headless from 4.1.2.30 to 4.2.0.32 in /ci

    Bump opencv-python-headless from 4.1.2.30 to 4.2.0.32 in /ci

    Bumps opencv-python-headless from 4.1.2.30 to 4.2.0.32.

    Release notes

    Sourced from opencv-python-headless's releases.

    4.2.0.32

    OpenCV version 4.2.0.

    Changes:

    • macOS environment updated from xcode8.3 to xcode 9.4
    • macOS uses now Qt 5 instead of Qt 4
    • Nasm version updated to Docker containers
    • multibuild updated

    Fixes:

    • don't use deprecated brew tap-pin, instead refer to the full package name when installing #267
    • replace get_config_var() with get_config_vars() in setup.py #274
    • add workaround for DLL errors in Windows Server #264
    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Fix the documentation bug

    Fix the documentation bug

    https://github.com/MIPT-Oulu/solt/blob/485bfb0d471134f75e09d06aa5e9ce4b57c0e13c/solt/transforms/_transforms.py#L1018 tells something about noise. Should talk about the intensities instead.

    opened by lext 1
  • Compatibility issues with latest pytorch

    Compatibility issues with latest pytorch

    Works fine with 1.1, but not 1.5. Traceback:

        as_dict=as_dict, scale_keypoints=scale_keypoints, normalize=normalize, mean=mean, std=std,
      File ".../site-packages/solt/core/_data.py", line 273, in to_torch
        img.sub_(mean)
    RuntimeError: output with backend CPU and dtype Byte doesn't match the desired backend CPU and dtype Float
    Process finished with exit code 1
    
    opened by soupault 1
  • Can't run unit tests locally on MacOS

    Can't run unit tests locally on MacOS

    🐛 Bug

    It is not possible to run unit tests locally on MacOS X (10.12).

    To Reproduce

    Steps to reproduce the behavior:

    1. Create a conda environment and install all of the packages as suggested here.

    2. Run the following code: pytest tests

    3. The unit tests fail and generate an error:

    ImportError while importing test module '/Users/melekhi1/projects/solt/tests/test_data_core.py'.
    Hint: make sure your test modules/packages have valid Python names.
    Traceback:
    tests/test_data_core.py:4: in <module>
        import cv2
    ../../anaconda2/envs/solt_test_env/lib/python3.8/site-packages/cv2/__init__.py:3: in <module>
        from .cv2 import *
    E   ImportError: dlopen(/Users/melekhi1/anaconda2/envs/solt_test_env/lib/python3.8/site-packages/cv2/cv2.cpython-38-darwin.so, 2): Symbol not found: _inflateValidate
    E     Referenced from: /Users/melekhi1/anaconda2/envs/solt_test_env/lib/python3.8/site-packages/cv2/.dylibs/libpng16.16.dylib (which was built for Mac OS X 10.13)
    E     Expected in: /usr/lib/libz.1.dylib
    E    in /Users/melekhi1/anaconda2/envs/solt_test_env/lib/python3.8/site-packages/cv2/.dylibs/libpng16.16.dylib
    

    Expected behavior

    The unit tests should pass

    =============================== test session starts ================================
    platform darwin -- Python 3.8.2, pytest-3.6.4, py-1.8.1, pluggy-0.7.1
    rootdir: /Users/melekhi1/projects/solt, inifile:
    plugins: pep8-1.0.6, flake8-1.0.2, cov-2.6.0
    collected 701 items
    
    tests/test_base_transforms.py .............................................. [  6%]
    ............................................................................ [ 17%]
    .....                                                                        [ 18%]
    tests/test_data_core.py .................................................... [ 25%]
    ............................................................................ [ 36%]
    ............................................................................ [ 47%]
    ..................................................................           [ 56%]
    tests/test_transforms.py ................................................... [ 63%]
    ............................................................................ [ 74%]
    ............................................................................ [ 85%]
    ............................................................................ [ 96%]
    .........                                                                    [ 97%]
    tests/test_utils.py ................                                         [100%]
    
    ============================ 701 passed in 6.79 seconds ============================
    

    Suggestions

    It seems the bug is related to the latest version of opencv-python-headless (4.2.0.32) package. Apparently, its previous version, 4.1.2.30, could solve the issue at least for this particular platform (MacOS 10.12). It would be great to test this solution on more recent MacOS.

    opened by imelekhov 1
  • Incorrect serialization of a cropping transform

    Incorrect serialization of a cropping transform

    The attribute crop_size needs to be renamed into crop_to to match the constructor. Otherwise, the serialization does not work well.

    https://github.com/MIPT-Oulu/solt/blob/770e397884bcafe80a11723c229e275c1c1f8b5a/solt/transforms/_transforms.py#L715

    bug 
    opened by lext 1
  • Fair benchmark with the other libraries

    Fair benchmark with the other libraries

    the augbench package needs further development and its results need to be reported in the README.

    the benchmark needs to cover random transformations rather than the static ones. It is important to make comparison for three cases: image, image mask, and image+10 masks (instance segmentation task).

    opened by lext 1
  • Brightness needs to support percentages of the mean intensity

    Brightness needs to support percentages of the mean intensity

    The use case for this is that for some images it might be better to rely on their mean intensity and increase their intensity by n% of the mean value.

    enhancement 
    opened by lext 0
  • Allow to randomly select random number of augmentations in Selective stream

    Allow to randomly select random number of augmentations in Selective stream

    Currently, selective stream takes a parameter that tells how many transforms to select from the list of transformations. I was recently suggested to make it a range and allow to sample random number of augmentations out of this range.

    feature request 
    opened by lext 0
Releases(v0.1.9)
  • v0.1.9(Mar 10, 2020)

    • [x] SOLT is now PyTorch Native
    • [x] All the transform subtract the mean and std by default
    • [x] Implemented a proper serialization and deserialization
    • [x] Allowed to use dict instead of a data container
    • [x] Introduced shorter transform names
    • [x] Implemented a fair benchmark to compare with other libraries
    • [x] Fixed a bug of JPEGCompression (@tiulpin )
    • [x] Added IntensityRemappping transform (@soupault )

    This release is not backward comparible

    Source code(tar.gz)
    Source code(zip)
  • v0.1.8(Jul 26, 2019)

  • v0.1.7(Jul 6, 2019)

    This release has some tests-related features and also adds two new transforms.

    Detailed description:

    • [x] Improved and parametrized many more testes
    • [x] Added CutOut data augmentation
    • [x] Added KeypointsJitter class that allows to apply random displacements to keypoints
    Source code(tar.gz)
    Source code(zip)
  • v0.1.6(May 25, 2019)

  • v0.1.5(Feb 15, 2019)

  • v0.1(Nov 13, 2018)

  • v0.0.6(Sep 21, 2018)

Owner
Research Unit of Medical Imaging, Physics and Technology
Research Unit of Medical Imaging, Physics and Technology
[NeurIPS 2021] COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining

COCO-LM This repository contains the scripts for fine-tuning COCO-LM pretrained models on GLUE and SQuAD 2.0 benchmarks. Paper: COCO-LM: Correcting an

Microsoft 106 Dec 12, 2022
Code for Paper "Evidential Softmax for Sparse MultimodalDistributions in Deep Generative Models"

Evidential Softmax for Sparse Multimodal Distributions in Deep Generative Models Abstract Many applications of generative models rely on the marginali

Stanford Intelligent Systems Laboratory 9 Jun 06, 2022
Utilities and information for the signals.numer.ai tournament

dsignals Utilities and information for the signals.numer.ai tournament using eodhistoricaldata.com eodhistoricaldata.com provides excellent historical

Degerhan Usluel 23 Dec 18, 2022
A Dataset of Python Challenges for AI Research

Python Programming Puzzles (P3) This repo contains a dataset of python programming puzzles which can be used to teach and evaluate an AI's programming

Microsoft 850 Dec 24, 2022
Online-compatible Unsupervised Non-resonant Anomaly Detection Repository

Online-compatible Unsupervised Non-resonant Anomaly Detection Repository Repository containing all scripts used in the studies of Online-compatible Un

0 Nov 09, 2021
Model Quantization Benchmark

Introduction MQBench is an open-source model quantization toolkit based on PyTorch fx. The envision of MQBench is to provide: SOTA Algorithms. With MQ

500 Jan 06, 2023
Atif Hassan 103 Dec 14, 2022
(ImageNet pretrained models) The official pytorch implemention of the TPAMI paper "Res2Net: A New Multi-scale Backbone Architecture"

Res2Net The official pytorch implemention of the paper "Res2Net: A New Multi-scale Backbone Architecture" Our paper is accepted by IEEE Transactions o

Res2Net Applications 928 Dec 29, 2022
FLSim a flexible, standalone library written in PyTorch that simulates FL settings with a minimal, easy-to-use API

Federated Learning Simulator (FLSim) is a flexible, standalone core library that simulates FL settings with a minimal, easy-to-use API. FLSim is domain-agnostic and accommodates many use cases such a

Meta Research 162 Jan 02, 2023
Code for "Long-tailed Distribution Adaptation"

Long-tailed Distribution Adaptation (Accepted in ACM MM2021) This project is built upon BBN. Installation pip install -r requirements.txt Usage Traini

Zhiliang Peng 10 May 18, 2022
Entity-Based Knowledge Conflicts in Question Answering.

Entity-Based Knowledge Conflicts in Question Answering Run Instructions | Paper | Citation | License This repository provides the Substitution Framewo

Apple 35 Oct 19, 2022
Implementation of ResMLP, an all MLP solution to image classification, in Pytorch

ResMLP - Pytorch Implementation of ResMLP, an all MLP solution to image classification out of Facebook AI, in Pytorch Install $ pip install res-mlp-py

Phil Wang 178 Dec 02, 2022
A criticism of a recent paper on buggy image downsampling methods in popular image processing and deep learning libraries.

A criticism of a recent paper on buggy image downsampling methods in popular image processing and deep learning libraries.

70 Jul 12, 2022
The repository is for safe reinforcement learning baselines.

Safe-Reinforcement-Learning-Baseline The repository is for Safe Reinforcement Learning (RL) research, in which we investigate various safe RL baseline

172 Dec 19, 2022
This is a official repository of SimViT.

SimViT This is a official repository of SimViT. We will open our models and codes about object detection and semantic segmentation soon. Our code refe

ligang 57 Dec 15, 2022
This repository contains the implementation of the paper: "Towards Frequency-Based Explanation for Robust CNN"

RobustFreqCNN About This repository contains the implementation of the paper "Towards Frequency-Based Explanation for Robust CNN" arxiv. It primarly d

Sarosij Bose 2 Jan 23, 2022
pytorchのスライス代入操作をonnxに変換する際にScatterNDならないようにするサンプル

pytorch_remove_ScatterND pytorchのスライス代入操作をonnxに変換する際にScatterNDならないようにするサンプル。 スライスしたtensorにそのまま代入してしまうとScatterNDになるため、計算結果をcatで新しいtensorにする。 python ver

2 Dec 01, 2022
Github project for Attention-guided Temporal Coherent Video Object Matting.

Attention-guided Temporal Coherent Video Object Matting This is the Github project for our paper Attention-guided Temporal Coherent Video Object Matti

71 Dec 19, 2022
VITS: Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech

VITS: Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech Jaehyeon Kim, Jungil Kong, and Juhee Son In our rece

Jaehyeon Kim 1.7k Jan 08, 2023
A Pytorch Implementation of Source Data-free Domain Adaptation for a Faster R-CNN

A Pytorch Implementation of Source Data-free Domain Adaptation for a Faster R-CNN Please follow Faster R-CNN and DAF to complete the environment confi

2 Jan 12, 2022