Connecting Java/ImgLib2 + Python/NumPy

Related tags

Deep Learningimglyb
Overview

build status

imglyb

imglyb aims at connecting two worlds that have been seperated for too long:

imglyb uses jpype to access numpy arrays and expose them to ImgLib2 through imglib2-imglyb. This means shared memory between numpy and ImgLib2, i.e. any ImgLib2 algorithm can run on numpy arrays without creating copies of the data! For example, Python users can now make use of the BigDataViewer extension to visualize dense volumetric data.

If you are interested in using imglyb, have a look at the examples folder and extend the examples as needed!

Note: NEP 18 has the potential to improve numpy - imglib interoperability, especially when converting imglib2 data structures to numpy.

Installation

Prerequisites

imglyb has been tested on Linux, macOS, and Windows.

The following tools are required:

  • Python 3
  • Java 8 or 11 JDK (JRE is not enough)
  • Apache Maven

If you use conda, these will be installed for you.

Installing with conda

conda install -c conda-forge imglyb

Installing with pip

First, install the prerequisites above. Then run:

pip install imglyb

It is recommended to do this from inside a virtualenv or conda environment, rather than system-wide.

Installing from source

First, install the prerequisites above. Then run:

git clone git://github.com/imglib/imglyb
cd imglyb
pip install -e .

It is recommended to do this from inside a virtualenv or conda environment, rather than system-wide.

Usage

It is suggested to follow and extend the examples in the examples folder according to your needs.

Or, for a higher-level way to use imglyb, check out pyimagej.

Known Issues

AWT on macOS

AWT and Cocoa do not get along perfectly. In general, the Cocoa event loop needs to be started before the JVM is loaded. (Thanks to @tpietzsch for figuring this out!) This requires some macOS specific code, written using PyObjC, to properly start up and shut down the Cocoa application and start the Java/Python code within it.

The OSXAWTwrapper.py script included in the imglyb library provides an example of Cocoa code and can be used to run the imglyb examples. Two packages from PyObjC are required for this wrapper (pyobjc-core and pyobjc-framework-cocoa), and they should be installed with imglyb on macOS.

When running the wrapper, one can either provide the name of the target module (as if using python -m) or the full path to the target script. So using the module name, the command to run the "butterfly" script in imglyb-examples looks like this:

python imglyb/OSXAWTwrapper.py imglyb-examples.butterfly

Running OSXAWTwrapper.py via python -m does not work at this time.

Comments
  • OSXAWTwrapper.main() unused app variable

    OSXAWTwrapper.main() unused app variable

    This line is not PEP8-compliant, as app is an unused variable. But, looking at the documentation, I am hesitant to delete it as it creates an application instance if it does not exist. Can we just delete this? (cc @ctrueden, the author of said line).

    Unfortunately, this code is not tested :frowning:

    opened by gselzer 4
  • Comparison with pyimagej

    Comparison with pyimagej

    Hi,

    I write very frequently scripts for Imagej in python using the Jython interpreter but starting to consider using directly python. For that I am considering these options:

    • imglyb (your library)
    • pyimagej [https://pypi.org/project/pyimagej/]
    • https://nbviewer.jupyter.org/github/imagej/tutorials/blob/master/notebooks/ImageJ-Tutorials-and-Demo.ipynb

    From the pure scripting approach, is there any significant differences among those?

    opened by phisanti 4
  • Use scyjava converters for zero-copy wrapping of array-backed imgs to python

    Use scyjava converters for zero-copy wrapping of array-backed imgs to python

    The built-in scyjava array converters will eventually be written to use python memoryview, allowing for zero-copy wrapping of primitive Java arrays. It seems sensible that we would use this logic in imglyb to convert compatible Imgs in the Java -> python direction.

    opened by hinerm 3
  • Package cleanup

    Package cleanup

    Goals of this PR:

    • [x] Refactor imglyb/ directory into src/imglyb/ directory. See this article for the benefits
    • [x] Minimize setup.py in favor of setup.cfg. This seems to be the preferred approach nowadays.
    • [x] Add a linting strategy (using black)
    • [x] Add a code coverage strategy
    • [x] Allow these features to block merge through Github Actions

    Closes #15

    opened by gselzer 3
  • Do not flip dimension order

    Do not flip dimension order

    When wrapping a numpy array to an ImgLib2 image, the dimension order is reversed. For example, a 3D numpy array dimensioned [3, 5, 7] would become a 3D RandomAccessibleInterval dimensioned [7, 5, 3]. This PR attempts to fix that inconsistency such that dimensions stay consistent across the two worlds.

    A numpy array has two different orders:

    C is column-contiguous order in memory (last index varies the fastest). C order means that operating row-rise on the array will be slightly quicker. FORTRAN-contiguous order in memory (first index varies the fastest). F order means that column-wise operations will be faster.

    ImgLib2 generally prefers F order. In particular, the Views.flatIterable method returns an IterableInterval in F order, and the IntervalIndexer methods perform F-ordered rasterization.

    On the Python side: if you do not specify an order, then numpy arrays default to C order.

    @hanslovsky Is this discrepancy between ImgLib2's general preference and NumPy's default preference the reason for inverting the axes? Or is there another reason?

    Since NumPy supports both C and F, my strong vote is to discontinue this dimension flipping behavior, in favor of consistently keeping everything the same.

    @hanslovsky If you agree, I can polish up this PR. Certainly, I would add unit tests to prove that all works as expected in the various cases.

    Alternately, if breaking existing behavior makes you nervous, we could add a flip_dimensions flag to the to_imglib method, defaulting to True. I would still change pyimagej to pass False for this flag though, because I think it's good to reduce the number of things our less technical users need to know about and work around.

    opened by ctrueden 2
  • TypeError: Invalid type conversion to ArrayImg [32x32x32] requested.

    TypeError: Invalid type conversion to ArrayImg [32x32x32] requested.

    Hi,

    It appears I am not able to use the imglyb.to_numpy(rai) method

    import imglyb
    import scyjava
    
    scyjava.start_jvm()
    
    Views = imglyb.util.Views
    
    ArrayImgs = scyjava.jimport("net.imglib2.img.array.ArrayImgs")
    
    img = ArrayImgs.floats(32,32,32)
    print(img.dimensionsAsLongArray())
    
    arr = imglyb.to_numpy(img)
    

    yields

    [32, 32, 32]
    Traceback (most recent call last):
      File "C:\Users\cameron.arshadi\Desktop\repos\imglyb\examples\wrap_arraylike.py", line 13, in <module>
        arr = imglyb.to_numpy(img)
              ^^^^^^^^^^^^^^^^^^^^
      File "C:\Users\cameron.arshadi\Miniconda3\envs\imglyb-test\Lib\site-packages\imglyb\__init__.py", line 25, in to_numpy
        return _ImgLibReferenceGuard(source)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "C:\Users\cameron.arshadi\Miniconda3\envs\imglyb-test\Lib\site-packages\imglyb\imglib_ndarray.py", line 67, in __new__
        address = get_address(rai)
                  ^^^^^^^^^^^^^^^^
      File "C:\Users\cameron.arshadi\Miniconda3\envs\imglyb-test\Lib\site-packages\imglyb\imglib_ndarray.py", line 50, in get_address
        access = jpype.JObject(class_name_full, rai).update(None)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "C:\Users\cameron.arshadi\Miniconda3\envs\imglyb-test\Lib\site-packages\jpype\_jobject.py", line 59, in __new__
        return _JObjectFactory(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "C:\Users\cameron.arshadi\Miniconda3\envs\imglyb-test\Lib\site-packages\jpype\_jobject.py", line 113, in _JObjectFactory
        raise TypeError("Invalid type conversion to %s requested." % tp)
    TypeError: Invalid type conversion to ArrayImg [32x32x32] requested.
    

    I used a fresh conda environment on WIndows 10

    conda create -n imglyb-test -c conda-forge python=3.9 imglyb
    conda activate imglyb-test
    

    Downgrading jpype to 1.4.0 and 1.3.0 did not help

    Output of conda list

    # packages in environment at C:\Users\cameron.arshadi\Miniconda3\envs\imglyb-test:
    #
    # Name                    Version                   Build  Channel
    bzip2                     1.0.8                h8ffe710_4    conda-forge
    ca-certificates           2022.12.7            h5b45459_0    conda-forge
    imglyb                    2.0.1              pyh8a188c0_0    conda-forge
    intel-openmp              2022.2.1         h57928b3_19741    conda-forge
    jgo                       1.0.4              pyhd8ed1ab_0    conda-forge
    jpype1                    1.3.0            py39h2e07f2f_2    conda-forge
    libblas                   3.9.0              16_win64_mkl    conda-forge
    libcblas                  3.9.0              16_win64_mkl    conda-forge
    libffi                    3.4.2                h8ffe710_5    conda-forge
    libhwloc                  2.8.0                h039e092_1    conda-forge
    libiconv                  1.17                 h8ffe710_0    conda-forge
    liblapack                 3.9.0              16_win64_mkl    conda-forge
    libsqlite                 3.40.0               hcfcfb64_0    conda-forge
    libxml2                   2.10.3               hc3477c8_0    conda-forge
    libzlib                   1.2.13               hcfcfb64_4    conda-forge
    maven                     3.8.6                h57928b3_0    conda-forge
    mkl                       2022.1.0           h6a75c08_874    conda-forge
    numpy                     1.23.5           py39hbccbffa_0    conda-forge
    openjdk                   17.0.3               h57928b3_4    conda-forge
    openssl                   3.0.7                hcfcfb64_1    conda-forge
    packaging                 22.0               pyhd8ed1ab_0    conda-forge
    pip                       22.3.1             pyhd8ed1ab_0    conda-forge
    psutil                    5.9.4            py39ha55989b_0    conda-forge
    pthreads-win32            2.9.1                hfa6e2cd_3    conda-forge
    python                    3.9.15          h4de0772_0_cpython    conda-forge
    python_abi                3.9                      3_cp39    conda-forge
    scyjava                   1.8.1              pyhd8ed1ab_0    conda-forge
    setuptools                65.5.1             pyhd8ed1ab_0    conda-forge
    symlink-exe-runtime       1.0                  hcfcfb64_0    conda-forge
    tbb                       2021.7.0             h91493d7_1    conda-forge
    tk                        8.6.12               h8ffe710_0    conda-forge
    tzdata                    2022g                h191b570_0    conda-forge
    ucrt                      10.0.22621.0         h57928b3_0    conda-forge
    vc                        14.3                 h3d8a991_9    conda-forge
    vs2015_runtime            14.32.31332          h1d6e394_9    conda-forge
    wheel                     0.38.4             pyhd8ed1ab_0    conda-forge
    xz                        5.2.6                h8d14728_0    conda-forge
    
    opened by carshadi 1
  • Move vistools into separate package

    Move vistools into separate package

    This could, possibly, be a separate package, e.g. imglyb_bdv or imglyb_vistools. It is not always necessary (or desired) to have the BDV dependencies on the classpath with imglyb.

    Alternatively, this could be controlled through imglyb_config.

    cc @ctrueden

    opened by hanslovsky 1
  • Set requirment jnius -> pyjnius

    Set requirment jnius -> pyjnius

    The setup.py file lists jnius as a requirement. Jnius has been renamed to pyjnius, and so this can send conda installation warnings when installing imglyb, which can be confusing for new users.

    This change just renames the requirement to pyjnius to reduce confusion.

    opened by mpinkert 1
  • Align Package Structure with PyPA Recommendations

    Align Package Structure with PyPA Recommendations

    While there is generally no consensus on how python projects should be packaged, there seem to be some good benefits to the src package structure and the Python Packaging Authority now promotes this structure.

    https://github.com/pypa/packaging.python.org/issues/320 describes how they came to the determination they did.

    We should consider adding it to imglyb.

    opened by gselzer 0
  • Replace os.getenv('HOME') with os.path.expanduser('~')

    Replace os.getenv('HOME') with os.path.expanduser('~')

    HOME env not defined on windows

    See also: https://forum.image.sc/t/analysis-with-imagej-and-visualization-in-the-jupyter-notebook/11052/27?u=hanslovsky

    opened by hanslovsky 0
  • imglyb with dask not working

    imglyb with dask not working

    I added support for converting dask arrays to imglib2 CachedCellImg but it does not seem to work correctly: imglyb-bdv-dask

    I spent a lot of time trying to fix that and my personal interest in dask is very low, so I will not continue working on this. Contributions always welcome.

    Relevant code: https://github.com/imglib/imglyb/blob/a47f6793e3b269f48c3ee397ed542388b9e1cbda/test.py https://github.com/imglib/imglyb/blob/a47f6793e3b269f48c3ee397ed542388b9e1cbda/imglyb/cell.py https://github.com/imglib/imglib2-imglyb/blob/401951b23146fee0e1bc4dd4a4f9b9302df7b8d4/src/main/java/net/imglib2/python/Helpers.java#L111-L146

    wontfix 
    opened by hanslovsky 0
  • Use jpype.nio.convertToDirectBuffer to wrap Python memory to Java, instead of Unsafe

    Use jpype.nio.convertToDirectBuffer to wrap Python memory to Java, instead of Unsafe

    Using JNI's NewDirectByteBuffer function, one can wrap an existing memory address and length into an off-heap ByteBuffer. Now that ImgLib2 supports ByteBuffer-backed data, we could make use of this to have zero-copy access to NumPy arrays from Java without the Unsafe hacks currently used. This would hopefully make the imglib2-unsafe library obsolete.

    ~~We need to determine the best way to call that JNI function. It is used in the JPype project in a few places; perhaps there is an existing JPype call we can use, without needing to resort to our own Cython code or use of JNA...~~ Edit: It should be as simple as calling jpype.nio.convertToDirectBuffer(narr) on the numpy array and feeding the resultant DirectByteBuffer to ImgLib2. After verifying this indeed does not copy the data, of course.

    enhancement 
    opened by ctrueden 9
  • Support numpy.bool_ dtype -> NativeBoolType

    Support numpy.bool_ dtype -> NativeBoolType

    This PR supports the conversion from numpy arrays with dtype np.bool8 to RandomAccessibleInterval<NativeBoolType>s, and adds a regression test to ensure this conversion is possible. (It actually adds more regression tests than that).

    This PR is the minimal possible change required to implement this functionality.

    Conversion the other way is tricky, as we have discussed in #13, and the PRs that this work leans on are not enough to implement that conversion. The main issue lies in creating an Image of some BooleanType that can be converted. I am unable to create an UnsafeImg of any BooleanType, and although I can create an ArrayImg<BitType>, I cannot convert that using imglyb.to_numpy. Therefore, I leave that work for another PR.

    This PR requires the work of imglib/imglib2-unsafe#8 and imglib/imglib2-imglyb#10 to make their way into releases before this will work. I have tested locally with snapshots of each.

    enhancement 
    opened by gselzer 0
  • Unable to convert ImgLib2 BooleanType image to numpy/xarray bool

    Unable to convert ImgLib2 BooleanType image to numpy/xarray bool

    imglyb does not support converting bool type images to numpy. This means that users are unable to convert masks/thresholded images into python numpy/xarray objects.

    Here is a minimal example:

    import imagej
    import scyjava as sj
    
    # initialize
    ij = imagej.init()
    
    # load blobs
    img = ij.io().open('blobs.tif')
    
    # threshold image w/ ops
    CreateNamespace = sj.jimport('net.imagej.ops.create.CreateNamespace')
    
    img_invert = ij.op().namespace(CreateNamespace).img(img)
    ij.op().image().invert(img_invert, img)
    threshold = ij.op().threshold().otsu(img_invert)
    
    # get threshold from java
    py_threshold = ij.py.from_java(threshold)
    

    Returns the traceback:

    File "/home/edward/Documents/repos/loci/imglyb/imglyb/util.py", line 149, in _to_imglib
        raise NotImplementedError("Cannot convert dtype to ImgLib2 type yet: {}".format(source.dtype))
    NotImplementedError: Cannot convert dtype to ImgLib2 type yet: bool
    
    opened by elevans 5
  • Use reference store for imglyb.to_imglib

    Use reference store for imglyb.to_imglib

    Similar to what is done with wrapping as cached cell images (#7).

    I am not 100% sure about this one. The user still needs to make sure that the reference store stays within scope as long as necessary. Instead, callers could do the same thing for the actual ndarray, and a reference store may be obsolete here.

    question 
    opened by hanslovsky 0
Releases(2.0.0)
Owner
ImgLib2
Open-source n-dimensional image data representation and manipulation
ImgLib2
Repository for reproducing `Model-Based Robust Deep Learning`

Model-Based Robust Deep Learning (MBRDL) In this repository, we include the code necessary for reproducing the code used in Model-Based Robust Deep Le

Alex Robey 16 Sep 19, 2022
This repository contains the code for the paper in EMNLP 2021: "HRKD: Hierarchical Relational Knowledge Distillation for Cross-domain Language Model Compression".

HRKD: Hierarchical Relational Knowledge Distillation for Cross-domain Language Model Compression This repository contains the code for the paper in EM

Chenhe Dong 2 Mar 24, 2022
Sequence modeling benchmarks and temporal convolutional networks

Sequence Modeling Benchmarks and Temporal Convolutional Networks (TCN) This repository contains the experiments done in the work An Empirical Evaluati

CMU Locus Lab 3.5k Jan 01, 2023
A CNN implementation using only numpy. Supports multidimensional images, stride, etc.

A CNN implementation using only numpy. Supports multidimensional images, stride, etc. Speed up due to heavy use of slicing and mathematical simplification..

2 Nov 30, 2021
audioLIME: Listenable Explanations Using Source Separation

audioLIME This repository contains the Python package audioLIME, a tool for creating listenable explanations for machine learning models in music info

Institute of Computational Perception 27 Dec 01, 2022
tf2onnx - Convert TensorFlow, Keras and Tflite models to ONNX.

tf2onnx converts TensorFlow (tf-1.x or tf-2.x), tf.keras and tflite models to ONNX via command line or python api.

Open Neural Network Exchange 1.8k Jan 08, 2023
A PyTorch Implementation of ViT (Vision Transformer)

ViT - Vision Transformer This is an implementation of ViT - Vision Transformer by Google Research Team through the paper "An Image is Worth 16x16 Word

Quan Nguyen 7 May 11, 2022
Library for machine learning stacking generalization.

stacked_generalization Implemented machine learning *stacking technic[1]* as handy library in Python. Feature weighted linear stacking is also availab

114 Jul 19, 2022
Realtime Face Anti Spoofing with Face Detector based on Deep Learning using Tensorflow/Keras and OpenCV

Realtime Face Anti-Spoofing Detection ๐Ÿค– Realtime Face Anti Spoofing Detection with Face Detector to detect real and fake faces Please star this repo

Prem Kumar 86 Aug 03, 2022
Implementation of Bottleneck Transformer in Pytorch

Bottleneck Transformer - Pytorch Implementation of Bottleneck Transformer, SotA visual recognition model with convolution + attention that outperforms

Phil Wang 621 Jan 06, 2023
A PyTorch implementation of "CoAtNet: Marrying Convolution and Attention for All Data Sizes".

CoAtNet Overview This is a PyTorch implementation of CoAtNet specified in "CoAtNet: Marrying Convolution and Attention for All Data Sizes", arXiv 2021

Justin Wu 268 Jan 07, 2023
Predicting Tweet Sentiment Maching Learning and streamlit

Predicting-Tweet-Sentiment-Maching-Learning-and-streamlit (I prefere using Visual Studio Code ) Open the folder in VS Code Run the first cell in requi

1 Nov 20, 2021
Retrieval.pytorch - The code we used in [2020 DIGIX]

Retrieval.pytorch - The code we used in [2020 DIGIX]

Guo-Hua Wang 2 Feb 07, 2022
PyTorch Implementation of Region Similarity Representation Learning (ReSim)

ReSim This repository provides the PyTorch implementation of Region Similarity Representation Learning (ReSim) described in this paper: @Article{xiao2

Tete Xiao 74 Jan 03, 2023
Official implementation of deep-multi-trajectory-based single object tracking (IEEE T-CSVT 2021).

DeepMTA_PyTorch Officical PyTorch Implementation of "Dynamic Attention-guided Multi-TrajectoryAnalysis for Single Object Tracking", Xiao Wang, Zhe Che

Xiao Wang๏ผˆ็Ž‹้€๏ผ‰ 7 Dec 03, 2022
project page for VinVL

VinVL: Revisiting Visual Representations in Vision-Language Models Updates 02/28/2021: Project page built. Introduction This repository is the project

308 Jan 09, 2023
Neural Point-Based Graphics

Neural Point-Based Graphics Project โ€‚ Video โ€‚ Paper Neural Point-Based Graphics Kara-Ali Aliev1 Artem Sevastopolsky1,2 Maria Kolos1,2 Dmitry Ulyanov3

Ali Aliev 252 Dec 13, 2022
code release for USENIX'22 paper `On the Security Risks of AutoML`

This project is a minimized runnable project cut from trojanzoo, which contains more datasets, models, attacks and defenses. This repo will not be mai

Ren Pang 5 Apr 19, 2022
The project of phase's key role in complex and real NN

Phase-in-NN This is the code for our project at Princeton (co-authors: Yuqi Nie, Hui Yuan). The paper title is: "Neural Network is heterogeneous: Phas

YuqiNie-lab 1 Nov 04, 2021
Weighing Counts: Sequential Crowd Counting by Reinforcement Learning

LibraNet This repository includes the official implementation of LibraNet for crowd counting, presented in our paper: Weighing Counts: Sequential Crow

Hao Lu 18 Nov 05, 2022