Description

We were running into a bug for large cubes in Jdaviz, where we couldn't retrieve the collapsed spectrum if the cube was above a certain size (above n_chunk_max in compute_statistic as it turned out). I'm not sure if this is the best fix, but it works to avoid the broadcast error we were encountering on line 1689 where there was an array size mismatch with a RangeSubsetState defined and number of array elements > n_chunk_max.

In the same way that we can have plug-in exporters, it would be nice to have plug-in imports that can bring up different or additional dialogs. Examples include:

• An additional dialog for e.g. FITS files which can allow one to select the HDU
• A dialog instead of the 'open file' which could get data from online sources (e.g. dataverse, Vizier, etc)

This would open up all kinds of new fun ways to bring data into glue. All these could be developed as plugins of course, and wouldn't clutter up the main code base.

With these changes, Glue will at least install and launch, but when trying to select a subset I still get the error:

Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/matplotlib/backends/backend_qt4.py", line 261, in mouseReleaseEvent
    FigureCanvasBase.button_release_event( self, x, y, button )
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/matplotlib/backend_bases.py", line 1808, in button_release_event
    self.callbacks.process(s, event)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/matplotlib/cbook.py", line 528, in process
    proxy(*args, **kwargs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/matplotlib/cbook.py", line 406, in __call__
    return mtd(*args, **kwargs)
  File "/Volumes/Raptor/Library/Python/3.3/lib/python/site-packages/Glue-0.1.0-py3.3.egg/glue/qt/mouse_mode.py", line 195, in release
    self._finish_roi(event)
  File "/Volumes/Raptor/Library/Python/3.3/lib/python/site-packages/Glue-0.1.0-py3.3.egg/glue/qt/mouse_mode.py", line 156, in _finish_roi
    self._roi_callback(self)
  File "/Volumes/Raptor/Library/Python/3.3/lib/python/site-packages/Glue-0.1.0-py3.3.egg/glue/qt/widgets/scatter_widget.py", line 99, in apply_mode
    return self.apply_roi(mode.roi())
  File "/Volumes/Raptor/Library/Python/3.3/lib/python/site-packages/Glue-0.1.0-py3.3.egg/glue/qt/decorators.py", line 18, in result
    return func(*args, **kwargs)
  File "/Volumes/Raptor/Library/Python/3.3/lib/python/site-packages/Glue-0.1.0-py3.3.egg/glue/core/application_base.py", line 223, in apply_roi
    self._session.command_stack.do(cmd)
  File "/Volumes/Raptor/Library/Python/3.3/lib/python/site-packages/Glue-0.1.0-py3.3.egg/glue/core/command.py", line 98, in do
    result = cmd.do(self._session)
  File "/Volumes/Raptor/Library/Python/3.3/lib/python/site-packages/Glue-0.1.0-py3.3.egg/glue/core/command.py", line 224, in do
    self.client.apply_roi(self.roi)
  File "/Volumes/Raptor/Library/Python/3.3/lib/python/site-packages/Glue-0.1.0-py3.3.egg/glue/clients/scatter_client.py", line 255, in apply_roi
    mode.update(self._data, subset_state, focus_data=focus)
  File "/Volumes/Raptor/Library/Python/3.3/lib/python/site-packages/Glue-0.1.0-py3.3.egg/glue/core/edit_subset_mode.py", line 67, in update
    self._combine_data(data, new_state, add_if_empty=doadd)
  File "/Volumes/Raptor/Library/Python/3.3/lib/python/site-packages/Glue-0.1.0-py3.3.egg/glue/core/edit_subset_mode.py", line 35, in _combine_data
    data.edit_subset = data.new_subset()
  File "/Volumes/Raptor/Library/Python/3.3/lib/python/site-packages/Glue-0.1.0-py3.3.egg/glue/core/data.py", line 610, in new_subset
    new_subset = Subset(self, color=color, label=label, **kwargs)
  File "/Volumes/Raptor/Library/Python/3.3/lib/python/site-packages/Glue-0.1.0-py3.3.egg/glue/core/subset.py", line 54, in __init__
    self.label = label  # trigger disambiguation
  File "/Volumes/Raptor/Library/Python/3.3/lib/python/site-packages/Glue-0.1.0-py3.3.egg/glue/core/subset.py", line 228, in __setattr__
    object.__setattr__(self, attribute, value)
  File "/Volumes/Raptor/Library/Python/3.3/lib/python/site-packages/Glue-0.1.0-py3.3.egg/glue/core/subset.py", line 92, in label
    value = Registry().register(self, value, group=self.data)
  File "/Volumes/Raptor/Library/Python/3.3/lib/python/site-packages/Glue-0.1.0-py3.3.egg/glue/core/registry.py", line 69, in register
    has_obj = obj in reg
TypeError: unhashable type: 'Subset'

I don't quite understand yet why this would work in Python 2 but not Python 3.

In part thinking about issue #642 and in dealing with datasets with many FITS extensions, came up with a generic FITS loader.

Haven't done tests/docs yet, but wanted to throw this out for thoughts and possibly becoming the default for FITS files.

Tests run fine using the command

    py.test glue

However, if trying with either

    ./runtests.py
    python setup.py test

one will be greeted with a stack trace similar to the below.

    (glue)[user glue-repo]$ python setup.py test
    running test
    INTERNALERROR> Traceback (most recent call last):
    INTERNALERROR>   File "_pytest.main", line 68, in wrap_session
    INTERNALERROR>     config.pluginmanager.do_configure(config)
    INTERNALERROR>   File "_pytest.core", line 267, in do_configure
    INTERNALERROR>     config.hook.pytest_configure(config=self._config)
    INTERNALERROR>   File "_pytest.core", line 419, in __call__
    INTERNALERROR>     return self._docall(methods, kwargs)
    INTERNALERROR>   File "_pytest.core", line 430, in _docall
    INTERNALERROR>     res = mc.execute()
    INTERNALERROR>   File "_pytest.core", line 348, in execute
    INTERNALERROR>     res = method(**kwargs)
    INTERNALERROR>   File "/Users/eisenham/Documents/ssbdev/play/glue/glue-repo/glue/conftest.py", line 7, in pytest_configure
    INTERNALERROR>     if config.getoption('no_optional_skip'):
    INTERNALERROR> AttributeError: 'Config' object has no attribute 'getoption'

At the moment, code for e.g. the histogram stuff is split into the client (in glue/clients) and the widget (in glue/qt/widgets). Of course, this separation makes sense in case we want to use different GUIs in future, but what I think might be easier to manage is something like this:

glue/
    viewers/
        histogram/
            client.py
            qt.py
            histogram.ui
        image/
            ...

i.e. to group the different viewers/widgets into individual folders and then inside that have a separation of the general glue client code, the qt code, and we can even add other front-ends in future, e.g. glue/viewers/histogram/web.py.

In my opinion, this would be a step towards more modularity, but there may be reasons why we might not want to do that. @ChrisBeaumont, what do you think?

If we agree something like this would be good, I can go ahead and open a PR with the changes.

This attempts to speed up CompositeArray by using several tricks:

• When using colormaps, we don't need to care about layers below the top-most layer that has alpha=1 since it will block everything below - this should speed up viewers with multiple images open.
• When using colormaps and assuming the colormaps don't have any transparency, we can avoid a bunch of temporary array creations.
• We make use of out= in the contrast/bias and stretch to avoid more temporary array creation.

I tried the installation command suggested in the documentation: pythonw -m pip install glueviz but the command pythonw is not available (at least directly) in my version of Anaconda in Linux (Debian Jessie). When I try directly with pip install glueviz I got the following error:

(...)
copying glue/qt/ui/subset_facet.ui -> build/lib/glue/qt/ui

running install_lib

Traceback (most recent call last):

  File "<string>", line 1, in <module>

  File "/tmp/pip-build-mp42hW/glueviz/setup.py", line 67, in <module>

    package_data={'': ['*.png', '*.ui']}

  File "/home/jsm/programs/soft/anaconda/lib/python2.7/distutils/core.py", line 151, in setup

    dist.run_commands()

  File "/home/jsm/programs/soft/anaconda/lib/python2.7/distutils/dist.py", line 953, in run_commands

    self.run_command(cmd)

  File "/home/jsm/programs/soft/anaconda/lib/python2.7/distutils/dist.py", line 972, in run_command

    cmd_obj.run()

  File "/usr/lib/python2.7/dist-packages/setuptools/command/install.py", line 61, in run

    return orig.install.run(self)

  File "/home/jsm/programs/soft/anaconda/lib/python2.7/distutils/command/install.py", line 575, in run

    self.run_command(cmd_name)

  File "/home/jsm/programs/soft/anaconda/lib/python2.7/distutils/cmd.py", line 326, in run_command

    self.distribution.run_command(command)

  File "/home/jsm/programs/soft/anaconda/lib/python2.7/distutils/dist.py", line 971, in run_command

    cmd_obj.ensure_finalized()

  File "/home/jsm/programs/soft/anaconda/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized

    self.finalize_options()

  File "/usr/lib/python2.7/dist-packages/setuptools/command/install_lib.py", line 16, in finalize_options

    self.set_undefined_options('install',('install_layout','install_layout'))

  File "/home/jsm/programs/soft/anaconda/lib/python2.7/distutils/cmd.py", line 302, in set_undefined_options

    getattr(src_cmd_obj, src_option))

  File "/home/jsm/programs/soft/anaconda/lib/python2.7/distutils/cmd.py", line 105, in __getattr__

    raise AttributeError, attr

AttributeError: install_layout

----------------------------------------
Cleaning up...

Any idea of the origin of the error?

This is a work in progress, and I am aiming to try and get the test suite to pass in Python 3.

Related issues:

• https://github.com/scikit-image/scikit-image/issues/1187
• https://github.com/numpy/numpy/issues/5145
• https://github.com/astropy/astropy/issues/3013

(jeez, this is turning out to be quite the bug hunt)

As part of the effort @aak65 is doing implementing some Bio-glue features he's going to need a more robust ability to pass around ROIs and subsets that are based on categorical data in a way that's more robust then my approach before ... which was basically convert everything to numbers and hope it all worked out.

This has all of the backings for the new ROI/Subset but I can't quite figure out where to fold it into the Historgram & Scatter viewers. My ideal situation would be after the user uses a Range-selector on an axis defined by a categorical component it would adjust the returned ROI accordingly. Any ideas on where I would find that logic? @ChrisBeaumont or @astrofrog

@ChrisBeaumont - I think it would be good to have a new release since there are already a number of improvements since 0.4 (in particular full Python 3 support). Would you object to having a 0.5 release soon? I'd be happy to help with that - if you would like me to do it, just let me know if there is a specific workflow you have been following to carry out a release.

By the way, looks like glueviz is now in the main conda package list now :)

Is your feature request related to a problem? Please describe it: I have a repetitive check that I need to perform on lots of different datasets which means I'm always repeating tasks to produce the same plots. The datasets have the same columns but obviously different data.

Describe the solution you'd like: I'd like to be able to store the session parameters used to generate these plots so that I can apply them to a new dataset.

Describe alternatives you've considered: The only alternative so far is re-doing all the plots for each new imported dataset.

If it is not related to a problem--please describe the feature: Time saving

Additional context: Add any other context or screenshots about the feature request here.

I installed via pip install glueviz pip install glue-wwt pip install PyQt5

I tried conda install first but it always gets stuck at solving environment, so I had to use pip. When I try doing 3D scatter, the plot cannot show up properly, but instead shows something like a mirror of the entire glue window:

By default the vispy version is 0.11.0, and the error messages are WARNING: Could not create NSOpenGLContext with shared context, falling back to unshared context. WARNING:vispy:Could not create NSOpenGLContext with shared context, falling back to unshared context. WARNING: Error drawing visual <glue_vispy_viewers.compat.text.TextVisual object at 0x18d234e50> WARNING:vispy:Error drawing visual <glue_vispy_viewers.compat.text.TextVisual object at 0x18d234e50> WARNING: Traceback (most recent call last): ...... RuntimeError: Shader compilation error in GL_FRAGMENT_SHADER: WARNING: 0:3: extension 'GL_OES_element_index_uint' is not supported WARNING: 0:4: extension 'GL_OES_standard_derivatives' is not supported on line 1482: Invalid call of undeclared identifier 'CatRom' rgb = CatRom(u_font_atlas, u_font_atlas_shape, uv); WARNING:vispy:Traceback (most recent call last): ...... ERROR: Invoking <bound method SceneCanvas.on_draw of <SceneCanvas (PyQt5) at 0x7faa83371d00>> for DrawEvent ERROR:vispy:Invoking <bound method SceneCanvas.on_draw of <SceneCanvas (PyQt5) at 0x7faa83371d00>> for DrawEvent WARNING: Error drawing visual <glue_vispy_viewers.compat.text.TextVisual object at 0x7faa833a6100> WARNING:vispy:Error drawing visual <glue_vispy_viewers.compat.text.TextVisual object at 0x7faa833a6100> ERROR: Invoking <bound method SceneCanvas.on_draw of <SceneCanvas (PyQt5) at 0x7faa83371d00>> repeat 2 ERROR:vispy:Invoking <bound method SceneCanvas.on_draw of <SceneCanvas (PyQt5) at 0x7faa83371d00>> repeat 2 WARNING: Error drawing visual <glue_vispy_viewers.compat.text.TextVisual object at 0x7faa833a6100> WARNING:vispy:Error drawing visual <glue_vispy_viewers.compat.text.TextVisual object at 0x7faa833a6100> WARNING: Error drawing visual <glue_vispy_viewers.compat.text.TextVisual object at 0x7faa833a6100> WARNING:vispy:Error drawing visual <glue_vispy_viewers.compat.text.TextVisual object at 0x7faa833a6100> ERROR: Invoking <bound method SceneCanvas.on_draw of <SceneCanvas (PyQt5) at 0x7faa83371d00>> repeat 4 ERROR:vispy:Invoking <bound method SceneCanvas.on_draw of <SceneCanvas (PyQt5) at 0x7faa83371d00>> repeat 4

I tried installing vispy 0.9.6 and 0.10.0 instead. Then there's no error message anymore, just warnings, but the 3D plots are still not able to show.

My python version is 3.10.6 (pip installed everything in /usr/local/lib/python3.10/site-packages), and my system is MacOS Monterey 12.6.

It would be great if in dark mode the "plot data here" background were black. I always make fake windows to block out that bright light, but inevitably am blinded by it when a window moves.

Describe the bug Pandas DataFrames created within glue and added to the data_collection manager may have columns of type 'object', which mean they cannot be save/restored by glue (glue.core.state._load_numpy calls np.load()without allow_pickle=True). This is generally not a problem when reading files using the Pandas data_factory (which converts columns), but does, for instance cause problems for datasets retrieved from external sources within a glue session.

To Reproduce Steps to reproduce the behavior such as:

1. Create a Pandas DataFrame within glue and add it to the data_collection. For instance, one might use the process described in the documentation

df1 = DataFrame()
df1['a'] = [1.2, 3.4, 2.9]
df1['g'] = ['r', 'q', 's']
dc['dataframe'] = df1

3. Save Session (this new Data object will be stored as a numpy array within the session file since it did not come from an external file)
4. Restore Session
5. Get the following error:

value error: 'Object arrays cannot be loaded when allow_pickle=False'

Expected behavior Pandas objects created within glue should not break session files.

We could simply add allow_pickle to np.load(), but perhaps this has undesired side effects?

Details:

• Operating System: MacOS 12.6
• Python version Python 3.9
• Glue version 1.6
• How you installed glue: conda

Additional context Sample session file attached: pandas_dataframe_session.glu.gz

Allow Categorical Variables to be Used in Color and Size Scaling in Scatter viewer

Description

This allows scatter plot cmap and size scaling attributes to be a categorical. This does not include a qualitative color-map for categorical attributes (as proposed in #1689) and obviously the size scaling is, in general, not going to be very meaningful, but for a small number of categorical levels it helps makes the points distinct.

An unresolved problem with this is that it might break glue-jupyter scatter plot because that does not consistently call ensure_numerical on the relevant attributes.