A number of features that are meant to improve the usability of pulse2percept:

• There should be a Jupyter notebook that showcases the current, up-to-date usage of the model: examples/notebook/0.0-example-usage.ipynb. This will be useful as we continue to change the functionality and syntax of the model.

• joblib and dask are now optional.

• It's now super-easy to create common electrode arrays, such as an Argus I with a given center location (x_center, y_center), a height h (either a list or a scalar), and a rotation angle rot:

  argus = e2cm.ArgusI(x_center, y_center, h, rot)
  

  Instead of stuff like this:

  # Location on retina for each electrode
  x_elec_loc = np.array([-6825.092215, -6332.563035, -5840.033855, -5347.504675,
                      -6194.683612, -5702.154432, -5209.625252, -4717.096072,
                      -5564.275010, -5071.745829, -4579.216649, -4086.687469,
                      -4933.866407, -4441.337226, -3948.808046, -3456.278866])
  y_elec_loc = np.array([-655.666769, -25.258166, 605.150437, 1235.559040,
                      -1148.195949, -517.787346, 112.621257, 743.029860,
                      -1640.725129, -1010.316526, -379.907924, 250.500679,
                      -2133.254310, -1502.845707, -872.437104, -242.028501])
  
  # Alternate electrode size (Argus I)
  r_arr = np.array([260, 520, 260, 520]) / 2.0
  r_arr = np.concatenate((r_arr, r_arr[::-1], r_arr, r_arr[::-1]), axis=0)
  h_arr = np.ones(16)*100
  argus = e2cm.ElectrodeArray(r_arr.tolist(), x_elec_loc, y_elec_loc, h_arr.tolist())
  

  ArgusI is an instance of type ElectrodeArray. It can automatically figure out the size and retinal location of each electrode in the array. No more math by hand.

• The same is true for e2cm.ArgusII.

• ElectrodeArray is now indexable. Electrodes can be addressed either by index or by name:

  # An ArgusI array centered on the fovea
  argus = e2cm.ArgusI()
  
  # Get the second electrode in the array, by index or by name:
  el = argus[1]
  el = argus['A2']
  
  # Find the index of an electrode:
  el_idx = argus.get_index('C3')
  

• ElectrodeArray no longer requires all arguments to be lists. Now supports ints, floats, lists, and numpy arrays as inputs. Electrode type is mandatory and comes first (to avoid confusion between epiretinal and subretinal arrays). For example, all the following are now possible:

  implant = e2cm.ElectrodeArray('epiretinal', 0, 1, 2, 3)
  implant = e2cm.ElectrodeArray('subretinal', [0, 0], [1, 1], [2, 2], [3, 3])
  # If you must:
  implant = e2cm.ElectrodeArray('epiretinal', 0, [1], np.array([2]), np.array([[3]]))
  

• The ec2b.pulse2percept call now accepts an input stimulus stim, an electrode array implant, a temporal model tm, and a retina retina. If not specified, a temporal model with default parameters is used. If not specified, a retina large enough to fit the electrode array is used. dolayer is no longer specified, since it is redundant (depends on the electrode type of implant). Order of arguments has changed, too:

  def pulse2percept(stim, implant, tm=None, retina=None,
                    rsample=30, scale_charge=42.1, tol=0.05, use_ecs=True,
                    engine='joblib', dojit=True, n_jobs=-1):
  

  Input arguments are type checked in order to make it easier to spot examples with outdated API calls.

• Input stimuli specification is much more flexible. Single-electrode arrays take just a single pulse train. For multi-electrode arrays, the user can specify a list of pulse trains, or specify all electrodes that should receive non-zero pulse trains by name:

  # Create an Argus I array centered on the fovea
  implant = e2cm.ArgusI()
  
  # Send some non-zero pulse train to electrodes 'C2' and 'D3'. All other
  # electrodes get zeros
  pt = e2cm.Psycho2Pulsetrain(tsample=5e-6, freq=50, amp=20)
  stim = {'C2': pt, 'D3': pt}
  ec2b.pulse2percept(stim, implant)
  

• The function ec2b.get_brightest_frame finds and returns the frame in a brightness movie that contains the brightest pixel.

• The retina object now automatically generates a descriptive filename based on the input arguments (e.g., '../retina_s250_l2.0_rot0.0_5000x5000.npz'), and checks if the file exists. The retina can now easily be rotated by specifying some angle rot. Axon maps, rot, and axon_lambda are now all members of the object.

• You can now have single-pixel retinas, which is handy for model-tuning when you only need the brightest pixel.

• All new functions have their own test cases, with increased coverage.

This PR adds a video2pulsetrain function to convert video files into stimuli. Videos can be loaded from file, and percepts can be stored to file. Uses Scikit-Video with either an ffmpeg or libav backend. Replaces some deprecated functionality such as p2p.files.savemoviefiles and p2p.stimuli.Movie2Pulsetrain.

• [X] Add video2pulsetrain
  • [X] processes every frame with image2pulsetrain
  • [X] supports both amplitude and frequency modulation
  • [X] makes Movie2Pulsetrain, receptive_field, and retinalmovie2electrodtimeseries obsolete
• [X] Add load/save utilities to files module
  • [X] load_video loads a video file either into a NumPy array or a TimeSeries object
  • [X] save_video stores either a NumPy ndarray or a TimeSeries object in a video file
  • [X] save_video_sidebyside stores both an input video and the TimeSeries output in a video file
  • [X] makes savemoviefiles and npy2movie obsolete
• [X] Add easy concatenation of pulse trains pt1.append(pt2)
• [X] Switch Travis build to trusty distro
• [X] Fix gap_size issue in image2pulsetrain
• [X] Fix effectivecurrent calculation when not all layers selected
• [X] Add tests that process example images/videos from Scikit-Learn/Scikit-Video
• [X] Complete documentation of new functionality
• [X] Add updated image2percept Jupyter Notebook
• [X] Add video2percept Jupyter Notebook

SciPy reviewers requested the following changes to the code:

• [X] Rename dojit to use_jit. Apply it only when both has_jit and use_jit are True.

The following changes will make the code equivalent to what is present in the SciPy paper, for a hypothetical 0.2 release.

• [X] Default value for as_timeseries in p2p.files.load_video should be True.
• [X] Add Horsager model.

Hi! I would really appreciate your help... first of all, I love this so thank you! I succeeded in creating a stimulus from an image for ARGUSII, now I would like to do the same for PRIMA40, but am getting this error ValueError: the input array must have size 3 along channel_axis, got (25, 28). I am not sure what to do... thank you so much in advance.

This PR fixes and extends the spatial component of the model (closing issue #71).

Determining the contribution of axonal stimulation to the effective current map is now done axon-by-axon, not pixel-by-pixel, which allows the model to correctly handle multi-electrode stimulation. (Before, if a pixel got zero input current, the model predicted zero output - although the soma at that pixel might be connected to an axon elsewhere.)

When it comes to sensitivity of an individual axon segment, users can select from two functions:

• where sensitivity drops exponentially with distance from the soma
• where sensitivity is roughly the inverse of axonal stimulation thresholds reported in Jeng, Tang, Molnar, Desai, and Fried (2011). The sodium channel band shapes the response to electric stimulation in retinal ganglion cells. J Neural Eng 8 (036022).

Highlights:

• Two bugs in axon map calculations fixed.
• All spatial calculations are now parallelized (including the one-time building of the retinal grid).
• One-time setup now includes the following steps:
  • Grow a number of fiber bundles using the model by Jansonius et al. (2009).
  • Assume there is a neuron at every grid location: Use the above axon bundles to assign an axon to each neuron.
  • For every axon segment, calculate distance to soma. Snap axon locations to the grid using a nearest-neighbor tree structure for reduced workload.
• Calculating effective current for a given stimulus now involves:
  • Determine sensitivity of each axon segment according to a sensitivity_rule using the distance term calculated above.
  • Based on the sensitivity of each axon segment, determine the current contribution of that axon according to a contribution_rule.
• Functions jansonius and make_axon_map are now deprecated.
• Better docs, more tests.

P.S. The p2p.retina.Grid class is becoming a God object and should be refactored. Soon.

A few improvements for the ElectrodeArray class and some helper functions. Most of these are already being used in my experiment branches in one form or the other, but are now also tested and integrated. So the purpose of their existence is mostly to make my life easier. 😸

Argus I arrays now support legacy naming (L1, L2, ..., L8, M1, M2, ..., M8).

The electrode-to-retina distance depends on electrode type and electrode location. This computation has been moved to Electrode.set_height so that heights can be overridden.

Keeping track of electrode names in ElectrodeArray.names is redundant, because every Electrode has a name itself. Setting custom (or new) names would have to be done in both places, otherwise indexing might break. Therefore, I removed ElectrodeArray.names. In order to do the indexing, the loop has to be over name of every Electrode in an ElectrodeArray. In order to speed up the average lookup time, the array is traversed in random order.

The parameter tol of pulse2percept now specifies a fraction instead of an absolute value: For example, tol=0.1 will discard all pixels whose effective current is <10% of the max effective current value.

I also added a helper function that can find files in a directory whose names match a regular expression pattern.

Bug Description Trying to visualize various custom parametrized square implants, it seems like the 'r' parameter does not agree with the radius of the electrode when the implant is visualized using plot_implant_on_axon_map.

To Reproduce plot_implant_on_axon_map(ProsthesisSystem(ElectrodeGrid((2, 2), r=1000, spacing=(2000), etype=DiskElectrode)))

Expected behavior In the diagram from the above snippet, there should be 4 electrodes with a radius of 1000 microns touching each other and covering a total area of 4000x4000 microns.

Screenshots However we get this

Why this is happening Looking around the code this seems to trackback to this line of code where the markersize is set https://github.com/pulse2percept/pulse2percept/blob/9acf65990c51ec4612eef5e0c32b8fb218beb13b/pulse2percept/viz/axon_map.py#L161

In the End In the end, what matter is if it is just a visual thing and you couldn't find how to better visualize it (because I looked into it and I didn't find something very useful except that the units of markersize are points^2 --> area) or if the 'r' in the code snippet above means something different than the radius of the electrode in microns (since this would have impact on the results from the simulations)

... and added respective isinstance checks and conditional sit.resize (in stimuli.py).

The implant parameters are from the following paper:

@article{Stingl2015, author = {Stingl, Katarina and Bartz-Schmidt, Karl Ulrich and Besch, Dorothea and Chee, Caroline K. and Cottriall, Charles L. and Gekeler, Florian and Groppe, Markus and Jackson, Timothy L. and MacLaren, Robert E. and Koitschev, Assen and Kusnyerik, Akos and Neffendorf, James and Nemeth, Janos and Naeem, Mohamed Adheem Naser and Peters, Tobias and Ramsden, James D. and Sachs, Helmut and Simpson, Andrew and Singh, Mandeep S. and Wilhelm, Barbara and Wong, David and Zrenner, Eberhart}, doi = {10.1016/j.visres.2015.03.001}, journal = {Vision Research}, pages = {149--160}, publisher = {Elsevier Ltd}, title = {{Subretinal Visual Implant Alpha IMS - Clinical trial interim report}}, url = {http://dx.doi.org/10.1016/j.visres.2015.03.001}, volume = {111}, year = {2015} }

Description

Attempt to speed up VideoStimulus encode function by cythonizing and optimizing loops. All tests pass, speedup is minimal. It is worth wondering whether the change is worth merging into master.

Closes #377 and #462

Type of Change

Please delete options that are not relevant.

• [x] New feature (non-breaking change which adds functionality)

Checklist

• [x] I have performed a self-review of my own code
• [x] I have commented my code, particularly in hard-to-understand areas
• [x] I have made corresponding changes to the documentation
• [x] New and existing unit tests pass locally with my changes

For detailed information on these and other aspects, see the contribution guidelines: https://pulse2percept.readthedocs.io/en/latest/developers/contributing.html

Hi guys, First of all thank you very much for your framework :+1: When simulating longer videos I have noticed that the percepts become black after time. After further investigation I think it is because of the charge accumulation making the system less sensitive the more charge is accumulated. However, sensitivity is never restored as far as I can see (and was of no interest in generating the model I guess).

So there are two questions that I have right now: 1.) Are my assumptions correct? 2.) Do you know of any "plausible" way, e.g. a known model etc. for "resetting" the sensitivity?

Thanks in advance

EDIT: To be more precise: I am simulating a 20 second clip (25fps) of a still image for testing (amplitude encoded between [0,1] and an implant with a working frequency between 1-20 Hz (1ms cathodic pulses)).

Attached you can see two responses (working frequency 5Hz and 20Hz) of a fixed position within the percept. It is clear to me that the sensitivity drops faster for pulses with 20Hz, since the accumulated charge will increase faster than with 5Hz, however, in both cases the accumulated charge will linearly increase over time making the system insensitive in the long run.

Single core benchmarks:

• stim = p2p.stimuli.PulseTrain(0.05 / 1000, freq=20, amp=150, pulse_dur=0.45 / 1000, dur=0.5)
• Convolutions (pure Python): 594 ms ± 6.72 ms
• Finite difference model (pure Python): 644 ms ± 13.8 ms
• Finite difference model (naive Cython): 166 ms ± 2.06 ms
• Finite difference model (pulse2percept): 2.96 ms ± 9.42 µs

Describe the bug percept.save() results in videos that look different from percept.plot() and percept.play(). Not completely sure of the cause yet, but I suspect it has something to do with normalization (e.g. vmin, vmax) or uint8 conversion

To Reproduce

import pulse2percept as p2p 
m = p2p.models.Model(spatial=p2p.models.AxonMapSpatial(), temporal=p2p.models.FadingTemporal())
m.build()
stim = p2p.stimuli.Stimulus({'A3' : p2p.stimuli.BiphasicPulseTrain(20, 1, 0.45)})
percept = m.predict_percept(p2p.implants.ArgusII(stim=stim))

percept.plot():

from IPython.display import Video
percept.save('test.mp4')
Video("test.mp4")

System specifications (please complete the following information): Python 3.10 Ubuntu 18.04

Some scikit-image functions take keyword arguments, but currently these cannot be set via the ImageStimulus methods.

For example, a user might want to specify order=0 for ImageStimulus.resize.

Passing a function to ImageStimulus.apply that changes the image resolution will currently break, because the new stimulus is supposed to have the same number of electrodes. Instead, add another keyword arguments that overwrites self.electrodes if given (similar to ImageStimulus.rgb2gray and others).

A pessimist might see this as a bug ("can't pass skimage.transform.resize to ImageStimulus.apply")...

There should be a way to make a GratingStimulus or BarStimulus from a single frame; not as a VideoStimulus, but as an ImageStimulus.

One could try passing time=[0], but this will still break the plotting method.

Passing an empty stimulus to an implant & predicting a percept leads to inconsistent results, depending on how the stimulus is passed.

If called like this:

model = p2p.models.ScoreboardModel().build()
model.predict_percept(p2p.implants.ArgusII(stim=None))

the model will realize that there is no stimulus to be applied, so the output is None (no error thrown).

However, sometimes the stimulus is empty by accident, for example if dynamically built as a list or dictionary. Currently, passing stim=[] gives a "Number of electrodes provided (60) does not match the number of electrodes in the data (0)" error (which is fair, but inconsistent with the above) and passing stim={} gives a "Buffer has wrong number of dimensions (expected 2, got 1)" error (again, inconsistent).

All of these should lead to the same outcome, which is no error and None output.

Description

Please include a summary of the change and which issue it closes below. Include relevant motivation and context.

Closes #274

Type of Change

Please delete options that are not relevant.

• [ ] New feature (non-breaking change which adds functionality)

Checklist

• [ ] I have performed a self-review of my own code
• [ ] I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [ ] I have added tests that prove my fix is effective or that my feature works
• [ ] New and existing unit tests pass locally with my changes

For detailed information on these and other aspects, see the contribution guidelines: https://pulse2percept.readthedocs.io/en/latest/developers/contributing.html