Hey,

I tried using the CLI tool but I'm getting this error: gpxpy.gpx.GPXException: latitude is mandatory in None (got None)

I tried setting --lon_max, --lat_max to 180,90 and --lat_min, --lon_min to 0,0 but still getting the same error.

Is this a problem with a specific .gpx file? And if so how can I find that one because the error doesn't show which file.

Thanks

Thanks for the app, it looks quite nice. But as I'm a bit new to Python, I'm having trouble to get it started.

I cloned the repo and tried from its root directory: [email protected] strava_py % python3 src/stravavis/cli.py /Users/jmizv/Downloads/gpx

But I just got that error message:

Traceback (most recent call last):
  File "/Users/jmizv/IdeaProjects/strava_py/src/stravavis/cli.py", line 91, in <module>
    main()
  File "/Users/jmizv/IdeaProjects/strava_py/src/stravavis/cli.py", line 41, in main
    from stravavis.plot_calendar import plot_calendar
ModuleNotFoundError: No module named 'stravavis'
[email protected] strava_py % 

I'm probably missing something obvious but I don't see it. Could you, @marcusvolz, please give an advice?

This PR adds a CLI for creating visualisations, and puts it into an installable package, ready for distributions via https://pypi.org so it can be installed using pip. (I can help with how to do that later!)

First, I moved the strava_py directory into a src directory. This is commonly used for Python projects. The main benefits is to make sure when you're testing, you're testing against something that has been installed, and not something that happens to have the same directory name in your current dir. Much more:

• https://packaging.python.org/en/latest/tutorials/packaging-projects/#a-simple-project
• https://blog.ionelmc.ro/2014/05/25/python-packaging/
• https://hynek.me/articles/testing-packaging/

The way to install from source:

pip install .

Or if you're developing, -e means an editable install, so you can make changes to your local source tree and they're reflected in what's run:

pip install -e .

Then I went for strava_py as the CLI name (this can be changed):

$ strava_py --help
usage: strava_py [-h] [-o OUTPUT_FILE] path

positional arguments:
  path                  Input path to folder with GPX and / or FIT files

options:
  -h, --help            show this help message and exit
  -o OUTPUT_FILE, --output_file OUTPUT_FILE
                        Output PNG file (default: plot.png)

Example run:

$ strava_py /tmp/my_strava_activities
Processing data...
Processing: 6628958226.gpx
Processing: 6626094662.gpx
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/fitdecode/reader.py:909: UserWarning: 'field "native_field_num" (idx #0) not found in message "field_description"' (local_mesg_num: 0; chunk_offset: 142); adding dummy dev data...
  warnings.warn(msg)
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/fitdecode/reader.py:909: UserWarning: 'field "native_field_num" (idx #0) not found in message "field_description"' (local_mesg_num: 0; chunk_offset: 162); adding dummy dev data...
  warnings.warn(msg)
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/fitdecode/reader.py:909: UserWarning: 'field "native_field_num" (idx #0) not found in message "field_description"' (local_mesg_num: 0; chunk_offset: 203); adding dummy dev data...
  warnings.warn(msg)
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/fitdecode/reader.py:909: UserWarning: 'field "native_field_num" (idx #0) not found in message "field_description"' (local_mesg_num: 0; chunk_offset: 249); adding dummy dev data...
  warnings.warn(msg)
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/fitdecode/reader.py:909: UserWarning: 'field "native_field_num" (idx #0) not found in message "field_description"' (local_mesg_num: 0; chunk_offset: 285); adding dummy dev data...
  warnings.warn(msg)
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/fitdecode/reader.py:909: UserWarning: 'field "native_field_num" (idx #0) not found in message "field_description"' (local_mesg_num: 0; chunk_offset: 326); adding dummy dev data...
  warnings.warn(msg)
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/fitdecode/reader.py:909: UserWarning: 'field "native_field_num" (idx #0) not found in message "field_description"' (local_mesg_num: 0; chunk_offset: 351); adding dummy dev data...
  warnings.warn(msg)
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/fitdecode/reader.py:909: UserWarning: 'field "native_field_num" (idx #0) not found in message "field_description"' (local_mesg_num: 0; chunk_offset: 397); adding dummy dev data...
  warnings.warn(msg)
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/fitdecode/reader.py:909: UserWarning: 'field "native_field_num" (idx #0) not found in message "field_description"' (local_mesg_num: 0; chunk_offset: 443); adding dummy dev data...
  warnings.warn(msg)
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/fitdecode/reader.py:909: UserWarning: 'field "native_field_num" (idx #0) not found in message "field_description"' (local_mesg_num: 0; chunk_offset: 487); adding dummy dev data...
  warnings.warn(msg)
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/fitdecode/reader.py:909: UserWarning: 'field "native_field_num" (idx #0) not found in message "field_description"' (local_mesg_num: 0; chunk_offset: 531); adding dummy dev data...
  warnings.warn(msg)
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/fitdecode/reader.py:909: UserWarning: 'field "native_field_num" (idx #0) not found in message "field_description"' (local_mesg_num: 0; chunk_offset: 575); adding dummy dev data...
  warnings.warn(msg)
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/fitdecode/reader.py:909: UserWarning: 'field "native_field_num" (idx #0) not found in message "field_description"' (local_mesg_num: 0; chunk_offset: 619); adding dummy dev data...
  warnings.warn(msg)
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/fitdecode/reader.py:909: UserWarning: 'field "native_field_num" (idx #0) not found in message "field_description"' (local_mesg_num: 0; chunk_offset: 663); adding dummy dev data...
  warnings.warn(msg)
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/fitdecode/reader.py:909: UserWarning: 'field "native_field_num" (idx #0) not found in message "field_description"' (local_mesg_num: 0; chunk_offset: 707); adding dummy dev data...
  warnings.warn(msg)
Processing: 6678512291.fit
Processing: 6619621876.gpx
Processing: 6604948302.gpx
Processing: 6860888905.fit
Processing: 6678648994.fit
Processing: 6676505895.fit
Processing: 6810573487.fit
Processing: 6623947053.gpx
Processing: 6630963143.gpx
Plotting facets...
Saved to plot.png

I'll also add a bunch of inline comments to explain what some things do, please feel free to ask more about any of this!

@hugovk Thanks for reviewing these and for the recent PRs. This one just adds the elevations small multiples plot.

I think I might have missed the suggested commits in your previous PRs, could you let me know if everything went through ok? Thanks!

Hi, I have been trying to apply this package on my data pull from Strava but in the first step of processing data I'm getting this error of "ValueError: All objects passed were None" attaching a screenshot for reference. I tried to give address of either the activities or routes folder which has gpx files but still the error remains constant

This is similar to https://github.com/marcusvolz/strava/issues/5.

Here's the same GPX using main:

Like https://stackoverflow.com/a/14457180/724176, rather than using latitude and longitude values, this uses a Mercator projection calculation to come up with x and y values instead.

Dummy, unit map width and height values are used, as we don't know and don't need to know the final size as matplotlib will take care of that.

And with this PR:

Measuring performance:

With main, on 2,946 GPX files (but commenting out some bits of cli.py to only run "Processing data..." and "Plotting map..."), it takes 11m45s on a Mac M1 16GB.

With the PR it takes a comparable 11m43s, which is really good because the transform is done by Pandas so presumably in C.

When I execute stravavis against this single attached file (5670595833.gpx.txt, you need to remove the txt extension) it fails with the below error. I tend to add emojis to my strava activity titles so this is in my opinion the problem. Not sure if you could exclude this easily? The files are certainly not encoded with Cp1252.

C:\Users\...>stravavis C:\Users\...
Processing data...
Processing ----------------------------------------   0% -:--:--
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "C:\Program Files\Python310\lib\multiprocessing\pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "C:\Users\...\AppData\Roaming\Python\Python310\site-packages\stravavis\process_data.py", line 13, in process_file
    return process_gpx(fpath)
  File "C:\Users\...\AppData\Roaming\Python\Python310\site-packages\stravavis\process_data.py", line 21, in process_gpx
    activity = gpxpy.parse(open(gpxfile))
  File "C:\Users\...\AppData\Roaming\Python\Python310\site-packages\gpxpy\__init__.py", line 37, in parse
    parser = mod_parser.GPXParser(xml_or_file)
  File "C:\Users\...\AppData\Roaming\Python\Python310\site-packages\gpxpy\parser.py", line 70, in __init__
    self.init(xml_or_file)
  File "C:\Users\...\AppData\Roaming\Python\Python310\site-packages\gpxpy\parser.py", line 82, in init
    text = xml_or_file.read() if hasattr(xml_or_file, 'read') else xml_or_file # type: ignore
  File "C:\Program Files\Python310\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 379: character maps to <undefined>
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Program Files\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Program Files\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\...\AppData\Roaming\Python\Python310\Scripts\stravavis.exe\__main__.py", line 7, in <module>
  File "C:\Users\...\AppData\Roaming\Python\Python310\site-packages\stravavis\cli.py", line 51, in main
    df = process_data(args.path)
  File "C:\Users\...\AppData\Roaming\Python\Python310\site-packages\stravavis\process_data.py", line 108, in process_data
    processed = list(it)
  File "C:\Users\...\AppData\Roaming\Python\Python310\site-packages\rich\progress.py", line 168, in track
    yield from progress.track(
  File "C:\Users\...\AppData\Roaming\Python\Python310\site-packages\rich\progress.py", line 1211, in track
    for value in sequence:
  File "C:\Program Files\Python310\lib\multiprocessing\pool.py", line 870, in next
    raise value
UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 379: character maps to <undefined>

5670595833.gpx.txt

It's a bit tedious to enter four params and values on the command line when creating a map:

stravavis activities --lon_min 23.516665 --lat_min 60.000467 --lon_max 26.390877 --lat_max 61.360291

This PR adds a --bbox shortcut for those, that takes a bounding box of comma-separated values of lon_min, lat_min, lon_max, lat_max:

stravavis activities --bbox 23.516665,60.000467,26.390877,61.360291

It's quite easy to find bounding boxes online, for example: http://bboxfinder.com/#60.000467,23.516665,61.360291,26.390877

I also noticed that these lat/lon values are not actually passed from the CLI to the plotting function, and neither are alpha or linewidth, so let's fix that too :)

Finally, bump some GitHub Actions versions for the CI.

Images are broken on PyPI:

Instead of linking to things like https://github.com/marcusvolz/strava_py/blob/main/plots/facets001.png, which is an HTML page showing the image, and which GitHub can apparently handle redirects to for the README, link directly to the image file, like https://raw.githubusercontent.com/marcusvolz/strava_py/main/plots/facets001.png

This also adds installation instructions to the README.

Also in Python we generally prefer to iterate over objects in a list (or tuple, or other so-called "iterable") rather than using an index of range(len(thing)).

So for example, instead of:

n = len(activities)
for i in range(n):
    print(activities[i])

This is more direct and readable:

for activity in activities:
    print(activity)

If we do need an index as well, enumerate is often used:

for i, activity in enumerate(activities):
    print(i, activity)

Use multiprocessing to speed up processing of input files, using the number of processes which matches the CPU count.

On my old dual-core Mac, the processing step speeds up:

• 601 files (all of 2021): 2m7s -> 1m20s
• 2,629 files (my full Strava archive): 10m29s -> 5m9s

Will be a larger gain for machines with more CPUs.

multiprocessing needs a single worker function, this is the new process_file which decides which of process_gpx and process_fit to call. They also need to be top-level functions to work with multiprocessing.

I also formatted the file using the popular Black autoformatter:

python -m pip install -U black
black . 

I also fixed a bug when using an input path like ~/dir, it needs to expand ~ before checking it's a dir (and appending *).