Python library for handling audio datasets.

Overview

AUDIOMATE

PyPI Build Status Documentation Status DeepSource DOI

Audiomate is a library for easy access to audio datasets. It provides the datastructures for accessing/loading different datasets in a generic way. This should ease the use of audio datasets for example for machine learning tasks.

import audiomate
from audiomate.corpus import io

# Download a dataset
esc_downloader = io.ESC50Downloader()
esc_downloader.download('/local/path')

# Load and work with the dataset
esc50 = audiomate.Corpus.load('/local/path', reader='esc-50')

# e.g. Read the audio signal and the label of specific sample/utterance
utterance = esc50.utterances['1-100032-A-0']
samples = utterance.read_samples()
label_list = utterance.label_lists[audiomate.corpus.LL_SOUND_CLASS]

for label in label_list:
  print(label.start, label.value)

Furthermore it provides tools for interacting with datasets (validation, splitting, subsets, merge, filter), extracting features, feeding samples for training ML models and more.

Currently supported datasets:

Currently supported formats:

Installation

pip install audiomate

Install the latest development version:

pip install git+https://github.com/ynop/audiomate.git

Dependencies

sox

For parts of the functionality (e.g. audio format conversion) sox is used. In order to use it, you have to install sox.

# macos
brew install sox

# with support for specific formats
brew install sox --with-lame --with-flac --with-libvorbis

# linux
apt-get install sox

# anaconda for macOS/windows/linux:
conda install -c conda-forge sox

Development

Prerequisites

It's recommended to use a virtual environment when developing audiomate. To create one, execute the following command in the project's root directory:

python -m venv .

To install audiomate and all it's dependencies, execute:

pip install -e .

Running the test suite

pip install -e .[dev]
pytest

With PyCharm you might have to change the default test runner. Otherwise, it might only suggest to use nose. To do so, go to File > Settings > Tools > Python Integrated Tools (on the Mac it's PyCharm > Preferences > Settings > Tools > Python Integrated Tools) and change the test runner to py.test.

Benchmarks

In order to check the runtime of specific parts, pytest-benchmark is used. Benchmarks are normal test functions, but call the benchmark fixture for the code under test.

To run benchmarks:

# Run all
pytest bench

# Specific benchmark
pytest bench/corpus/test_merge_corpus.py

To compare between different runs:

pytest-benchmark compare

Editing the Documentation

The documentation is written in reStructuredText and transformed into various output formats with the help of Sphinx.

To generate the documentation, execute:

pip install -e .[dev]
cd docs
make html

The generated files are written to docs/_build/html.

Versions

Versions is handled using bump2version. To bump the version:

bump2version [major,minor,patch,release,num]

In order to directly go to a final relase version (skip .dev/.rc/...):

bump2version [major,minor,patch] --new-version x.x.x

Release

Commands to create a new release on pypi.

rm -rf build
rm -rf dist

python setup.py sdist
python setup.py bdist_wheel
twine upload dist/*
Comments
  • Downloader for the Mozilla Common Voice Datasets

    Downloader for the Mozilla Common Voice Datasets

    This 3 commits add support for downloading the Common Voice datasets. All today available languages are supported. If new languages will be added, the provided tests fail or a new release of the Common Voice dataset is available then the DOWNLOAD_URLS need to be modified to include/replace the new/broken URL's.

    • Hopefully the commit message is OK
    • flake8 gives no errors
    • Docstring is added to the Downloader Class
    • Used ArchivDownloader as Base so Downloader is tested -> ~~Dataset availability tests (checks if HTTP HEAD Status Code is 200) are added~~
    • Is straight with master
    opened by Mastercuber 10
  • Show tatoeba process

    Show tatoeba process

    I also changed the label type, because this solved a problem i had and the other datasets seemed to have LL_WORD_TRANSCRIPT as default and LL_WORD_TRANSCRIPT_RAW only optionally.

    opened by DanBmh 10
  • Corpus with new common-voice dataset released june 2020 idx out of range

    Corpus with new common-voice dataset released june 2020 idx out of range

    Dear Support.

    Can you just readjust it with the new common-voice dataset? like I tested with German and it is throwing error, idx out of range. I tried to remove first two columns and use just client_id, file_path and text. But still error exists.

    Can you help?

    opened by Tortoise17 6
  • Check if file exists before downloading - GH #125

    Check if file exists before downloading - GH #125

    I've added the following changes per our discussion in #125:

    • Added force_redownload flag as optional arg
    • Added check for existing path to prevent duplicate downloads of corporum (unless flag is turned on)
    • Updated docstring to reflect new arg and exception

    pytest on "audiomate/test/corpus/io/" yields 8 failures both before and after changes. Please let me know what you think whenever you have a chance!

    opened by cdkini 6
  • Check if files exist before downloading

    Check if files exist before downloading

    Running

    esc_downloader = io.ESC50Downloader()
    esc_downloader.download('esc')
    esc_downloader.download('esc')
    

    downloads a dataset twice. Whereas the second time the download is completed not saved to the target folder because the folder already exists.

    Download File: 647Mit [01:24, 7.65Mit/s]
    Download File: 647Mit [01:35, 6.79Mit/s]
    Traceback (most recent call last):
      File "test.py", line 7, in <module>
        esc_downloader.download('esc')
      File "/Users/faro/repositories/audiomate_test/env/lib/python3.6/site-packages/audiomate/corpus/io/base.py", line 28, in download
        return self._download(target_path)
      File "/Users/faro/repositories/audiomate_test/env/lib/python3.6/site-packages/audiomate/corpus/io/downloader.py", line 52, in _download
        delete_subfolders=True
      File "/Users/faro/repositories/audiomate_test/env/lib/python3.6/site-packages/audiomate/utils/files.py", line 29, in move_all_files_from_subfolders_to_top
        shutil.move(src, target)
      File "/Users/faro/repositories/audiomate_test/env/lib/python3.6/shutil.py", line 548, in move
        raise Error("Destination path '%s' already exists" % real_dst)
    shutil.Error: Destination path 'esc/tests/tests' already exists
    
    • either check the existence of a root path before downloading any files at least raise a warning.
    • or add a force_redownload flag that would forcefully redownload the dataset (even if already existing)

    this issue is part of a JOSS review

    enhancement good first issue 
    opened by faroit 6
  • Catch file conversion error and add a lot of invalid files.

    Catch file conversion error and add a lot of invalid files.

    Note: I did not check these automatically excluded files. They threw an error in my deepspeech training. But earlier i did check some of the tuda files throwing the error and they all had a wrong transcription.

    opened by DanBmh 5
  • Unit test error with `TestLabel.test_read_samples`

    Unit test error with `TestLabel.test_read_samples`

    I am getting the following error with the unit tests:

    _________________________ TestLabel.test_read_samples __________________________
    
    self = <tests.annotations.test_label.TestLabel object at 0x7f800dfdb7f0>
    
        def test_read_samples(self):
            path = resources.sample_wav_file('wav_1.wav')
            track = tracks.FileTrack('wav', path)
            issuer = issuers.Issuer('toni')
            utt = tracks.Utterance('t', track, issuer=issuer, start=1.0, end=2.30)
        
            l1 = annotations.Label('a', 0.15, 0.448)
            l2 = annotations.Label('a', 0.5, 0.73)
            ll = annotations.LabelList(labels=[l1, l2])
        
            utt.set_label_list(ll)
        
            expected, __ = librosa.core.load(
                path,
                sr=None,
                offset=1.15,
                duration=0.298
            )
            assert np.array_equal(l1.read_samples(), expected)
        
            expected, __ = librosa.core.load(
                path,
                sr=None,
                offset=1.5,
                duration=0.23
            )
    >       assert np.array_equal(l2.read_samples(), expected)
    E       assert False
    E        +  where False = <function array_equal at 0x7f803bdbe598>(array([-0.00274658, -0.00021362, -0.00115967, ..., -0.00204468,\n       -0.00167847,  0.00067139], dtype=float32), array([-0.00274658, -0.00021362, -0.00115967, ..., -0.00167847,\n        0.00067139,  0.00082397], dtype=float32))
    E        +    where <function array_equal at 0x7f803bdbe598> = np.array_equal
    E        +    and   array([-0.00274658, -0.00021362, -0.00115967, ..., -0.00204468,\n       -0.00167847,  0.00067139], dtype=float32) = <bound method Label.read_samples of Label(a, 0.5, 0.73)>()
    E        +      where <bound method Label.read_samples of Label(a, 0.5, 0.73)> = Label(a, 0.5, 0.73).read_samples
    

    I am not sure what the issue is.

    opened by toddrme2178 4
  • New datasets and improved features

    New datasets and improved features

    This adds following new datasets:

    • Css10
    • LinguaLibre
    • Zamia-Speech

    The common voice reader now can import the new v5 release and has more languages.

    Some dataset loading problems were fixed.

    I also did add the progress logging bar again, because I missed this feature (but through audiomates logging interface that the used library can be easily exchanged)

    Note: This also changes the license to LGPL and I did use black for formatting

    opened by DanBmh 3
  • Test failure in `test_split_by_length_of_utterances`

    Test failure in `test_split_by_length_of_utterances`

    I am getting the following unit test error:

    ______________________ test_split_by_length_of_utterances ______________________
    
    benchmark = <pytest_benchmark.fixture.BenchmarkFixture object at 0x7f076e177a58>
    
        def test_split_by_length_of_utterances(benchmark):
            corpus = resources.generate_corpus(
                179,
                (250, 500),
                (1, 9),
                (0, 6),
                (1, 20),
                random.Random(x=234)
            )
        
            splitter = subset.Splitter(corpus, random_seed=324)
        
    >       benchmark(run, splitter)
    
    bench/corpus/subset/splitting/test_split_by_length_of_utterances.py:33: 
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    /usr/lib/python3.7/site-packages/pytest_benchmark/fixture.py:126: in __call__
        return self._raw(function_to_benchmark, *args, **kwargs)
    /usr/lib/python3.7/site-packages/pytest_benchmark/fixture.py:148: in _raw
        duration, iterations, loops_range = self._calibrate_timer(runner)
    /usr/lib/python3.7/site-packages/pytest_benchmark/fixture.py:276: in _calibrate_timer
        duration = runner(loops_range)
    /usr/lib/python3.7/site-packages/pytest_benchmark/fixture.py:91: in runner
        function_to_benchmark(*args, **kwargs)
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    
    splitter = <audiomate.corpus.subset.splitting.Splitter object at 0x7f076e177e80>
    
        def run(splitter):
    >       out = splitter.split_by_length_of_utterances(
                {
                    'train': 0.7,
                    'test': 0.15,
                    'dev': 0.15
                },
                separate_issuers=True
            )
    E       AttributeError: 'Splitter' object has no attribute 'split_by_length_of_utterances'
    
    bench/corpus/subset/splitting/test_split_by_length_of_utterances.py:9: AttributeError
    

    It looks like the method split_by_length_of_utterances has been renamed. Having CI tests running, such as travis, might help with this sort of issue.

    opened by toddrme2178 3
  • Updating license

    Updating license

    I'd like to add some new datasets and improved features (see https://github.com/ynop/audiomate/pull/129), but I'd also like to update the licence for this (MIT->LGPL).

    I'm developing all of this in my free time and want to make sure that everyone who uses and extends it also contributes back again, that it becomes an exchange of work and time. I also had some bad experiences with a partly open source and partly proprietary project where I put a lot of time into and which I can't use any more because the community support was stopped and development is now continued proprietary. That's why I generally prefer (weak) copyleft licenses. (Not that I believe this will happen here.)

    Using LGPL shouldn't affect any users, so from my point of view it has no negative consequences except the effort of the changing process, but ensures the exchange of work.

    opened by DanBmh 2
  • Kaldi I/O type error

    Kaldi I/O type error

    Dear friend.

    as I suppose, the audiomate is handling lot of formats very efficiently and perfectly.

    but, there is one format issue. Maybe you can understand. while data preparation, it throws error below

    File "/home/user/anaconda3/envs/zml/lib/python3.6/site-packages/audiomate/corpus/corpus.py", line 123, in load return reader.load(path) File "/home/user/anaconda3/envs/zml/lib/python3.6/site-packages/audiomate/corpus/io/base.py", line 95, in load self.type(), ' '.join(missing_files), path))

    OSError: Invalid data set of type kaldi: files wav.scp text not found at data/swc

    The data in each folder is consisting files below.
    aligned.swc audio.ogg audiometa.txt info.json wiki.html wiki.html wiki.xml

    Do I need to run any script first to convert these formats? or ?any of the library is ignored by me to install. Can you comment? why is it so? or any script to resolve and make like to be useable for audiomate.

    opened by Tortoise17 2
  • CVE-2007-4559 Patch

    CVE-2007-4559 Patch

    Patching CVE-2007-4559

    Hi, we are security researchers from the Advanced Research Center at Trellix. We have began a campaign to patch a widespread bug named CVE-2007-4559. CVE-2007-4559 is a 15 year old bug in the Python tarfile package. By using extract() or extractall() on a tarfile object without sanitizing input, a maliciously crafted .tar file could perform a directory path traversal attack. We found at least one unsantized extractall() in your codebase and are providing a patch for you via pull request. The patch essentially checks to see if all tarfile members will be extracted safely and throws an exception otherwise. We encourage you to use this patch or your own solution to secure against CVE-2007-4559. Further technical information about the vulnerability can be found in this blog.

    If you have further questions you may contact us through this projects lead researcher Kasimir Schulz.

    opened by TrellixVulnTeam 0
  • Bump numpy from 1.18.1 to 1.22.0

    Bump numpy from 1.18.1 to 1.22.0

    Bumps numpy from 1.18.1 to 1.22.0.

    Release notes

    Sourced from numpy's releases.

    v1.22.0

    NumPy 1.22.0 Release Notes

    NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are:

    • Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release.
    • A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across application such as CuPy and JAX.
    • NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data.
    • New methods for quantile, percentile, and related functions. The new methods provide a complete set of the methods commonly found in the literature.
    • A new configurable allocator for use by downstream projects.

    These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation.

    The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays.

    Expired deprecations

    Deprecated numeric style dtype strings have been removed

    Using the strings "Bytes0", "Datetime64", "Str0", "Uint32", and "Uint64" as a dtype will now raise a TypeError.

    (gh-19539)

    Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio

    numpy.loads was deprecated in v1.15, with the recommendation that users use pickle.loads instead. ndfromtxt and mafromtxt were both deprecated in v1.17 - users should use numpy.genfromtxt instead with the appropriate value for the usemask parameter.

    (gh-19615)

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • do not call head if filesize is provided with headers

    do not call head if filesize is provided with headers

    The download URL in swc.py is not valid anymore. The new server location does not support a HTTP head request. This patch is taking the file size from the previous GET and if the file size is known the head call is not executed.

    opened by kaoh 0
  • Retry download if closed connection without response

    Retry download if closed connection without response

    If server close connection, you dont have a "response code" and fail into a Exception.

    Proof's:

    `/usr/local/lib/python3.6/dist-packages/librosa/util/decorators.py:9: NumbaDeprecationWarning: An import was requested from a module that has moved location. Import of 'jit' requested from: 'numba.decorators', please update to use 'numba.core.decorators' or pin to Numba version 0.48.0. This alias will not be present in Numba version 0.50.0. from numba.decorators import jit as optional_jit Downloading voxforge-es ... INFO:audiomate:Found 2244 available archives to download Download Files: 0%| | 0/2244 [00:00<?, ?it/s]INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/ARI-20100216-hds.tgz" with size: 1361655 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AbdielNavarro-20130409-xvp.tgz" with size: 2625640 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AGOs-20100831-jdm.tgz" with size: 1915692 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AGOs-20100831-tmi.tgz" with size: 2009016 B INFO:audiomate:Finished download INFO:audiomate:Finished download INFO:audiomate:Finished download INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Adrian-20120203-lit.tgz" with size: 2607824 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AdrianRamirezEscalante-20111202-pgl.tgz" with size: 2594505 B INFO:audiomate:Finished download INFO:audiomate:Finished download Download Files: 0%| | 1/2244 [00:12<7:30:05, 12.04s/it]INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Adrin-20120203-uup.tgz" with size: 2230035 B INFO:audiomate:Finished download Download Files: 0%|▎ | 5/2244 [00:18<5:31:54, 8.89s/it]INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Aguatacuperche-20100823-zvv.tgz" with size: 2132406 B INFO:audiomate:Finished download Download Files: 0%|▍ | 7/2244 [00:21<4:11:57, 6.76s/it]INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Agus-20120528-sza.tgz" with size: 1533718 B INFO:audiomate:Finished download Download Files: 0%|▌ | 8/2244 [00:28<4:06:45, 6.62s/it]INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AlexVixgeck-20090827-qms.tgz" with size: 1105585 B INFO:audiomate:Finished download Download Files: 0%|▌ | 9/2244 [00:29<3:05:11, 4.97s/it]INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AlanNaiman-20121014-tke.tgz" with size: 2065020 B INFO:audiomate:Finished download INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Alejandra-20111109-oaf.tgz" with size: 1311472 B INFO:audiomate:Finished download Download Files: 0%|▋ | 10/2244 [00:37<3:37:16, 5.84s/it]INFO:audiomate:Finished download Download Files: 0%|▋ | 11/2244 [00:38<2:45:46, 4.45s/it]INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Alf_-20091127-dls.tgz" with size: 2144399 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Alf_-20091127-wrq.tgz" with size: 2084240 B INFO:audiomate:Finished download INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AlexanderRubioCaceres-20090902-set.tgz" with size: 1260802 B INFO:audiomate:Finished download INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Alf_-20091128-fan.tgz" with size: 2444151 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Alf_-20091127-dis.tgz" with size: 2013014 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Alf_-20091128-ozc.tgz" with size: 2397126 B INFO:audiomate:Finished download Download Files: 1%|▊ | 13/2244 [00:46<2:39:44, 4.30s/it]INFO:audiomate:Finished download INFO:audiomate:Finished download INFO:audiomate:Finished download Download Files: 1%|▉ | 14/2244 [00:53<3:14:10, 5.22s/it]INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Alf_-20100520-svz.tgz" with size: 2348385 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Alf_-20091128-ygd.tgz" with size: 2095451 B INFO:audiomate:Finished download INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Alf_-20091128-zoj.tgz" with size: 2226773 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Alf_-20100520-wvo.tgz" with size: 2189445 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Alf_-20091128-suc.tgz" with size: 2266510 B INFO:audiomate:Finished download INFO:audiomate:Finished download INFO:audiomate:Finished download Download Files: 1%|█▏ | 19/2244 [01:06<2:43:43, 4.42s/it]INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Alf_-20110915-mnf.tgz" with size: 2442328 B INFO:audiomate:Finished download Download Files: 1%|█▍ | 23/2244 [01:08<2:00:56, 3.27s/it]INFO:audiomate:Finished download Download Files: 1%|█▌ | 24/2244 [01:09<1:38:17, 2.66s/it]INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Alf_-20110915-sff.tgz" with size: 2128985 B INFO:audiomate:Finished download Download Files: 1%|█▋ | 25/2244 [01:12<1:44:04, 2.81s/it]INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Alf_-20110915-wfh.tgz" with size: 2021008 B INFO:audiomate:Finished download Download Files: 1%|█▋ | 26/2244 [01:20<2:35:21, 4.20s/it]INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Alfredo-20090314-you.tgz" with size: 2117498 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Alfredo-20090314-bwm.tgz" with size: 1780407 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Alfredo-20090314-nui.tgz" with size: 2000040 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AndreaBurgos-20120424-pef.tgz" with size: 1929018 B INFO:audiomate:Finished download INFO:audiomate:Finished download Download Files: 1%|█▊ | 27/2244 [01:32<4:01:22, 6.53s/it]INFO:audiomate:Finished download Download Files: 1%|█▊ | 28/2244 [01:33<2:58:04, 4.82s/it]INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AndreyBarrientosCostaRica-20130708-inn.tgz" with size: 1721089 B INFO:audiomate:Finished download INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AndreyBarrientosCostaRica-20130710-dtw.tgz" with size: 1736300 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AndreyBarrientosCostaRica-20130708-eaz.tgz" with size: 1909391 B INFO:audiomate:Finished download Download Files: 1%|█▉ | 30/2244 [01:38<2:35:58, 4.23s/it]INFO:audiomate:Finished download INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AndreyBarrientosCostaRica-20130710-bhv.tgz" with size: 1776404 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AndreyBarrientosCostaRica-20130710-iqt.tgz" with size: 1986977 B INFO:audiomate:Finished download INFO:audiomate:Finished download Download Files: 1%|██ | 31/2244 [01:45<3:04:53, 5.01s/it]INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AndreyBarrientosCostaRica-20130710-ttq.tgz" with size: 1749826 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AndreyBarrientosCostaRica-20130710-stu.tgz" with size: 1753877 B INFO:audiomate:Finished download Download Files: 1%|██▏ | 33/2244 [01:49<2:27:32, 4.00s/it]INFO:audiomate:Finished download INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AndreyBarrientosCostaRica-20130710-rat.tgz" with size: 1553412 B INFO:audiomate:Finished download INFO:audiomate:Finished download Download Files: 2%|██▎ | 36/2244 [01:55<2:07:09, 3.46s/it]INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AndreyBarrientosCostaRica-20130710-twg.tgz" with size: 1853932 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AndreyBarrientosCostaRica-20130720-mkx.tgz" with size: 1973814 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AndreyBarrientosCostaRica-20130720-gix.tgz" with size: 1687520 B INFO:audiomate:Finished download INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AndreyBarrientosCostaRica-20130711-jje.tgz" with size: 1589503 B INFO:audiomate:Finished download Download Files: 2%|██▌ | 39/2244 [02:02<1:55:15, 3.14s/it]INFO:audiomate:Finished download INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AntonioH-20160531-gwl.tgz" with size: 849178 B INFO:audiomate:Finished download INFO:audiomate:Finished download Download Files: 2%|██▌ | 40/2244 [02:08<2:22:21, 3.88s/it]INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Andros-20120408-age.tgz" with size: 2205975 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Argorn-20141126-igm.tgz" with size: 2165540 B INFO:audiomate:Finished download INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Ases-20120213-esa.tgz" with size: 2552878 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AndreyBarrientosCostaRica-20130720-slz.tgz" with size: 1946756 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/AntonioMedina-20121208-kzq.tgz" with size: 2329390 B INFO:audiomate:Finished download INFO:audiomate:Finished download Download Files: 2%|██▊ | 43/2244 [02:22<2:33:13, 4.18s/it]INFO:audiomate:Finished download INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Ashrey-20100714-yhd.tgz" with size: 1937711 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/BLs-20151219-kwe.tgz" with size: 2128918 B INFO:audiomate:Finished download Download Files: 2%|██▉ | 46/2244 [02:26<2:01:53, 3.33s/it]INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/BLs-20151219-tsj.tgz" with size: 1742771 B INFO:audiomate:Finished download Download Files: 2%|███▏ | 49/2244 [02:32<1:44:45, 2.86s/it]INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/BLs-20151219-vvx.tgz" with size: 2222457 B INFO:audiomate:Finished download INFO:audiomate:Finished download INFO:audiomate:Finished download INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Azul4-20131019-ssk.tgz" with size: 2093735 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Campotrigo-20130607-bnz.tgz" with size: 3093233 B INFO:audiomate:Finished download INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/CarlosEspinoAngulo-20140525-vip.tgz" with size: 2302281 B INFO:audiomate:Finished download INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/BLs-20151219-xzd.tgz" with size: 1994095 B INFO:audiomate:Finished download INFO:audiomate:Finished download Download Files: 2%|███▎ | 50/2244 [02:49<4:22:27, 7.18s/it]INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/CarlosMedina-20130524-wqx.tgz" with size: 2357299 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/CarlosFiebigCosmelli-20121128-aud.tgz" with size: 1756883 B INFO:audiomate:Finished download INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Carlos_Daniel-20120325-khj.tgz" with size: 2414791 B INFO:audiomate:Finished download INFO:audiomate:Finished download INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Condor-20120214-prz.tgz" with size: 1716730 B INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Bernardo-20101210-tiq.tgz" with size: 1849584 B INFO:audiomate:Finished download INFO:audiomate:Download file from "http://www.repository.voxforge1.org/downloads/es/Trunk/Audio/Main/16kHz_16bit/Cixert-20150207-ywe.tgz" with size: 1980326 B INFO:audiomate:Finished download Download Files: 3%|███▊ | 58/2244 [03:05<1:56:45, 3.20s/it] Traceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 677, in urlopen chunked=chunked, File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 426, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 421, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.6/http/client.py", line 1356, in getresponse response.begin() File "/usr/lib/python3.6/http/client.py", line 307, in begin version, status, reason = self._read_status() File "/usr/lib/python3.6/http/client.py", line 276, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/requests/adapters.py", line 449, in send timeout=timeout File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 727, in urlopen method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] File "/usr/local/lib/python3.6/dist-packages/urllib3/util/retry.py", line 403, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/local/lib/python3.6/dist-packages/urllib3/packages/six.py", line 734, in reraise raise value.with_traceback(tb) File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 677, in urlopen chunked=chunked, File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 426, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 421, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.6/http/client.py", line 1356, in getresponse response.begin() File "/usr/lib/python3.6/http/client.py", line 307, in begin version, status, reason = self._read_status() File "/usr/lib/python3.6/http/client.py", line 276, in _read_status raise RemoteDisconnected("Remote end closed connection without" urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "/DeepSpeech/deepspeech-polyglot/preprocessing/download_data.py", line 108, in main() File "/DeepSpeech/deepspeech-polyglot/preprocessing/download_data.py", line 97, in main dl.download(os.path.join(args.target_path, "voxforge")) File "/usr/local/lib/python3.6/dist-packages/audiomate/corpus/io/base.py", line 40, in download return self._download(target_path) File "/usr/local/lib/python3.6/dist-packages/audiomate/corpus/io/voxforge.py", line 73, in _download downloaded = self.download_files(available, temp_folder) File "/usr/local/lib/python3.6/dist-packages/audiomate/corpus/io/voxforge.py", line 110, in download_files dl_result = download.download_files(url_to_target, num_threads=self.num_workers) File "/usr/local/lib/python3.6/dist-packages/audiomate/utils/download.py", line 35, in download_files description='Download Files' File "/usr/local/lib/python3.6/dist-packages/audiomate/logutil.py", line 21, in progress for x in tqdm.tqdm(iterable, desc=description, total=total): File "/usr/local/lib/python3.6/dist-packages/tqdm/std.py", line 1165, in iter for obj in iterable: File "/usr/lib/python3.6/multiprocessing/pool.py", line 735, in next raise value File "/usr/lib/python3.6/multiprocessing/pool.py", line 119, in worker result = (True, func(*args, **kwds)) File "/usr/local/lib/python3.6/dist-packages/audiomate/utils/download.py", line 43, in _download_file return download_file(item[0], item[1]) File "/usr/local/lib/python3.6/dist-packages/audiomate/utils/download.py", line 62, in download_file r = requests.get(url, stream=True) File "/usr/local/lib/python3.6/dist-packages/requests/api.py", line 76, in get return request('get', url, params=params, **kwargs) File "/usr/local/lib/python3.6/dist-packages/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 530, in request resp = self.send(prep, **send_kwargs) File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 643, in send r = adapter.send(request, **kwargs) File "/usr/local/lib/python3.6/dist-packages/requests/adapters.py", line 498, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))`

    opened by CostanzoPablo 0
  • Reproducibility of dataset downloads

    Reproducibility of dataset downloads

    When downloading corpora from versioned data stores, I would expect to take into account a tag or specific hash of that dataset. That way users are sure if a specific version of audiomate yields an identical corpus to foster reproducibility.

    e.g. lets take the esc-50 corpus: the root url downloads directly from master branch https://github.com/ynop/audiomate/blob/28696c0e46ab1d7f3d3f53f8a9086c724b7b947a/audiomate/corpus/io/esc.py#L11

    To improve reproducibility, I suggest that audiomate uses tags where possible (github, zenodo, ...) and furthermore provide a checksum mechanism that verifies a successful download.

    This issue is part of a JOSS review https://github.com/openjournals/joss-reviews/issues/2135

    enhancement 
    opened by faroit 2
Releases(v6.0.0)
Small Python application that links a Digico console and Reaper, handling automatic marker insertion and tracking.

Digico-Reaper-Link This is a small GUI based helper application designed to help with using Digico's Copy Audio function with a Reaper DAW used for re

Justin Stasiw 10 Oct 24, 2022
Gammatone-based spectrograms, using gammatone filterbanks or Fourier transform weightings.

Gammatone Filterbank Toolkit Utilities for analysing sound using perceptual models of human hearing. Jason Heeris, 2013 Summary This is a port of Malc

Jason Heeris 188 Dec 14, 2022
:sound: Play and Record Sound with Python :snake:

Play and Record Sound with Python This Python module provides bindings for the PortAudio library and a few convenience functions to play and record Nu

spatialaudio.net 750 Dec 31, 2022
This bot can stream audio or video files and urls in telegram voice chats

Voice Chat Streamer This bot can stream audio or video files and urls in telegram voice chats :) 🎯 Follow me and star this repo for more telegram bot

WiskeyWorm 4 Oct 09, 2022
Synthesia but open source, made in python and free

PyPiano Synthesia but open source, made in python and free Requirements are in requirements.txt If you struggle with installation of pyaudio, run : pi

DaCapo 11 Nov 06, 2022
A2DP agent for promiscuous/permissive audio sinc.

Promiscuous Bluetooth audio sinc A2DP agent for promiscuous/permissive audio sinc for Linux. Once installed, a Bluetooth client, such as a smart phone

Jasper Aorangi 4 May 27, 2022
A python program to cut longer MP3 files (i.e. recordings of several songs) into the individual tracks.

I'm writing a python script to cut longer MP3 files (i.e. recordings of several songs) into the individual tracks called ReCut. So far there are two

Dönerspiess 1 Oct 27, 2021
Python wrapper around sox.

pysox Python wrapper around sox. Read the Docs here. This library was presented in the following paper: R. M. Bittner, E. J. Humphrey and J. P. Bello,

Rachel Bittner 446 Dec 07, 2022
Klangbecken: The RaBe Endless Music Player

Klangbecken Klangbecken is the minimalistic endless music player for Radio Bern RaBe based on liquidsoap. It supports configurable and editable playli

Radio Bern RaBe 8 Oct 09, 2021
A Youtube audio player for your terminal

AudioLine A lightweight Youtube audio player for your terminal Explore the docs » View Demo · Report Bug · Request Feature · Send a Pull Request About

Haseeb Khalid 26 Jan 04, 2023
An AI for Music Generation

An AI for Music Generation

Hao-Wen Dong 1.3k Dec 31, 2022
Open-Source bot to play songs in your Telegram's Group Voice Chat. Powered by @Akki_ThePro

VcPlayer Telegram Voice-Chat Bot [PyTGCalls] ⇝ Requirements ⇜ Account requirements A Telegram account to use as the music bot, You cannot use regular

Akki ThePro 2 Dec 25, 2021
Implementation of "Slow-Fast Auditory Streams for Audio Recognition, ICASSP, 2021" in PyTorch

Auditory Slow-Fast This repository implements the model proposed in the paper: Evangelos Kazakos, Arsha Nagrani, Andrew Zisserman, Dima Damen, Slow-Fa

Evangelos Kazakos 57 Dec 07, 2022
Audio2midi - Automatic Audio-to-symbolic Arrangement

Automatic Audio-to-symbolic Arrangement This is the repository of the project "Audio-to-symbolic Arrangement via Cross-modal Music Representation Lear

Ziyu Wang 24 Dec 05, 2022
Okaeri-Music is a telegram music bot project, allow you to play music on voice chat group telegram.

Okaeri-Music is a telegram bot project that's allow you to play music on telegram voice chat group

Wahyusaputra 1 Dec 22, 2021
Multi-Track Music Generation with the Transfomer and the Johann Sebastian Bach Chorales dataset

MMM: Exploring Conditional Multi-Track Music Generation with the Transformer and the Johann Sebastian Bach Chorales Dataset. Implementation of the pap

102 Dec 08, 2022
DeepSpeech is an open source embedded (offline, on-device) speech-to-text engine which can run in real time on devices ranging from a Raspberry Pi 4 to high power GPU servers.

Project DeepSpeech DeepSpeech is an open-source Speech-To-Text engine, using a model trained by machine learning techniques based on Baidu's Deep Spee

Mozilla 20.8k Jan 03, 2023
A Quick Music Player Made Fully in Python

Quick Music Player Made Fully In Python. Pure Python, cross platform, single function module with no dependencies for playing sounds. Installation & S

1 Dec 24, 2021
Synchronize a local directory of songs' (MP3, MP4) metadata (genre, ratings) and playlists with a Plex server.

PlexMusicSync Synchronize a local directory of songs' (MP3, MP4) metadata (genre, ratings) and playlists (m3u, m3u8) with a Plex server. The song file

Tom Goetz 9 Jul 07, 2022
TONet: Tone-Octave Network for Singing Melody Extraction from Polyphonic Music

TONet Introduction The official implementation of "TONet: Tone-Octave Network for Singing Melody Extraction from Polyphonic Music", in ICASSP 2022 We

Knut(Ke) Chen 29 Dec 01, 2022