Python wrapper for Xeno-canto API 2.0. Enables downloading bird data with one command line

Last update: Dec 10, 2022

Overview

birdData

BirdData is a python wrapper for Xeno-canto API 2.0. Enables user to download bird data with one command line. BirdData supports multithreading download.

Environment

Download repo to local:

git clone [email protected]:realzza/birdData.git

Set up environment:

pip install -r requirement.txt

Usage

Single-thread

Download audio data for one bird species. Use scientific name starting with lowercase. e.g, cettia cetti.

python download.py --name "cettia cetti"

Download audio data for a file of species names. Format requirement: names divided by "\n"

python download.py --name name_file

General Usage:

usage: download.py [-h] --name NAME

download bird audios

optional arguments:
  -h, --help   show this help message and exit
  --name NAME  [1] name of one bird species; [2] file of bird species spaced
               by '\n'

Multi-thread

Speed up downloading using multiple threads.

python download-mult.py --name "cettia cetti" --process-ratio 0.6

Download multiple birds in a file, format requirement: names divided by "\n"

python download-mult.py --name name_file --process-ratio 0.6

General Usage:

usage: download-mult.py [-h] --name NAME [--process-ratio PROCESS_RATIO]

download bird audios

optional arguments:
  -h, --help            show this help message and exit
  --name NAME           [1] name of one bird species; [2] file of bird species
                        spaced by '\n'
  --process-ratio PROCESS_RATIO
                        float[0~1], define cpu utilities in downloading audios
                        [default: 0.8]

To-do

[12.29] multiprocess download
define sample rate prior to download

Contact

Feel free to file an issue had you encountered any problems. Have fun!

PRAW, an acronym for "Python Reddit API Wrapper", is a python package that allows for simple access to Reddit's API.

PRAW: The Python Reddit API Wrapper PRAW, an acronym for "Python Reddit API Wrapper", is a Python package that allows for simple access to Reddit's AP

3k Dec 29, 2022

A Python Instagram Scraper for Downloading Profile's Posts, stories, ProfilePic and See the Details of Particular Instagram Profile.

✔ ✔ InstAstra ⚡ ⚡ ⁜ Description ~ A Python Instagram Scraper for Downloading Profile's Posts, stories, ProfilePic and See the Details of Particular In

12 Jun 23, 2022

Python API wrapper around Trello's API

A wrapper around the Trello API written in Python. Each Trello object is represented by a corresponding Python object. The attributes of these objects

904 Jan 2, 2023

Async ready API wrapper for Revolt API written in Python.

Mutiny Async ready API wrapper for Revolt API written in Python. Installation Python 3.9 or higher is required To install the library, you can just ru

16 Mar 29, 2022

A Python API wrapper for the Twitter API!

PyTweet PyTweet is an api wrapper made for twitter using twitter's api version 2! Installation Windows py3 -m pip install PyTweet Linux python -m pip

1 Nov 19, 2022

Python API wrapper library for Convex Value API

convex-value-python Python API wrapper library for Convex Value API. Further Links: Convex Value homepage @ConvexValue on Twitter JB on Twitter Authen

2 May 11, 2022

This an API wrapper library for the OpenSea API written in Python 3.

OpenSea NFT API Python 3 wrapper This an API wrapper library for the OpenSea API written in Python 3. The library provides a simplified interface to f

159 Dec 26, 2022

YARSAW is an Async Python API Wrapper for the Random Stuff API.

Yet Another Random Stuff API Wrapper - YARSAW YARSAW is an Async Python API Wrapper for the Random Stuff API. This module makes it simpler for you to

6 Mar 27, 2022

EpikCord.py - This is an API Wrapper for Discord's API for Python

EpikCord.py - This is an API Wrapper for Discord's API for Python! We've decided not to fork discord.py and start completely from scratch for a new, better structuring system!

28 Oct 10, 2022

Comments

ZeroDivisionError
您好，我在尝试批量下载的时候，发现当查询结果为0个的时候，会出现 ZeroDivisionError: integer division or modulo by zero 的错误，查询trackback，错误源头在utils.py文件的第4行：portion = len(lst) //n的位置。

除此之外，我还发现当查询以下物种名称的时候也会出现同样的error。我的查询时间范围是从1971-01-01：

['Maleo', 'Moluccan Megapode', 'Nicobar Megapode', 'Long-billed Partridge', 'Black Partridge', 'Udzungwa Forest Partridge', 'Rubeho Forest Partridge', "Roll's Partridge", 'Sumatran Partridge', 'Grey-breasted Partridge', 'Red-billed Partridge', 'Chestnut-necklaced Partridge', 'Crestless Fireback', 'Crested Fireback', 'Siamese Fireback', 'Great Argus', 'Madagascan Pochard']

以下这个list是对应以上的common_name的查询结果数量

[7, 6, 2, 18, 3, 6, 34, 10, 1, 31, 8, 35, 1, 17, 24, 83, 5]

期待回复，感谢！
small fix
opened by ZhongJunhong 1
Something about Download Timeout
您好，我在下载的时候发现一些音频文件在某些情况下（可能是网络的问题）下载时间过长，而且进度条一直停滞不前，有时甚至停滞十分钟到二十分钟不等，有时候我只能手动中断使之进入下一个循环。

后来我在stackoverflow上找到一个非常非常简单的解决方案，就是在执行请求的代码前加入socket.setdefaulttimeout(30)，控制socket打开的时间，比如此处我设置为30秒。

socket.setdefaulttimeout(30) q.retrieve_recordings(multiprocess=True, nproc=10, attempts=10, outdir="/mnt/database/xcdata/")

同时我加入一些其他设置，让代码在触发socket.timeout错误后，可以将对应的请求参数记录下来，并继续执行下一个循环。由此可以避免下载停滞的问题。当我完成所有循环以后，会将socket打开的时间再增加（比如增加到120秒），对先前记录下的请求参数再次执行。由此循环多次以将所有参数请求完毕。

https://stackoverflow.com/questions/32763720/timeout-a-file-download-with-python-urllib

希望能帮助到大家。

Hello, when I was downloading, I found that the download time of some audio files was too long under certain circumstances (may be a problem with the network), and the progress bar has been stagnant, sometimes even for ten to twenty minutes, there are Sometimes I can only manually interrupt it to enter the next cycle.

Later, I found a very, very simple solution on stackoverflow, which is to add socket.setdefaulttimeout(30) before executing the requested code to control the opening time of the socket. For example, I set it to 30 seconds here.

socket.setdefaulttimeout(30) q.retrieve_recordings(multiprocess=True, nproc=10, attempts=10, outdir="/mnt/database/xcdata/")

At the same time, I added some other settings so that after the code triggers the socket.timeout error, it can record the corresponding request parameters and continue to execute the next loop. This can avoid the problem of download stagnation. When I finish all the loops, I will increase the socket opening time (for example, to 120 seconds), and execute again on the previously recorded request parameters. Loop for multiple times to complete the request for all parameters.

https://stackoverflow.com/questions/32763720/timeout-a-file-download-with-python-urllib

I Hope that can help everyone.
opened by ZhongJunhong 3

Releases(v0.0.4)

v0.0.4(May 13, 2022)
Support Query by bird name.

Automatically cut inessential processes in query traffic.

Optimized query assignment strategy in recording retrieval.

Source code(tar.gz)
Source code(zip)

Owner

_zza

Computational (Para)linguistics | Multimodal Analysis | Founder of @Presento | Previously @ByteDance AI-lab Shanghai

GitHub Repository

Github-Checker - Simple Tool To Check If Github User Available Or Not

Github Checker Simple Tool To Check If Github User Available Or Not Socials: Lan

7 Jan 30, 2022

Get some python in google cloud functions

[NOTE]: This is a highly experimental (and proof of concept) library so do not expect all python packages to work flawlessly. Also, cloud functions ar

200 Nov 24, 2022

Jupyter notebooks and AWS CloudFormation template to show how Hudi, Iceberg, and Delta Lake work

Modern Data Lake Storage Layers This repository contains supporting assets for my research in modern Data Lake storage layers like Apache Hudi, Apache

25 Oct 31, 2022

A Telegram bot to download posts, videos, reels, IGTV and a user profile picture from Instagram!

Telegram Bot A telegram bot to download media from Instagram! No API Key or Login Needed! Requirements You must have python installed (of course) You

2 Apr 10, 2022

LEC_Ditto is a bot that tracks the follows and unfollows of Twitter accounts

✨ LEC_Ditto ✨ I'm Ditto, and I'm a bot 🤖 . Getting Started | Installation | Usage Getting Started LEC_Ditto is a bot that tracks the follows and unfo

2 Mar 30, 2022

PepeSniper is an open-source Discord Nitro auto claimer/redeemer made in python.

PepeSniper is an open-source Discord Nitro auto claimer made in python. It sure as hell is not the fastest sniper out there but it gets the job done in a timely and stable manner. It also supports ho

1 Dec 22, 2021

A way to export your saved reddit posts to a Notion table.

reddit-saved-to-notion A way to export your saved reddit posts and comments to a Notion table.Uses notion-sdk-py and praw for interacting with Notion

19 Sep 12, 2022

Widevine CDM API

wvproxy Widevine CDM API Setup Install Python 3.6 or newer and Poetry Install Python package dependencies using poetry install Activate the virtual en

24 Dec 09, 2022

Automatically Forward files from groups to channel & FSub

Backup & ForceSub Automatically Forward files from groups to channel & Do force sub on members Variables API_ID : Get from my.telegram.org API_HASH :

7 Nov 06, 2022

Savecontentbot - Telegram Save Content Bot With Same more Features

Save Restricted Content Bot A simple telegram bot to save restricted content wit

3 Jan 26, 2022

Python client for Arista eAPI

Arista eAPI Python Library The Python library for Arista's eAPI command API implementation provides a client API work using eAPI and communicating wit

124 Nov 23, 2022

veez music bot is a telegram music bot project, allow you to play music on voice chat group telegram.

🎶 Veez Music Bot Music bot for playing music on telegram voice chat group. Requirements 📝 FFmpeg NodeJS nodesource.com Python 3.7+ PyTgCalls 🧪 Get

143 Jun 19, 2022

Using GNU Radio and HackRF One to Receive, Analyze and Send ASK/OOK signals

play_with_ask NIS-8016 Lab A code: Recv.grc/py: Receive signals and match with ASK button using HackRF and GNU radio. I use AM demod block(can also in

1 Jul 04, 2022

Discord Bot that leverages the idea of nested containers using podman, runs untrusted user input, executes Quantum Circuits, allows users to refer to the Qiskit Documentation, and provides the ability to search questions on the Quantum Computing StackExchange.

Discord Bot that leverages the idea of nested containers using podman, runs untrusted user input, executes Quantum Circuits, allows users to refer to the Qiskit Documentation, and provides the abilit

23 Oct 18, 2022

The most Advanced yet simple Multi Cloud tool to transfer Your Data from any cloud to any cloud remotely based on Rclone.⚡

Multi Cloud Transfer (Advanced!) 🔥 1.Setup and Start using Rclone on Google Colab and Create/Edit/View and delete your Rclone config file and keep th

162 Jan 08, 2023

A complete Python application to automatize the process of uploading files to Amazon S3

Upload files or folders (even with subfolders) to Amazon S3 in a totally automatized way taking advantage of: Amazon S3 Multipart Upload: The uploaded

1 Nov 20, 2021

A Python Library to interface with LinkedIn API, OAuth and JSON responses

#Overview Here's another library based on the LinkedIn API, OAuth and JSON responses. Hope this documentation explains everything you need to get star

69 Dec 11, 2022

Crypto trading bot that detects surges in the bitcoin price and executes trades.

The bot will be trading Bitcoin automatically if the price has increased by more than 3% in the last 10 minutes. We will have a stop loss of 5% and t

164 Oct 20, 2022

可基于【腾讯云函数】/【GitHub Actions】/【Docker】的每日签到脚本（支持多账号使用）签到列表: ｜爱奇艺｜全民K歌｜腾讯视频｜有道云笔记｜网易云音乐｜一加手机社区官方论坛｜百度贴吧｜Bilibili｜V2EX｜咔叽网单｜什么值得买｜AcFun｜天翼云盘｜WPS｜吾爱破解｜芒果TV｜联通营业厅｜Fa米家｜小米运动｜百度搜索资源平台｜每日天气预报｜每日一句｜哔咔漫画｜和彩云｜智友邦｜微博｜CSDN｜王者营地｜

每日签到集合基于【腾讯云函数】/【GitHub Actions】/【Docker】的每日签到脚本支持多账号使用特别声明: 本仓库发布的脚本及其中涉及的任何解锁和解密分析脚本，仅用于测试和学习研究，禁止用于商业用途，不能保证其合法性，准确性，完整性和有效性，请根据情况自行判断。

87 Nov 12, 2022

IMDb + Auto + Unlimited Filter BoT

Telegram Movie Bot Features Auto Filter Manuel Filter IMDB Admin Commands Broadcast Index IMDB search Inline Search Random pics ids and User info Stat

1 Dec 03, 2021