ASF Sentinel-1 Metadata Download tool

Related tags

Downloaderasfsmd
Overview

ASF Sentinel-1 Metadata Download tool

Copyright: 2021-2022 Antonio Valentino

Small Python tool (asfsmd) that allows to download XML files containing Sentinel-1 products metadata from the ASF archive.

Sentinel-1 products are stored in the ASF arcive as ZIP files that are quite large because they comntain both the products annotations and the binary image data.

The asfsmd tool is able to retrieve only the relatively samll annotation files (in XML format) without downloading the entire ZIP archive.

asfsmd exploits the httpio Python package for reading HTTP resources as random-access file-like objects. In order to do it the remote server must support the Range header.

This approach allows to open the ZIP archive remotely, inspects contents, and download only the pieces of data that are actually necessary to the user.

Performnces of this approach are quite poor but, in the specific case of Sentinel-1 products, the entire process results to be faster than downloading the entire ZIP archive and extracting only annotation files.

Command Line Interface

$ python3 asfsmd.py --help

usage: asfsmd [-h] [--version]
              [--loglevel {DEBUG,INFO,WARNING,ERROR,CRITICAL}]
              [-q] [-v] [-d] [-f] [-o OUTDIR] [-u USERNAME] [-p PASSWORD]
              INPUT [INPUT ...]

ASF Sentinel-1 Metadata Download tool. Small Python tool (`asfsmd`) that
allows to download XML files containing Sentinel-1 products metadata from
the ASF archive. Sentinel-1 products are stored in the ASF arcive as ZIP
files that are quite large because they comntain both the products
annotations and the binary image data. The `asfsmd` tool is able to
retrieve only the relatively samll annotation files (in XML format) without
downloading the entire ZIP archive.

positional arguments:
  INPUT                 Sentinel-1 product name(s). If the '-f' flag is set
                        then the argument is interpreted as the filename
                        containing the list of products.

options:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  --loglevel {DEBUG,INFO,WARNING,ERROR,CRITICAL}
                        logging level (default: WARNING)
  -q, --quiet           suppress standard output messages, only errors are
                        printed to screen
  -v, --verbose         print verbose output messages
  -d, --debug           print debug messages
  -f, --file-list       read the list of products form file. The file is
                        expected to contain one product name per line.
  -o OUTDIR, --outdir OUTDIR
                        path of the output directory (default='.')
  -u USERNAME, --username USERNAME
                        username for ASF authentication. If not provided
                        the tool attempts to retireve the authentication
                        parameters for the user's '.netrc' file looking
                        for the host 'urs.earthdata.nasa.gov'
  -p PASSWORD, --password PASSWORD
                        password for ASF authentication. If not provided
                        the tool attempts to retireve the authentication
                        parameters for the user's '.netrc' file looking
                        for the host 'urs.earthdata.nasa.gov'
  --block-size BLOCK_SIZE
                        httpio block size in bytes (default: 1024)

License

The asfsmd package is distributed under the MIT License.

You might also like...
Tool to download Netflix in 4k

Netflix-4K-Script Tool to download Netflix in 4k You will need to get a L1 CDM that is whitelsited with Netflix CDM In this script are downgraded

A cli tool to download purchased products from the DLsite.

dlsite-downloader A cli tool to download purchased products from the DLsite. How can I use? This program runs with configurations defined at settings.

Tool To download 4KHDR DV SDR from AppleTV
Tool To download 4KHDR DV SDR from AppleTV

# APPLE-TV 4K Downloader Tool To download 4K HDR DV SDR from AppleTV Hello Fellow Developers/ ! Hi! My name is WVDUMP. I am Leaking the scripts to

The tool allows to download a list of tiktok sounds

dependencies: pip install requests how to use LAUCH THE PROGRAM file (option f)

Tool To download Amazon 4k SDR HDR 1080, CDM IS Not Included

WV-AMZN-4K-RIPPER Tool To download Amazon 4k SDR HDR 1080, CDM IS Not Included For CDM You can Mail :- [email protected] Note : CDM is not free L

Tool To download - Amazon - Netflix- Disney+ - VideoLand - Boomerang - RTE.ie
Tool To download - Amazon - Netflix- Disney+ - VideoLand - Boomerang - RTE.ie

vinetrimmer Widevine Decryption Script for Python Modules Amazon Netflix (with [email protected] support) Disney+ VideoLand Boomerang RTE.ie Hello Fellow Dev

A modern CLI to download animes automatically from Twist
A modern CLI to download animes automatically from Twist

Kurby Kurby is a nice and simple CLI that use Twist website, and their huge collection to download animes for free and automatically Animes from Twist

Download courses from khanacademy.org

khan-dl A python script to download courses from Khan Academy using youtube-dl and beautifulsoup4.

1Fichier Download Manager.
1Fichier Download Manager.

1fichier-dl 1Fichier Download Manager. Features ⭐ Manage your downloads ⭐ Bypass time limits Credits All icons, including the app icon, were provided

Comments
  • Consider alternative backends to httpio

    Consider alternative backends to httpio

    The httpio package is very small seems to fit quite well the needs of this project but:

    • it seems to be a not-active project (last commit 2019)
    • it is necessary to have a specialisation of the httpio.SyncHTTPIOFile class to be able to use this package in asfsmd

    Other alternatives exists that could replace httpio as shall be assessed:

    • [X] RemoteZip

      Very interesting, it can be used out of the box and allows to simplify the codebase quite a lot. The performance to get the list of files included in the ZIP archive seems to be better (x2) than the ones that one can get with httpio but the test consisting in the download of annotations of 5 S1 products seems to be 12% slower than httpio.

    • [X] pyremotezip

      Not maintained upstream, Python2 only.

    • [X] smart-open

      Interesting project bur slower (x3-x4) than httpio to retrieve the list of files contained in the ZIP archive.

    • [X] fsspec

      Extremely interesting and feature complete. Used also in Dask. Direct use of the ZipFileSystem is very slow. The use of the HttpFileSystem + zipfile from the Python stdlib seems to work fine, but it is 33% slower than httpio

    opened by avalentino 2
  • Add filter for polarization and additional calibration/noise/rfi downloads

    Add filter for polarization and additional calibration/noise/rfi downloads

    This adds two functions:

    1. Filter the download by polarization (e.g. so you can grab only 3 'vv' files instead of the 6 for vv/vh)
    2. Download the calibration, noise, and RFI xml files

    I also made a separate branch to further filter to a single subswath by checking iw{number}, but these are the additions I've already found useful when testing https://github.com/opera-adt/s1-reader

    opened by scottstanie 1
Releases(v1.3.0)
  • v1.3.0(Dec 18, 2022)

  • v1.2.0(Dec 5, 2022)

    • Refactoring to convert the `asfsmd.py`` module into a package.
    • Support multiple backends for remote file access: httpio, fsspec, remotezip. The httpio based implementation seems to be slightly faster w.r.t. the other ones.
    • Fix issue with the management of default values for the make_patterns function.
    • Improve the management of the download of large files (chunking and progress).
    Source code(tar.gz)
    Source code(zip)
  • v1.1.0(Dec 3, 2022)

    • Now it is possible to customize the selection of files to be downloaded. Beyond the manifest and the annotation files, now it is also possible to download:

      • calibration annotations
      • noise annotations
      • rfi annotations
      • measurement files

      Moreover now it is possible to select a specific beams or polarizations. Patch developed by @scottstanie and @avalentino.

    Source code(tar.gz)
    Source code(zip)
  • v1.0.0(Dec 3, 2022)

Owner
Antonio Valentino
Antonio Valentino
In this repository you will find the test carried out to enter, as a python developer, the company Keeper Solutions.

Bookmarks In this repository you will find the test carried out to enter, as a python developer, the company Keeper Solutions. First it is necessary t

0 Jan 12, 2022
Simple avogadr.io batch downloader python script

Simple avogadr.io batch downloader python script

2 Jan 19, 2022
The sole purpose of this script is to download any NFT collection from OpenSea

OpenSea NFT Stealer The sole purpose of this script is to download any NFT collection from OpenSea. Setup Prerequisites: Python 3 Python requests libr

Phillip 9 Sep 04, 2022
Vinetrimmer-DRM-TOOL - Widevine DRM downloader and decrypter for AMZN|NF|STAN And all

🍃 ✂️ Vinetrimmer Widevine DRM downloader and decrypter. Thanks to wvleaks for t

Vlad Tănăsescu 20 Jan 13, 2022
A script that downloads YouTube videos/audio

YouTube-Downloader A script that downloads YouTube videos/audio from youtube. Usage Download the script by executing the following in your terminal :

Debayan Sarkar 2 Jan 04, 2022
Youtube videos and channels scraper python wrapper!

YouTubeCrawle Wrapper for python Why This wrapper? This is wrapper is not limited to videos only it can scrape both channel and videos seperately ;D

Kei 16 Aug 08, 2022
A cli tool to download purchased products from the DLsite.

dlsite-downloader A cli tool to download purchased products from the DLsite. How can I use? This program runs with configurations defined at settings.

AcrylicShrimp 9 Dec 23, 2022
Make YouTube videos tasks in Todoist faster and time efficient!

Youtubist Basically fork of yt-dlp python module to my needs. You can paste playlist or channel link on the YouTube. It will automatically format to s

Konrad Konieczny 1 Dec 04, 2022
Download YouTube videos/music and images in MP4, JPG with this tool.

ABOUT THE TOOL Download YouTube videos, music and images in MP4, JPG with this tool, with an easy to understand interface. This tool works with both,

TrollSkull 5 Jan 02, 2023
Throttle qBittorrent on Plex stream Start/Stop

Dependencies Python 3.6+ 'qbittorrent-api' Python Library Tautulli Script Setup Edit qbittorrent_throttle.py and set qBittorrent username, password an

6 Sep 24, 2022
Implementation of Cross-category Video Highlight Detection via Set-based Learning (ICCV 2021).

Cross-category Video Highlight Detection via Set-based Learning Introduction This project is an implementation of ``Cross-category Video Highlight Det

Minghao (Alan) Xu 49 Dec 17, 2022
Simple Youtube Video Downloader

Simple Youtube Video Downloader Download Youtube video using link and Will output result in D:/ (You can change the path in main.py file) Installation

Hansen Gianto 1 Oct 28, 2021
lo2: Simple youtube-dl web frontend

Simple youtube-dl web frontend

Denis Volk 22 Jun 03, 2022
Shit-fetch - Shitpost fetcher (downloader)

shit-fetch Download shitpost (random) from https://random-shitpost.com/ Usage ./shitfetch.py --nsfw (true/false) --output ~/Downloads (default : ./)

Pinokaille 1 Jan 02, 2022
Download all games from a public Itch.io Game Jam

Itch Jam Downloader Downloads all games from a public Itch.io Game Jam. What you'll need: Python 3.8+ pip install -r requirements.txt For site mirrori

Dragoon Aethis 19 Dec 07, 2022
命令行版本的HLS/DASH流下载器,支持标准AES-128-CBC解密

XstreamDL-CLI 基于python 3.7.4+的,命令行版本的,HLS/DASH流下载器,支持标准AES-128-CBC解密 使用 首先安装必要的库

xhlove 239 Dec 31, 2022
Python youtube playlist downloader

Youtube-Playlist-Downloader-python 👍 This program is a simple Youtube playlist downloader where you input the playlist link, and then the desired pat

Pepczenko 2 Dec 25, 2021
A python module to download ISO Standards

ISO Standards Downloader A python module to download ISO Standards from https://standards.iso.org/iso-iec/ Report Bug · Request Feature Table of conte

Daniel 1 Dec 29, 2021
A youtube downloader, built with flask yt-dlp

Built With Python Flask - The Python micro framework for building web applications. yt-dlp - A youtube-dl fork with additional features and fixes

Abhijith N T 13 Dec 17, 2022