Read and write rasters in parallel using Rasterio and Dask

Related tags

Geolocationpythongdaldaskrasterio
Overview

dask-rasterio

Build Status codecov Join the chat at https://gitter.im/dymaxionlabs/dask-rasterio

dask-rasterio provides some methods for reading and writing rasters in parallel using Rasterio and Dask arrays.

Usage

Read a multiband raster

>>> from dask_rasterio import read_raster

>>> array = read_raster('tests/data/RGB.byte.tif')
>>> array
dask.array<stack, shape=(3, 718, 791), dtype=uint8, chunksize=(1, 3, 791)>

>>> array.mean()
dask.array<mean_agg-aggregate, shape=(), dtype=float64, chunksize=()>
>>> array.mean().compute()
40.858976977533935

Read a single band from a raster

>>> from dask_rasterio import read_raster

>>> array = read_raster('tests/data/RGB.byte.tif', band=3)
>>> array
dask.array<raster, shape=(718, 791), dtype=uint8, chunksize=(3, 791)>

Write a singleband or multiband raster

>>> from dask_rasterio import read_raster, write_raster

>>> array = read_raster('tests/data/RGB.byte.tif')

>>> new_array = array & (array > 100)
>>> new_array
dask.array<and_, shape=(3, 718, 791), dtype=uint8, chunksize=(1, 3, 791)>

>>> prof = ... # reuse profile from tests/data/RGB.byte.tif...
>>> write_raster('processed_image.tif', new_array, **prof)

Chunk size

Both read_raster and write_raster accept a block_size argument that acts as a multiplier to the block size of rasters. The default value is 1, which means the dask array chunk size will be the same as the block size of the raster file. You will have to adjust this value depending on the specification of your machine (how much memory do you have, and the block size of the raster).

Install

Install with pip:

pip install dask-rasterio

Development

This project is managed by Poetry. If you do not have it installed, please refer to Poetry instructions.

Now, clone the repository and run poetry install. This will create a virtual environment and install all required packages there.

Run poetry run pytest to run all tests.

Run poetry build to build package on dist/.

Issue tracker

Please report any bugs and enhancement ideas using the GitHub issue tracker:

https://github.com/dymaxionlabs/dask-rasterio/issues

Feel free to also ask questions on our Gitter channel, or by email.

Help wanted

Any help in testing, development, documentation and other tasks is highly appreciated and useful to the project.

For more details, see the file CONTRIBUTING.md.

License

Source code is released under a BSD-2 license. Please refer to LICENSE.md for more information.

You might also like...
Download and process satellite imagery in Python using Sentinel Hub services.

Description The sentinelhub Python package allows users to make OGC (WMS and WCS) web requests to download and process satellite images within your Py

Sentinel Hub 659 Dec 23, 2022
An API built to format given addresses using Python and Flask.
An API built to format given addresses using Python and Flask.

An API built to format given addresses using Python and Flask. About The API returns properly formatted data, i.e. removing duplicate fields, distingu

null 1 Feb 27, 2022
This is a simple python code to get IP address and its location using python

IP address & Location finder @DEV/ED : Pavan Ananth Sharma Dependencies: ip2geotools Note: use pip install ip2geotools to install this in your termin

Pavan Ananth Sharma 2 Jul 5, 2022
A package built to support working with spatial data using open source python

EarthPy EarthPy makes it easier to plot and manipulate spatial data in Python. Why EarthPy? Python is a generic programming language designed to suppo

Earth Lab 414 Dec 23, 2022
Using SQLAlchemy with spatial databases

GeoAlchemy GIS Support for SQLAlchemy. Introduction GeoAlchemy is an extension of SQLAlchemy. It provides support for Geospatial data types at the ORM

null 109 Dec 1, 2022
Solving the Traveling Salesman Problem using Self-Organizing Maps
Solving the Traveling Salesman Problem using Self-Organizing Maps

Solving the Traveling Salesman Problem using Self-Organizing Maps This repository contains an implementation of a Self Organizing Map that can be used

Diego Vicente 3.1k Dec 31, 2022
Example of animated maps in matplotlib + geopandas using entire time series of congressional district maps from UCLA archive. rendered, interactive version below
Example of animated maps in matplotlib + geopandas using entire time series of congressional district maps from UCLA archive. rendered, interactive version below

Example of animated maps in matplotlib + geopandas using entire time series of congressional district maps from UCLA archive. rendered, interactive version below

Apoorva Lal 5 May 18, 2022
Hapi is a Python library for building Conceptual Distributed Model using HBV96 lumped model & Muskingum routing method
Hapi is a Python library for building Conceptual Distributed Model using HBV96 lumped model & Muskingum routing method

Current build status All platforms: Current release info Name Downloads Version Platforms Hapi - Hydrological library for Python Hapi is an open-sourc

Mostafa Farrag 15 Dec 26, 2022
Daily social mapping project in November 2021. Maps made using PyGMT whenever possible.
Daily social mapping project in November 2021. Maps made using PyGMT whenever possible.

Daily social mapping project in November 2021. Maps made using PyGMT whenever possible.

Wei Ji 20 Nov 24, 2022
Comments
  • the time to write a dask array in tif is too long?

    the time to write a dask array in tif is too long?

    I have a image which have 40000cols and 40000 rows ,while I use data = read_raster(filename, band=1), and the use witer_raster(out_filename, data), it need about one minutes, I want to know can it be quickly

    opened by DeZhao-Zhang 2
  • Add a Gitter chat badge to README.md

    Add a Gitter chat badge to README.md

    dymaxionlabs/dask-rasterio now has a Chat Room on Gitter

    @munshkr has just created a chat room. You can visit it here: https://gitter.im/dymaxionlabs/dask-rasterio.

    This pull-request adds this badge to your README.md:

    Gitter

    If my aim is a little off, please let me know.

    Happy chatting.

    PS: Click here if you would prefer not to receive automatic pull-requests from Gitter in future.

    opened by gitter-badger 1
  • Does dask-rasterio support masked array?

    Does dask-rasterio support masked array?

    I'm working with dask masked array, and was wondering what would be the translation of these lines?

        import rasterio
    with rasterio.open(inputFile) as source:
        # this is a 3D numpy array, with dimensions [band, row, col]
        src_array = source.read(masked=True)

    Thank you for your cool lib!

    opened by Becheler 0
  • TypeError: self._hds cannot be converted to a Python object for pickling

    TypeError: self._hds cannot be converted to a Python object for pickling

    Seems that rasterio's _hds object is no more serializable

    distributed.protocol.pickle - INFO - Failed to serialize ("('filled-2f9fe0560be0502eda038fa941309294', 0, 0)", <dask_rasterio.write.RasterioDataset object at 0x7f8f9deac828>, (slice(0, 748, None), slice(0, 22415, None)), <unlocked _thread.lock object at 0x7f8f9cb2af58>, False). Exception: self._hds cannot be converted to a Python object for pickling
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~/miniconda3/envs/jupyter/lib/python3.6/site-packages/distributed/protocol/pickle.py in dumps(x)
     37     try:
---> 38         result = pickle.dumps(x, protocol=pickle.HIGHEST_PROTOCOL)
     39         if len(result) < 1000:

~/miniconda3/envs/jupyter/lib/python3.6/site-packages/rasterio/_io.cpython-36m-x86_64-linux-gnu.so in rasterio._io.DatasetWriterBase.__reduce_cython__()

TypeError: self._hds cannot be converted to a Python object for pickling
    opened by arkanoid87 3
Releases(0.2.1)
Owner
Dymaxion Labs
Creating new value from geospatial imagery with deep learning
Dymaxion Labs
GitHub Repository
Python script to locate mobile number

Python script to locate mobile number How to use this script run the command to install the required libraries pip install -r requirements.txt run the

Shekhar Gupta 8 Oct 10, 2022
Platform for building statistical models of cities and regions

UrbanSim UrbanSim is a platform for building statistical models of cities and regions. These models help forecast long-range patterns in real estate d

Urban Data Science Toolkit 419 Dec 30, 2022
prettymaps - A minimal Python library to draw customized maps from OpenStreetMap data.

A small set of Python functions to draw pretty maps from OpenStreetMap data. Based on osmnx, matplotlib and shapely libraries.

Marcelo de Oliveira Rosa Prates 9k Jan 08, 2023
Tile Map Service and OGC Tiles API for QGIS Server

Tiles API Add tiles API to QGIS Server Tiles Map Service API OGC Tiles API Tile Map Service API - TMS The TMS API provides these URLs: /tms/? to get i

3Liz 6 Dec 01, 2021
A Python tool to display geolocation information in the traceroute.

IP2Trace Python IP2Trace Python is a Python tool allowing user to get IP address information such as country, region, city, latitude, longitude, zip c

IP2Location 22 Jan 08, 2023
Earthengine-py-notebooks - A collection of 360+ Jupyter Python notebook examples for using Google Earth Engine with interactive mapping

earthengine-py-notebooks A collection of 360+ Jupyter Python notebook examples for using Google Earth Engine with interactive mapping Contact: Qiushen

Qiusheng Wu 1.1k Dec 29, 2022
Tools for the extraction of OpenStreetMap street network data

OSMnet Tools for the extraction of OpenStreetMap (OSM) street network data. Intended to be used in tandem with Pandana and UrbanAccess libraries to ex

Urban Data Science Toolkit 47 Sep 21, 2022
A ready-to-use curated list of Spectral Indices for Remote Sensing applications.

A ready-to-use curated list of Spectral Indices for Remote Sensing applications. GitHub: https://github.com/davemlz/awesome-ee-spectral-indices Docume

David Montero Loaiza 488 Jan 03, 2023
Hapi is a Python library for building Conceptual Distributed Model using HBV96 lumped model & Muskingum routing method

Current build status All platforms: Current release info Name Downloads Version Platforms Hapi - Hydrological library for Python Hapi is an open-sourc

Mostafa Farrag 15 Dec 26, 2022
python toolbox for visualizing geographical data and making maps

geoplotlib is a python toolbox for visualizing geographical data and making maps data = read_csv('data/bus.csv') geoplotlib.dot(data) geoplotlib.show(

Andrea Cuttone 976 Dec 11, 2022
Global topography (referenced to sea-level) in a 10 arcminute resolution grid

Earth - Topography grid at 10 arc-minute resolution Global 10 arc-minute resolution grids of topography (ETOPO1 ice-surface) referenced to mean sea-le

Fatiando a Terra Datasets 1 Jan 20, 2022
Yet Another Time Series Model

Yet Another Timeseries Model (YATSM) master v0.6.x-maintenance Build Coverage Docs DOI | About Yet Another Timeseries Model (YATSM) is a Python packag

Chris Holden 60 Sep 13, 2022
Asynchronous Client for the worlds fastest in-memory geo-database Tile38

This is an asynchonous Python client for Tile38 that allows for fast and easy interaction with the worlds fastest in-memory geodatabase Tile38.

Ben 53 Dec 29, 2022
Enable geospatial data mining through Google Earth Engine in Grasshopper 3D, via its most recent Hops component.

AALU_Geo Mining This repository is produced for a masterclass at the Architectural Association Landscape Urbanism programme. Requirements Rhinoceros (

4 Nov 16, 2022
Constraint-based geometry sketcher for blender

Geometry Sketcher Constraint-based sketcher addon for Blender that allows to create precise 2d shapes by defining a set of geometric constraints like

1.7k Jan 02, 2023
Simple CLI for Google Earth Engine Uploads

geeup: Simple CLI for Earth Engine Uploads with Selenium Support This tool came of the simple need to handle batch uploads of both image assets to col

Samapriya Roy 79 Nov 26, 2022
Calculate & view the trajectory and live position of any earth-orbiting satellite

satellite-visualization A cross-platform application to calculate & view the trajectory and live position of any earth-orbiting satellite in 3D. This

Space Technology and Astronomy Cell - Open Source Society 3 Jan 08, 2022
An API built to format given addresses using Python and Flask.

An API built to format given addresses using Python and Flask. About The API returns properly formatted data, i.e. removing duplicate fields, distingu

1 Feb 27, 2022
This repository contains the scripts to derivate the ENU and ECEF coordinates from the longitude, latitude, and altitude values encoded in the NAD83 coordinates.

This repository contains the scripts to derivate the ENU and ECEF coordinates from the longitude, latitude, and altitude values encoded in the NAD83 coordinates.

Luigi Cruz 1 Feb 07, 2022
Obtain a GNSS position fix from an 11-millisecond raw GNSS signal snapshot

Obtain a GNSS position fix from an 11-millisecond raw GNSS signal snapshot without any prior knowledge about the position of the receiver and only coarse knowledge about the time.

Jonas Beuchert 2 Nov 17, 2022