Cached file system for online resources in Python

Overview

Minato

Actions Status Python version License pypi version

Cache & file system for online resources in Python

Features

Minato enables you to:

  • Download & cache online recsources
    • minato supports the following protocols: HTTP(S) / AWS S3 / Google Cloud Storage
    • You can manage cached files via command line interface
  • Automatically update cached files based on ETag
    • minato downloads new versions if available when you access cached files
  • Open online files super easily
    • By using minato.open, you can read/write online resources like the built-in open method

Installation

pip install minato[all]

Usage

Python

import minato

# Read / write files on online storage
with minato.open("s3://your_bucket/path/to/file", "w") as f:
    f.write("Create a new file on AWS S3!")

# Cache & manage online resources in local storage
local_filename = minato.cached_path("http://example.com/path/to/archive.zip!inner/path/to/file")

CLI

❯ poetry run minato --help
usage: minato

positional arguments:
  {cache,list,remove,update}
    cache               cache remote file and return cached local file path
    list                show list of cached files
    remove              remove cached files
    update              update cached files

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
You might also like...
Exploit Discord's cache system to remote upload payloads on Discord users machines
Exploit Discord's cache system to remote upload payloads on Discord users machines

Exploit Discord's cache system to hide payloads PoC Remote upload embedded payload from image using EOF to Discord users machines through cache. Depen

A generic system for filtering Django QuerySets based on user selections

Django Filter Django-filter is a reusable Django application allowing users to declaratively add dynamic QuerySet filtering from URL parameters. Full

A Django based shop system

django-SHOP Django-SHOP aims to be a the easy, fun and fast e-commerce counterpart to django-CMS. Here you can find the full documentation for django-

Login System Django

Login-System-Django Login System Using Django Tech Used Django Python Html Run Locally Clone project git clone https://link-to-project Get project for

A Blog Management System Built with django
A Blog Management System Built with django

Blog Management System Backend use: Django Features Enhanced Ui

This repository contains django library management system project.

Library Management System Django ** INSTALLATION** First of all install python on your system. Then run pip install -r requirements.txt to required se

Djang Referral System
Djang Referral System

Djang Referral System About | Features | Technologies | Requirements | Starting | License | Author 🎯 About I created django referral system and I wan

Advanced school management system written in Django :)
Advanced school management system written in Django :)

Advanced school management system written in Django :) ⚙️ Config the project First you should make venv for this project. So in the main root of proje

A visual indicator of what environment/system you're using in django

A visual indicator of what environment/system you're using in django

Comments
  • Bump protobuf from 4.21.5 to 4.21.6

    Bump protobuf from 4.21.5 to 4.21.6

    Bumps protobuf from 4.21.5 to 4.21.6.

    Release notes

    Sourced from protobuf's releases.

    Protocol Buffers v3.0.0-beta-4

    Version 3.0.0-beta-4

    General

    • Added a deterministic serialization API for C++. The deterministic serialization guarantees that given a binary, equal messages will be serialized to the same bytes. This allows applications like MapReduce to group equal messages based on the serialized bytes. The deterministic serialization is, however, NOT canonical across languages; it is also unstable across different builds with schema changes due to unknown fields. Users who need canonical serialization, e.g. persistent storage in a canonical form, fingerprinting, etc, should define their own canonicalization specification and implement the serializer using reflection APIs rather than relying on this API.

    • Added OneofOptions. You can now define custom options for oneof groups.

      import "google/protobuf/descriptor.proto";
      extend google.protobuf.OneofOptions {
        optional int32 my_oneof_extension = 12345;
      }
      message Foo {
        oneof oneof_group {
          (my_oneof_extension) = 54321;
          ...
        }
      }
      

    C++ (beta)

    • Introduced a deterministic serialization API in CodedOutputStream::SetSerializationDeterministic(bool). See the notes about deterministic serialization in the General section.
    • Added google::protobuf::Map::swap() to swap two map fields.
    • Fixed a memory leak when calling Reflection::ReleaseMessage() on a message allocated on arena.
    • Improved error reporting when parsing text format protos.
    • JSON
      • Added a new parser option to ignore unknown fields when parsing JSON.
      • Added convenient methods for message to/from JSON conversion.
    • Various performance optimizations.

    Java (beta)

    • File option "java_generate_equals_and_hash" is now deprecated. equals() and hashCode() methods are generated by default.
    • Added a new JSON printer option "omittingInsignificantWhitespace" to produce a more compact JSON output. The printer will pretty-print by default.
    • Updated Java runtime to be compatible with 2.5.0/2.6.1 generated protos.

    Python (beta)

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump ujson from 5.3.0 to 5.4.0

    Bump ujson from 5.3.0 to 5.4.0

    Bumps ujson from 5.3.0 to 5.4.0.

    Release notes

    Sourced from ujson's releases.

    5.4.0

    Added

    Fixed

    Commits
    • 9c20de0 Merge pull request from GHSA-fm67-cv37-96ff
    • b21da40 Fix double free on string decoding if realloc fails
    • 67ec071 Merge pull request #555 from JustAnotherArchivist/fix-decode-surrogates-2
    • bc7bdff Replace wchar_t string decoding implementation with a uint32_t-based one
    • cc70119 Merge pull request #548 from JustAnotherArchivist/arbitrary-ints
    • 4b5cccc Merge pull request #553 from bwoodsend/pypy-ci
    • abe26fc Merge pull request #551 from bwoodsend/bye-bye-travis
    • 3efb5cc Delete old TravisCI workflow and references.
    • 404de1a xfail test_decode_surrogate_characters() on Windows PyPy.
    • f7e66dc Switch to musl docker base images.
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • CVE-2007-4559 Patch

    CVE-2007-4559 Patch

    Patching CVE-2007-4559

    Hi, we are security researchers from the Advanced Research Center at Trellix. We have began a campaign to patch a widespread bug named CVE-2007-4559. CVE-2007-4559 is a 15 year old bug in the Python tarfile package. By using extract() or extractall() on a tarfile object without sanitizing input, a maliciously crafted .tar file could perform a directory path traversal attack. We found at least one unsantized extractall() in your codebase and are providing a patch for you via pull request. The patch essentially checks to see if all tarfile members will be extracted safely and throws an exception otherwise. We encourage you to use this patch or your own solution to secure against CVE-2007-4559. Further technical information about the vulnerability can be found in this blog.

    If you have further questions you may contact us through this projects lead researcher Kasimir Schulz.

    opened by TrellixVulnTeam 0
Releases(v0.10.0)
  • v0.10.0(Aug 11, 2022)

    What's Changed

    • Bump ujson from 5.3.0 to 5.4.0 by @dependabot in https://github.com/altescy/minato/pull/36
    • Table supports full-width characters by @altescy in https://github.com/altescy/minato/pull/37
    • Modify auto-format settings by @altescy in https://github.com/altescy/minato/pull/38
    • Add progress & remove tqdm by @altescy in https://github.com/altescy/minato/pull/39
    • Add disable-progressbar option by @altescy in https://github.com/altescy/minato/pull/40
    • Add interactive selector by @altescy in https://github.com/altescy/minato/pull/41
    • Bump version number to v0.10.0 by @altescy in https://github.com/altescy/minato/pull/42

    New Contributors

    • @dependabot made their first contribution in https://github.com/altescy/minato/pull/36

    Full Changelog: https://github.com/altescy/minato/compare/v0.9.0...v0.10.0

    Source code(tar.gz)
    Source code(zip)
  • v0.9.0(Jul 2, 2022)

    What's Changed

    • Use options for open_file function by @altescy in https://github.com/altescy/minato/pull/33
    • Support Python 3.9 and 3.10 by @altescy in https://github.com/altescy/minato/pull/34
    • Bump version number to v0.9.0 by @altescy in https://github.com/altescy/minato/pull/35

    Full Changelog: https://github.com/altescy/minato/compare/v0.8.2...v0.9.0

    Source code(tar.gz)
    Source code(zip)
  • v0.8.2(Feb 24, 2022)

    What's Changed

    • Modify log level by @altescy in https://github.com/altescy/minato/pull/30
    • Upgrade dependent packages by @altescy in https://github.com/altescy/minato/pull/31
    • Bump version number to v0.8.2 by @altescy in https://github.com/altescy/minato/pull/32

    Full Changelog: https://github.com/altescy/minato/compare/v0.8.1...v0.8.2

    Source code(tar.gz)
    Source code(zip)
  • v0.8.1(Feb 11, 2022)

    What's Changed

    • Fix s3 progress by @altescy in https://github.com/altescy/minato/pull/27
    • Add all command to Makefile by @altescy in https://github.com/altescy/minato/pull/28
    • Bump version number to v0.8.1 by @altescy in https://github.com/altescy/minato/pull/29

    Full Changelog: https://github.com/altescy/minato/compare/v0.8.0...v0.8.1

    Source code(tar.gz)
    Source code(zip)
  • v0.8.0(Feb 8, 2022)

    What's Changed

    • Modify tqdm progress by @altescy in https://github.com/altescy/minato/pull/22
    • Upgrade dependent packages by @altescy in https://github.com/altescy/minato/pull/23
    • Split github workflows into ci and pypi-publish by @altescy in https://github.com/altescy/minato/pull/24
    • Modify type annotations by @altescy in https://github.com/altescy/minato/pull/25
    • Bump version number to v0.8.0 by @altescy in https://github.com/altescy/minato/pull/26

    Full Changelog: https://github.com/altescy/minato/compare/v0.7.0...v0.8.0

    Source code(tar.gz)
    Source code(zip)
  • v0.7.0(Nov 19, 2021)

    • Shrink table column width automatically (#16)
    • Modify S3 download (#17)
    • Move boto3-stubs to dev-dependencies (#18)
    • Add upload method to S3FileSystem (#19)
    • Add upload method to GCSFileSystem (#20)
    • Bump version number to v0.7.0 (#21)
    Source code(tar.gz)
    Source code(zip)
  • v0.6.2(Oct 23, 2021)

  • v0.6.1(Oct 1, 2021)

  • v0.6.0(Sep 30, 2021)

    • Remove quotations of S3 Etags (#2)
    • Modify CI settings (#3)
    • Update dependent packages (#4)
    • Make filesystems optional (#5)
    • Add opening options (#6)
    • Add helps to each command line options (#7)
    • Add feature description to readme (#8)
    • Bump version number to v0.6.0 (#9)
    Source code(tar.gz)
    Source code(zip)
  • v0.5.0(Jul 31, 2021)

  • v0.4.0(Jun 25, 2021)

    • Support file-level expiration
    • Enable automatic cache updating based on HTTP ETag
    • Show download progress of S3 and GCS
    • Deprecate upload/download commands
    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Jun 23, 2021)

Owner
Yasuhiro Yamaguchi
Research Engineer / NLP / ML
Yasuhiro Yamaguchi
Django Audit is a simple Django app that tracks and logs requests to your application.

django-audit Django Audit is a simple Django app that tracks and logs requests to your application. Quick Start Install django-audit pip install dj-au

Oluwafemi Tairu 6 Dec 01, 2022
Code coverage measurement for Python

Coverage.py Code coverage testing for Python. Coverage.py measures code coverage, typically during test execution. It uses the code analysis tools and

Ned Batchelder 2.3k Jan 05, 2023
Use webpack to generate your static bundles without django's staticfiles or opaque wrappers.

django-webpack-loader Use webpack to generate your static bundles without django's staticfiles or opaque wrappers. Django webpack loader consumes the

2.4k Dec 24, 2022
The uncompromising Python code formatter

The Uncompromising Code Formatter “Any color you like.” Black is the uncompromising Python code formatter. By using it, you agree to cede control over

Python Software Foundation 30.7k Jan 03, 2023
This is a basic Todo Application API using Django Rest Framework

Todo Application This is a basic Todo Application API using Django Rest Framework. Todo Section - User can View his previously added todo items, creat

Atharva Parkhe 1 Aug 09, 2022
Yummy Django API, it's the exclusive API used for the e-yummy-ke vue web app

Yummy Django API, it's the exclusive API used for the e-yummy-ke vue web app

Am.Chris_KE 1 Feb 14, 2022
Django-shared-app-isolated-databases-example - Django - Shared App & Isolated Databases

Django - Shared App & Isolated Databases An app that demonstrates the implementa

Ajai Danial 5 Jun 27, 2022
A standalone package to scrape financial data from listed Vietnamese companies via Vietstock

Scrape Financial Data of Vietnamese Listed Companies - Version 2 A standalone package to scrape financial data from listed Vietnamese companies via Vi

Viet Anh (Vincent) Tran 45 Nov 16, 2022
A modern looking portfolio build with Django.

Django Portfolio A portfolio template using html/css/js in the frontend and Django as the backend framework. Cool features: smooth scrolling responsiv

1 Jan 19, 2022
django-tables2 - An app for creating HTML tables

django-tables2 - An app for creating HTML tables django-tables2 simplifies the task of turning sets of data into HTML tables. It has native support fo

Jan Pieter Waagmeester 1.6k Jan 03, 2023
A Minimalistic Modern Django Boilerplate

A Minimalistic Modern Django Boilerplate This boilerplate is mainly for educational purposes. It is meant to be cloned as a starter code for future tu

Jonathan Adly 21 Nov 02, 2022
A simple Django dev environment setup with docker for demo purposes for GalsenDev community

GalsenDEV Docker Demo This is a basic Django dev environment setup with docker and docker-compose for a GalsenDev Meetup. The main purposes was to mak

3 Jul 03, 2021
Django API that scrapes and provides the last news of the city of Carlos Casares by semantic way (RDF format).

"Casares News" API Api that scrapes and provides the last news of the city of Carlos Casares by semantic way (RDF format). Usage Consume the articles

Andrés Milla 6 May 12, 2022
Django Starter is a simple Skeleton to start with a Django project.

Django Starter Template Description Django Starter is a simple Skeleton to start

Numan Ibn Mazid 1 Jan 10, 2022
Duckiter will Automatically dockerize your Django projects.

Duckiter Duckiter will Automatically dockerize your Django projects. Requirements : - python version : python version 3.6 or upper version - OS :

soroush safari 23 Sep 16, 2021
Inject an ID into every log message from a Django request. ASGI compatible, integrates with Sentry, and works with Celery

Django GUID Now with ASGI support! Django GUID attaches a unique correlation ID/request ID to all your log outputs for every request. In other words,

snok 300 Dec 29, 2022
Simple yet powerful and really extendable application for managing a blog within your Django Web site.

Django Blog Zinnia Simple yet powerful and really extendable application for managing a blog within your Django Web site. Zinnia has been made for pub

Julien Fache 2.1k Dec 24, 2022
Django Email Sender

Email-Sender Django Email Sender Installation 1.clone Repository & Install Packages git clone https://github.com/telman03/Email-Sender.git pip install

Telman Gadimov 0 Dec 26, 2021
https://django-storages.readthedocs.io/

Installation Installing from PyPI is as easy as doing: pip install django-storages If you'd prefer to install from source (maybe there is a bugfix in

Josh Schneier 2.3k Jan 06, 2023
Dynamic, database-driven Django forms

Django Dataforms django-dataforms is a wrapper for the Django forms API that lets you dynamically define forms in a database, rather than hard-coding

35 Dec 16, 2022