Async boto3 with Autogenerated Data Classes

Overview

awspydk

Async boto3 with Autogenerated JIT Data Classes


Motivation

This library is forked from an internal project that works with a lot of backend AWS APIs, and I got tired of having to constantly parse the returned responses before being able to work with them for (most of the time), a few seconds. Any API-driven application that uses boto3 tends to suffer from being syncronous only. So this library solves a few major problems:

  • Enables both sync and async from the same client.

  • Client can be called implicitly without needing initialization, i.e. you can directly use AwsClient without needing to initialize.

  • Dynamically generates class functions based on the boto3.client child functions, and allows you to call them. This is useful in repl or ipython environments where type-hints are always helpful, especially when the names are so long.

  • Automatically initializes the aws client if its not initialized from the defaults, simply by calling it.

  • Translates all results into Automatically Generated Dataclasses through lazycls.

    • Can be disabled by setting aws.config.AutoCls = False

Quickstart

pip install --upgrade awspydk
from aws import AwsClient

# Sync Method
buckets = AwsClient.v1.s3_list_buckets(as_cls=True)

# Async Method
buckets = await AwsClient.v1.async_s3_list_buckets(as_cls=True)

"""
Both yield the same results.
The underlying classes are auto-generated from Pydantic BaseModels, so anything you can do with Pydantic Models, you can do with these.

{
    'Buckets': [
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 8, 25, 16, 42, 46, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 2, 17, 54, 56, tzinfo=tzutc()), Name='...',
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 3, 4, 20, 10, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 1, 20, 50, 33, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 2, 4, 2, 28, tzinfo=tzutc()), Name='...')
    ],
    'Owner': AwsS3Owner(DisplayName='...', ID='...')
}
"""

## Change Regions
AwsClient.reset(region='us-west-1')

## Change the defaut clients created
from aws.config import DefaultClients

## Modify to only create ec2 client
DefaultClients = {
    'ec2': 'ec2'
}

## Reset implicitly
AwsClient.reset()


BotoKwargs = {
    'AWS_PROFILE': ...,
}

## Reset Explicitly
AwsClient.reset(clients=DefaultClients, boto_kwargs=BotoKwargs)

Client Defaults

These are found in aws.config

AwsRegion = envToStr('AWS_REGION', 'us-east-1')
AutoCls = envToBool('AWSSDK_AUTOCLS', 'true')

## These are the default clients that will be autogenerated.
## Key is the shorthand, value is the actual AWS API Name in boto3
DefaultClients = {
    'ec2': 'ec2', 
    'ecr': 'ecr', 
    'r53' :'route53', 
    'acm': 'acm',
    'elb': 'elb',
    'elbv2': 'elbv2',
    'asg': 'autoscaling',
    's3': 's3'
}

## These are the default resources that will be autogenerated.
## Key is the shorthand, value is the actual AWS Resource API Name in boto3
DefaultResources = {
    'Ec2': 'ec2',
    'S3': 's3',
    'Iam': 'iam'
}

# These are the default filter args for querying
DefaultFilterArgs = {
    'string_only': True,
    'remove_null': True
}
Owner
Chief Architect @ Growth Engine
A modular dynamical-systems model of Ethereum's validator economics.

CADLabs Ethereum Economic Model A modular dynamical-systems model of Ethereum's validator economics, based on the open-source Python library radCAD, a

CADLabs 104 Jan 03, 2023
Neko is An Anime themed advance Telegram group management bot.

NekoRobot A modular telegram Python bot running on python3 with an sqlalchemy, mongodb database. ╒═══「 Status 」 Maintained Support Group Included Free

Lovely Prince 11 Oct 11, 2022
Automatically scrape all of your artifacts in Genshin Impact.

Genshin Artifact Scraper Automatically scrape all of your artifacts in Genshin Impact. Features: Simple recalibration (2 steps). GUI to select OCR reg

21 Dec 17, 2022
Скрипт, позволяющий импортировать плейлисты из Spotify, а также обычные треклисты в VK музыку.

vk-music-import Программа для переноса плейлистов из Spotify и текстовых треклистов в VK Музыку. Преимущества: Позволяет быстро импортировать плейлист

Mew Forest 32 Nov 23, 2022
Lamblayer: a minimal deployment tool for AWS Lambda layers

lamblayer lamblayer is a minimal deployment tool for AWS Lambda layers. lamblayer does, Create a Layers of built pip-installable python packages. Crea

Yusuke Takahashi 2 Aug 19, 2022
A template that help you getting started with Pycord.

A Pycord Template with some example! Getting Started: Clone this repository using git clone https://github.com/AungS8430/pycord-template.git If you ha

2 Feb 10, 2022
WILSON Cloud Respwnder is a Web Interaction Logger Sending Out Notifications with the ability to serve custom content in order to appropriately respond to client-issued requests.

WILSON Cloud Respwnder What is this? WILSON Cloud Respwnder is a Web Interaction Logger Sending Out Notifications (WILSON) with the ability to serve c

48 Oct 31, 2022
ARKHAM X GOD MULTISPAM BOT

ARKHAM-X-GOD-MULTISPAM-BOT 𝗗𝗘𝗣𝗟𝗢𝗬 𝗨𝗣𝗧𝗢 30 𝗕𝗢𝗧𝗦 𝗜𝗡 𝗔 𝗦𝗜𝗡𝗚𝗟?

ArkhamXGod 2 Jan 08, 2022
The accompanying code for the paper "GMAT: Global Memory Augmentation for Transformers" (Ankit Gupta and Jonathan Berant).

GMAT: Global Memory Augmentation for Transformers This repository contains the accompanying code for the paper: "GMAT: Global Memory Augmentation for

Ankit Gupta 7 Oct 21, 2021
A Pancakeswap v2 trading client (and bot) with limit orders, stop-loss, custom gas strategies, a GUI and much more.

Pancakeswap v2 trading client A Pancakeswap trading client (and bot) with limit orders, stop-loss, custom gas strategies, a GUI and much more. If you

571 Mar 15, 2022
SpautiNoFay - A simple and beautiful music player created with Python

SpautiNoFay A simple and beautiful music player created with Python Why SpautiNo

8 Jan 19, 2022
An advanced Filter Bot with nearly unlimitted filters!

Unlimited Filter Bot ㅤㅤㅤㅤㅤㅤㅤ ㅤㅤㅤㅤㅤㅤㅤ An advanced Filter Bot with nearly unlimitted filters! Features Nearly unlimited filters Supports all type of fil

1 Nov 20, 2021
A Discord bot coded in Python

Perseverance-Bot By Toricane Replit Code | GitHub Code | Discord Server | Website Perseverance is a multi-purpose bot coded in Python. It has moderati

4 Mar 30, 2022
The aim is to contain multiple models for materials discovery under a common interface

Aviary The aviary contains: - roost, - wren, cgcnn. The aim is to contain multiple models for materials discovery under a common interface Environment

Rhys Goodall 20 Jan 06, 2023
GG Dorking is a tool to generate GitHub and Google dorking for pentesters and bug bounty hunters.

GG-Dorking GG Dorking is a python tool to generate GitHub and Google dorking links for pentesters and bug bounty hunters. It will help you to find imp

Eslam Akl 80 Nov 24, 2022
Chronocalc - Calculates the dates and times when the sun or moon is in a given position in the sky

Chronocalc I wrote this script after I was busy updating my article on chronoloc

16 Dec 13, 2022
An advanced crypto trading bot written in Python

Jesse Jesse is an advanced crypto trading framework which aims to simplify researching and defining trading strategies. Why Jesse? In short, Jesse is

Jesse 4.4k Jan 09, 2023
A quick way to verify your Climate Hack.AI (2022) submission locally!

Climate Hack.AI (2022) Submission Validator This repository contains code that allows you to quickly validate your Climate Hack.AI (2022) submission l

Jeremy 3 Mar 03, 2022
Repo-cloner - Script takes user public liked repos and clone it to a local folder

Liked repos cloner Script takes user public liked repos and clone it to a local

Aleksei 2 Jun 18, 2022
Analyzed the data of VISA applicants to build a predictive model to facilitate the process of VISA approvals.

Analyzed the data of Visa applicants, built a predictive model to facilitate the process of visa approvals, and based on important factors that significantly influence the Visa status recommended a s

Jesus 1 Jan 08, 2022