Async boto3 with Autogenerated Data Classes

Overview

awspydk

Async boto3 with Autogenerated JIT Data Classes


Motivation

This library is forked from an internal project that works with a lot of backend AWS APIs, and I got tired of having to constantly parse the returned responses before being able to work with them for (most of the time), a few seconds. Any API-driven application that uses boto3 tends to suffer from being syncronous only. So this library solves a few major problems:

  • Enables both sync and async from the same client.

  • Client can be called implicitly without needing initialization, i.e. you can directly use AwsClient without needing to initialize.

  • Dynamically generates class functions based on the boto3.client child functions, and allows you to call them. This is useful in repl or ipython environments where type-hints are always helpful, especially when the names are so long.

  • Automatically initializes the aws client if its not initialized from the defaults, simply by calling it.

  • Translates all results into Automatically Generated Dataclasses through lazycls.

    • Can be disabled by setting aws.config.AutoCls = False

Quickstart

pip install --upgrade awspydk
from aws import AwsClient

# Sync Method
buckets = AwsClient.v1.s3_list_buckets(as_cls=True)

# Async Method
buckets = await AwsClient.v1.async_s3_list_buckets(as_cls=True)

"""
Both yield the same results.
The underlying classes are auto-generated from Pydantic BaseModels, so anything you can do with Pydantic Models, you can do with these.

{
    'Buckets': [
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 8, 25, 16, 42, 46, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 2, 17, 54, 56, tzinfo=tzutc()), Name='...',
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 3, 4, 20, 10, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 1, 20, 50, 33, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 2, 4, 2, 28, tzinfo=tzutc()), Name='...')
    ],
    'Owner': AwsS3Owner(DisplayName='...', ID='...')
}
"""

## Change Regions
AwsClient.reset(region='us-west-1')

## Change the defaut clients created
from aws.config import DefaultClients

## Modify to only create ec2 client
DefaultClients = {
    'ec2': 'ec2'
}

## Reset implicitly
AwsClient.reset()


BotoKwargs = {
    'AWS_PROFILE': ...,
}

## Reset Explicitly
AwsClient.reset(clients=DefaultClients, boto_kwargs=BotoKwargs)

Client Defaults

These are found in aws.config

AwsRegion = envToStr('AWS_REGION', 'us-east-1')
AutoCls = envToBool('AWSSDK_AUTOCLS', 'true')

## These are the default clients that will be autogenerated.
## Key is the shorthand, value is the actual AWS API Name in boto3
DefaultClients = {
    'ec2': 'ec2', 
    'ecr': 'ecr', 
    'r53' :'route53', 
    'acm': 'acm',
    'elb': 'elb',
    'elbv2': 'elbv2',
    'asg': 'autoscaling',
    's3': 's3'
}

## These are the default resources that will be autogenerated.
## Key is the shorthand, value is the actual AWS Resource API Name in boto3
DefaultResources = {
    'Ec2': 'ec2',
    'S3': 's3',
    'Iam': 'iam'
}

# These are the default filter args for querying
DefaultFilterArgs = {
    'string_only': True,
    'remove_null': True
}
Owner
Chief Architect @ Growth Engine
This is a story bot, that will scrape stories from r/stories subreddit and convert it into an Audio File.

Introduction This is a story bot, that will scrape stories from r/stories subreddit and convert it into an Audio File. Installation pip install -r req

Yasho 11 Jun 30, 2022
Python API Client for Twitter API v2

🐍 Python Client For Twitter API v2 🚀 Why Twitter Stream ? Twitter-Stream.py a python API client for Twitter API v2 now supports FilteredStream, Samp

Twitivity 31 Nov 19, 2022
A Twitter bot written in Python using Tweepy and hosted on a server.

A Twitter bot written in Python using Tweepy. It can like and/or retweet tweets that contain single or multiple keywords and hashtags.

anniedotexe 11 Dec 15, 2022
Telegram bot with various Sticker Tools

Sticker Tools Bot @Sticker_Tools_Bot A star ⭐ from you means a lot to us! Telegram bot with various Sticker Tools Usage Deploy to Heroku Tap on above

Stark Bots 20 Dec 08, 2022
Utility for downloading fanfiction in bulk from the Archive of Our Own

What is this? This is a program intended to help you download fanfiction from the Archive of Our Own in bulk. This program is primarily intended to wo

73 Dec 30, 2022
A Telegram Video Watermark Adder Bot

Watermark-Bot A Telegram Video Watermark Adder Bot by @VideosWaterMarkRobot Features: Save Custom Watermark Image. Auto Resize Watermark According to

5 Jun 17, 2022
Decode the Ontario proof of vaccination QR code

Decode the contents of the Ontario Proof of Vaccination (the "Smart Health Card QR Code") Output This is from my QR code, hopefully fully redacted alt

Wesley Ellis 4 Oct 22, 2021
Generate discord nitro codes and check them

Discord Nitro Generator and Checker A discord nitro generator and checker for all your nitro needs Explore the docs » Report Bug · Request Feature · J

509 Jan 02, 2023
A collection of automation aids to connect various database systems into Lookout for Metrics

A collection of automation aids to connect various database systems into Lookout for Metrics

AWS Samples 3 Apr 28, 2022
Nflmetrics - Johns Hopkins Spring 2022 Sports Analytics research project about NFL Draft Metrics

nflmetrics GitHub repo for Johns Hopkins Spring 2022 Sports Analytics research p

Anish Kulkarni 4 Feb 24, 2022
PyDiscord, a maintained fork of discord.py, is a python wrapper for the Discord API.

discord.py A modern, easy to use, feature-rich, and async ready API wrapper for Discord written in Python. The Future of discord.py Please read the gi

Omkaar 1 Jan 16, 2022
⭐️ Pyro String Generator ⭐️ Genrate String Session Using this bot.Made by TeamUltronX 🔥

⭐️ Pyro String Generator ⭐️ Genrate String Session Using this bot.Made by TeamUltronX 🔥 Configs: API_HASH Get from Here. API_ID Get from Here. API_KE

TheUltronX 2 Dec 16, 2022
提供火币网交易接口API最简封装,提供现货买入、卖出、huobi币安查询账户余额等接口,数字货币,虚拟货币,BTC量化交易框架,自动交易,轻量便携,不用安装,即开即用

火币网交易接口的最简封装(只管用,不用再关注细节) 提供火币网交易接口的python封装,提供买入、卖出、查询账户余额等接口 接口说明 order_value() 进行买入操作,参数为买入的币和买入的金额 买入返回的详情数据: {'单号': '272229546125038', '成交数量': 0.

dev 95 Sep 24, 2021
This is a unofficial library for making bots in rubika.

rubika this is a unofficial library for making bots in rubika using this library you can make your own0 rubika bot and control that those bots that ma

Bahman 50 Jan 02, 2023
Discord bot for the IOTA Wiki

IOTA Wiki Bot Discord bot for the IOTA Wiki Report Bug · Request Feature About The Project This is a Discord bot for the IOTA Wiki. It's currently use

IOTA Community 2 Nov 14, 2021
Discord bot to display private leaderboards for Advent of Code.

Advent Of Code Discord Bot Discord bot for displaying Advent of Code private leardboards, as well as custom leaderboards where participants can set th

The Future Gadgets Lab 6 Nov 29, 2022
Heroku app to explore boardgame data

A Dashboard for the Board Game Geeks among us Link to Application As many Board Game Geeks like myself track the scores of board game matches I decide

Maarten Grootendorst 20 Nov 23, 2022
Bitstamp API wrapper for Python

NOTICE: THIS REPOSITORY IS NO LONGER ACTIVELY MAINTAINED It is highly unlikely that I will respond to PRs and questions about usage. This library was

Jack Preston 53 Mar 09, 2022
Stock Market Insights is a Dashboard that gives the 360 degree view of the particular company stock

fedora-easyfix A collection of self-contained and well-documented issues for newcomers to start contributing with How to setup the local development e

Ganesh N 3 Sep 10, 2021
A Telegram bot written in python.

telegram_bot This bot is currently a beta project. Features A telegram bot which can: Send current COVID-19 cases/stats of Germany Send current worth

HuhnCares 1 Jan 11, 2022