The elegance of Airflow + the power of AWS

Overview

Orkestra

The elegance of Airflow + the power of AWS

Docs Codecov

PyPI PyPI - Downloads PyPI - License PyPI - Python Version GitHub issues Mentioned in Awesome CDK

examples/hello_orkestra.py

list: logger.info(item.dict()) return [item.dict()] * 10 @compose(model=Item, is_map_job=True, **default_args) def double_price(item: Item, context): item.price = item.price * 2 return item.dict() @compose(**default_args) def assert_false(event, context): assert False @compose(**default_args) def do_nothing(event, context): logger.info({"doing": "nothing"}) @compose(**default_args) def say_hello(event, context): return "hello, world" @compose(**default_args) def say_goodbye(event, context): return "goodbye" @compose(**default_args) def random_int(event, context): return random.randrange(100) @compose(**default_args) def random_float(event, context): return float(random_int(event, context)) dag() ">
import random
from typing import *
from uuid import uuid4

from aws_lambda_powertools import Logger, Tracer
from pydantic import BaseModel

from orkestra import compose
from orkestra.interfaces import Duration


def dag():
    (
        generate_item
        >> add_price
        >> copy_item
        >> double_price
        >> (do_nothing, assert_false)
        >> say_hello
        >> [random_int, random_float]
        >> say_goodbye
    )


class Item(BaseModel):
    id: str
    name: str
    price: Optional[float] = None

    @classmethod
    def random(cls):
        return cls(
            id=str(uuid4()),
            name=random.choice(
                [
                    "potato",
                    "moon rock",
                    "hat",
                ]
            ),
        )


logger = Logger()

tracer = Tracer()


default_args = dict(
    enable_powertools=True,
    timeout=Duration.seconds(6),
)


@compose(**default_args)
def generate_item(event, context):
    logger.info("generating random item")
    item = Item.random().dict()
    logger.info(item)
    tracer.put_metadata("GenerateItem", "SUCCESS")
    return item


@compose(model=Item, **default_args)
def add_price(item: Item, context):
    price = 3.14
    logger.info(
        "adding price to item",
        extra={
            "item": item.dict(),
            "price": price,
        },
    )
    item.price = price
    return item.dict()


@compose(model=Item, **default_args)
def copy_item(item: Item, context) -> list:
    logger.info(item.dict())
    return [item.dict()] * 10


@compose(model=Item, is_map_job=True, **default_args)
def double_price(item: Item, context):
    item.price = item.price * 2
    return item.dict()


@compose(**default_args)
def assert_false(event, context):
    assert False


@compose(**default_args)
def do_nothing(event, context):
    logger.info({"doing": "nothing"})


@compose(**default_args)
def say_hello(event, context):
    return "hello, world"


@compose(**default_args)
def say_goodbye(event, context):
    return "goodbye"


@compose(**default_args)
def random_int(event, context):
    return random.randrange(100)


@compose(**default_args)
def random_float(event, context):
    return float(random_int(event, context))


dag()

app.py

#!/usr/bin/env python3
from aws_cdk import core as cdk

from examples.hello_orkestra import generate_item


class HelloOrkestra(cdk.Stack):
    def __init__(self, scope, id, **kwargs):

        super().__init__(scope, id, **kwargs)

        generate_item.schedule(
            self,
            expression="rate(5 minutes)",
            state_machine_name="hello_orkestra",
        )


app = cdk.App()


app.synth()

state machine

Owner
Stephan Fitzpatrick
“Talk is cheap. Show me the code.”
Stephan Fitzpatrick
A library that allows you to easily mock out tests based on AWS infrastructure.

Moto - Mock AWS Services Install $ pip install moto[ec2,s3,all] In a nutshell Moto is a library that allows your tests to easily mock out AWS Services

Steve Pulec 6.5k Jan 02, 2023
This repository is used to simplify the process of cloning the SSM documents across the AWS regions.

SSM Cloner Introduction This module is created in order to simplify the process of copying the SSM documents from one region to another regions. As an

6 Jun 04, 2022
Using Streamlit to build a simple UI on top of the OpenSea API

OpenSea API Explorer Using Streamlit to build a simple UI on top of the OpenSea API. 🤝 Contributing Contributions, issues and feature requests are we

Gavin Capriola 1 Jan 04, 2022
Get Notified about vaccine availability in your location on email & sms ✉️! Vaccinator Octocat tracks & sends personalised vaccine info everday. Go get your shot ! 💉

Vaccinater Get Notified about vaccine availability in your location on email & sms ✉️ ! Vaccinator Octocat tracks & sends personalised vaccine info ev

Mayukh Pankaj 6 Apr 28, 2022
This is a scalable system that reads messages from public Telegram channels using Telethon and stores the data in a PostgreSQL database.

This is a scalable system that reads messages from public Telegram channels using Telethon and stores the data in a PostgreSQL database. Its original intention is to monitor cryptocurrency related ch

Greg 3 Jun 07, 2022
Simple PoC script that allows you to exploit telegram's "send with timer" feature by saving any media sent with this functionality.

Simple PoC script that allows you to exploit telegram's "send with timer" feature by saving any media sent with this functionality.

Matteo 52 Nov 29, 2022
This is a Discord script that will provide a QR Code to your scholars for Axie Infinity.

DiscordQRCodeBot This is a Discord script that will provide a QR Code to your Axie Infinity scholars. Setup Run Ubuntu on AWS ec2 instance Dowloads al

ZracheSs | xyZ 24 Oct 05, 2022
A Python Client to View F1TV Content the right way

F1Hub is a terminal application running directly on your computer -- no connection to the website needed* *In theory. As of now, the F1TV website is needed for some content

kodos 3 Jun 14, 2022
API para realizar parser de frases

NLP API Simple api to parse and apply some preprocessing steps in portuguses phrases (pt_BR) This api uses the great FastAPI and spaCy packages! Usage

⟠ Rodolfo De Nadai 1 Dec 28, 2021
Fetch torrent links from nyaa, according to releases by smoke index.

Nyaa - Smoke's index torrent fetcher Description This script parses the local (or online) anime release index (csv format) made by Big Smoke. And uses

Dinank 21 Jun 08, 2022
Telegram bot for downloading covid-19 vaccine certificate

cowin-certificate-bot This is the source code of @cowincertbot, A telegram bot inspired by the whatsapp bot implementation of indian government for co

ArUn Pt 30 Oct 07, 2022
Fully undetected auto skillcheck hack for dead by daylight that works decently well

Auto-skillcheck was made by Love ❌ code ✅ ❔ ・How to use Start off by installing python ofc Open cmd in the same directory and type pip install -r requ

Rdimo 10 Aug 13, 2022
Url-shortener - A url shortener made in python using the API's from the pyshorteners lib

URL Shortener Um encurtador de link feito em python usando as API's da lib pysho

Spyware 3 Jan 07, 2022
Zero2 Discord bot is written with Discord.py using Python.

Zero2 Discord bot is written with Discord.py using Python.

Siva Avanish 4 Nov 08, 2021
Baby Villager Bot

This script scrapes and stores the availability of timeslots for Car Driving Test at all RTA Serivce NSW centres in the state. Dependencies Account wi

Milo Weinberg 3 Dec 16, 2021
A simple Python app to provide RPC for iTunes and the Music app. MacOS exclusive.

Ongaku You know, ongaku. A port of Ongaku to Python. Why? I don't know. A simple application providing the now playing state from iTunes (or the Music

Deltaion Lee 4 Oct 22, 2022
Telegram bot untuk mencari jawaban dibrainly, support inline juga

Brainly-Telebot Bot Untuk Mencari Jawaban Dibrainly Jika ingin clone. Boleh kok Dibuat dengan python menggunakan MTproto Library. Yaitu Pyrogram Bot y

... 7 Mar 17, 2022
Um painel de consultas completo, com metodos atualizados.

Meu pix para eu comprar um café :D "25ef499b-d184-4aa1-9797-0a294be40d83" Painel-de-Consultas Completo. Feito por JOESTAR-TEAM Painel de consultas Com

Dio brando 10 Nov 19, 2021
Azure free vpn for students only! (Self hosted/No sketchy services/Fast and free)

Azpn-Azure-Free-VPN Azure free vpn for students only! (Self hosted/No sketchy services/Fast and free) This is an alternative secure way of accessing f

Harishankar Kumar 6 Mar 19, 2022
Automatically download any NFT collection from OpenSea.

OpenSea NFT Stealer The sole purpose of this script is to download any NFT collection from OpenSea. How does it work? Basically, the OpenSea website a

Dan 111 Dec 29, 2022