The elegance of Airflow + the power of AWS

Overview

Orkestra

The elegance of Airflow + the power of AWS

Docs Codecov

PyPI PyPI - Downloads PyPI - License PyPI - Python Version GitHub issues Mentioned in Awesome CDK

examples/hello_orkestra.py

list: logger.info(item.dict()) return [item.dict()] * 10 @compose(model=Item, is_map_job=True, **default_args) def double_price(item: Item, context): item.price = item.price * 2 return item.dict() @compose(**default_args) def assert_false(event, context): assert False @compose(**default_args) def do_nothing(event, context): logger.info({"doing": "nothing"}) @compose(**default_args) def say_hello(event, context): return "hello, world" @compose(**default_args) def say_goodbye(event, context): return "goodbye" @compose(**default_args) def random_int(event, context): return random.randrange(100) @compose(**default_args) def random_float(event, context): return float(random_int(event, context)) dag() ">
import random
from typing import *
from uuid import uuid4

from aws_lambda_powertools import Logger, Tracer
from pydantic import BaseModel

from orkestra import compose
from orkestra.interfaces import Duration


def dag():
    (
        generate_item
        >> add_price
        >> copy_item
        >> double_price
        >> (do_nothing, assert_false)
        >> say_hello
        >> [random_int, random_float]
        >> say_goodbye
    )


class Item(BaseModel):
    id: str
    name: str
    price: Optional[float] = None

    @classmethod
    def random(cls):
        return cls(
            id=str(uuid4()),
            name=random.choice(
                [
                    "potato",
                    "moon rock",
                    "hat",
                ]
            ),
        )


logger = Logger()

tracer = Tracer()


default_args = dict(
    enable_powertools=True,
    timeout=Duration.seconds(6),
)


@compose(**default_args)
def generate_item(event, context):
    logger.info("generating random item")
    item = Item.random().dict()
    logger.info(item)
    tracer.put_metadata("GenerateItem", "SUCCESS")
    return item


@compose(model=Item, **default_args)
def add_price(item: Item, context):
    price = 3.14
    logger.info(
        "adding price to item",
        extra={
            "item": item.dict(),
            "price": price,
        },
    )
    item.price = price
    return item.dict()


@compose(model=Item, **default_args)
def copy_item(item: Item, context) -> list:
    logger.info(item.dict())
    return [item.dict()] * 10


@compose(model=Item, is_map_job=True, **default_args)
def double_price(item: Item, context):
    item.price = item.price * 2
    return item.dict()


@compose(**default_args)
def assert_false(event, context):
    assert False


@compose(**default_args)
def do_nothing(event, context):
    logger.info({"doing": "nothing"})


@compose(**default_args)
def say_hello(event, context):
    return "hello, world"


@compose(**default_args)
def say_goodbye(event, context):
    return "goodbye"


@compose(**default_args)
def random_int(event, context):
    return random.randrange(100)


@compose(**default_args)
def random_float(event, context):
    return float(random_int(event, context))


dag()

app.py

#!/usr/bin/env python3
from aws_cdk import core as cdk

from examples.hello_orkestra import generate_item


class HelloOrkestra(cdk.Stack):
    def __init__(self, scope, id, **kwargs):

        super().__init__(scope, id, **kwargs)

        generate_item.schedule(
            self,
            expression="rate(5 minutes)",
            state_machine_name="hello_orkestra",
        )


app = cdk.App()


app.synth()

state machine

Owner
Stephan Fitzpatrick
“Talk is cheap. Show me the code.”
Stephan Fitzpatrick
Modular Python-based Twitch bot optimized for customizability and ease of use.

rasbot Modular Python-based Twitch bot optimized for customizability and ease of use. rasbot is a Python-based Twitch bot that runs on your Twitch acc

raspy 9 Dec 14, 2022
Bot made by BLACKSTORM[BM] Contact Us - t.me/BLACKSTORM18

ᴡʜᴀᴛ ɪs ᴊᴀʀᴠɪs sᴇᴄᴜʀɪᴛʏ ʙᴏᴛ ᴊᴀʀᴠɪs ʙᴏᴛ ɪs ᴛᴇʟᴇɢʀᴀᴍ ɢʀᴏᴜᴘ ᴍᴀɴᴀɢᴇʀ ʙᴏᴛ ᴡɪᴛʜ ᴍᴀɴʏ ғᴇᴀᴛᴜʀᴇs. ᴛʜɪs ʙᴏᴛ ʜᴇʟᴘs ʏᴏᴜ ᴛᴏ ᴍᴀɴᴀɢᴇ ʏᴏᴜʀ ɢʀᴏᴜᴘs ᴇᴀsɪʟʏ. ᴏʀɪɢɪɴᴀʟʟʏ ᴀ

1 Dec 11, 2021
Generate and Visualize Data Lineage from query history

Tokern Lineage Engine Tokern Lineage Engine is fast and easy to use application to collect, visualize and analyze column-level data lineage in databas

Tokern 237 Dec 29, 2022
Enumerate Microsoft 365 Groups in a tenant with their metadata

Enumerate Microsoft 365 Groups in a tenant with their metadata Description The all_groups.py script allows to enumerate all Microsoft 365 Groups in a

Clément Notin 46 Dec 26, 2022
Telegram Group Management Bot based on phython !!!

How to setup/deploy. For easiest way to deploy this Bot click on the below button Mᴀᴅᴇ Bʏ Sᴜᴘᴘᴏʀᴛ Sᴏᴜʀᴄᴇ Find This Bot on Telegram A modular Telegram

Mukesh Solanki 5 Nov 17, 2021
A taskbar clock for secondary taskbars on Windows 11

ElevenClock A taskbar clock for secondary taskbars on Windows 11. When microsoft's engineers were creating Windows 11, they forgot to add a clock on t

Martí Climent 1.7k Jan 07, 2023
Me and @nathanmargni did a small analysis on what are the best strategies to win more games of League of Legends.

Me and @nathanmargni did a small analysis on what are the best strategies to win more games of League of Legends.

Christian Berchtold 2 Jan 19, 2022
The purpose of this bot is to take soundcloud track requests, that are posted in the stream-requests channel, and add them to a playlist, to make the process of scrolling through the requests easier for Root

Discord Song Collector Dont know if anyone is actually going to read this, but the purpose of this bot is to check every message in the stream-request

2 Mar 01, 2022
Some random bot for Discord which was created just for fun (Made with Discord.py library)

Ghosty Previously known as 'secondthunder-py-bot' This is repository of some random bot for Discord which was created just for fun and for some educat

Владислав 8 Oct 02, 2022
Karen is a Discord Bot that will check for a list of forbidden words/expressions, removing the message that contains them and replying with another message.

Karen is a Discord Bot that will check for a list of forbidden words/expressions, removing the message that contains them and replying with another message. Everything is highly customizable.

Rafael Almeida 1 Nov 03, 2021
A discord bot written in discord.py to manage custom roles assigned to boosters of your server.

BBotty A discord bot written in discord.py to manage custom roles assigned to boosters of your server. v0.0.1-alpha released! This version is incomple

Oui002 1 Nov 27, 2021
Python tool to Check running WebClient services on multiple targets based on @leechristensen

WebClient Service Scanner Python tool to Check running WebClient services on multiple targets based on @tifkin_ idea. This tool uses impacket project.

Pixis 153 Dec 28, 2022
Diablo II Resurrected helper

Diablo II Resurrected 快捷施法辅助 功能: + 创建守护进程,注册全局热键 alt+/ 启用和关闭功能 (todo: 播放声音提示) + 按 x 强制移动 + 按 1 ~ 0 快捷施法到鼠标区域 使用 编辑配置 settings.py 技能信息做如下定义: SKILLS:

Wan 2 Nov 06, 2022
Framework for creating and running trading strategies. Blatantly stolen copy of qtpylib to make it work for Indian markets.

_• Kinetick Trade Bot Kinetick is a framework for creating and running trading strategies without worrying about integration with broker and data str

Vinay 41 Dec 31, 2022
Manage Proxmox KVM Virtual Machines via Slack bot.

proxmox-slack-bot Create KVM Virtual Machines on Proxmox, the easy way. Not much works works here yet... Setup dev environment Setup fully editable st

Plenus Pyramis 3 Mar 20, 2022
An API wrapper for convertio.co written in Python.

An API wrapper for convertio.co written in Python.

Moonrise 9 Sep 27, 2022
An NFTGenerator to generate NFTs and send them to nft.storage

NFTGenerator Table of Contents Overview Installation Introduction Features Reflection Issues & bug reports Show your support Credits Overview The NFTG

3 Mar 14, 2022
Report-snapchat - Report Snapchat acc with python

report-snapchat Report Snapchat acc Report users on Snapchat about the tool : 4

17 Dec 01, 2022
WhatsApp API Python ChatBot

WhatsApp Api Python - Esta documentação tem como objetivo exemplificar o uso do Moorse Whatsapp API na linguagem Python.

Douglas Alves 2 Jan 06, 2022