Using AWS's API Gateway + Lambda + Python to run a simple websocket application. For learning/testing

Overview

aws-simple-websocket

Using AWS's API Gateway + Lambda + Python to run a simple websocket application. For learning/testing. The AWS Resources seemed overly complex and were missing some critical gotchas in setting up a system like this.

Example terminal showing usage

Using the following for guidance:

Architecture

To keep things as basic as possible we're using a bare minimum of resources and CLI helpers where possible.

A client makes a connection via Websocket to an API Gateway V2. That gateway maintains a socket connection for us, and sends events to some sort of "integration" or handler. In our case, this will be a Lambda function that will handle the incoming socket events ($connect/$disconnect). It will handle messages sent from websocket clients, and to further expand this example, an outside data source via SNS topic. The API Gateway requires us to keep track of Connection IDs, so we can programmatically and precisely send messages to specific clients.

Using Terraform (in ./deployment) the following are created:

Architecture Diagram

  1. API Gateway V2 (Websocket) - The primary Websocket management service which holds sockets for connections and can hit a variety of AWS integrations

  2. Lambda - The main executor of business logic - where all our code will live

  3. S3 - A basic Key/Value store for our connections

  4. SNS - To demonstrate an external publisher, our Lambda function is also listening to an SNS Topic

Some additional resources are needed:

  1. CloudWatch - Logging for API Gateway and Lambda function with retention periods set by default

  2. IAM - Permissions to glue everything together

Deployment

This demo repo uses Terraform to manage cloud resources. These are all stored in the ./deployment repository. NOTE: Creating resources in AWS may incur charges to your account. Ensure you have billing alarms setup and understand AWS costs. This demo repo should cost almost nothing, however.

  1. Install Terraform
  2. Change to ./deployment directory
  3. Init Terraform (terraform init)
  4. It's best practice to use "Workspaces" to namespace resources in terraform for different environments, so create a dev workspace (terraform workspace new dev)
  5. Check if you need to enable API Gateway Logging in your current region. Feel free to set ./deployment/enf.tf:init_api_gw_logging_role to false if your account already has this setup
  6. Create the resources terraform apply
  7. Run /util/lambda-deploy dev to build and deploy the lambda code. There are no external dependencies, just boto3

Usage

Contained in ./util are a lot of small CLI scripts to allow us to interact with the system without bloating the core too much with things like static web pages for the app side.

  • lambda-build - builds a zip archive for deployment to Lambda

  • lambda-deploy - runs lambda-build and deploys the archive to the provided Lambda environment. Uses terraform to get the name of our Lambda function to deploy to

  • send-data [json] - Sends a JSON payload to connected clients via SNS Topic. Uses terraform to get the name of our SNS Topic

  • tail-logs - Watches logs for the API Gateway and Lambda function. Uses terraform to get the name of our the log groups

  • connect - Connect to the websocket using a small utility, websocat

Quick Example

  1. Deploy the stack using the above instructions.
  2. ./util/connect to connect and listen to the websocket
  3. Using a new terminal session, ./util/send-data '{"hello": "world"}'
  4. View that in your first terminal with connect running, you'll see {"hello": "world"}

Users can broadcast messages to other users as well, not just through SNS. With a connected client, send:

{"action": "broadcast", "message": "Good news, everyone!"}

You will then see: {"message": "Good news, everyone!"} in all the connected clients.

Improvements

  1. Move from print() to logging module, for the sake of keeping this really simple, I left print in there
Owner
Seth Miller
Seth Miller
PcapConverter - A project for generating 15min frames out of a .pcap file containing network traffic

CMB Assignment 02 code + notebooks This is a project for containing code for the

Yannik S 2 Jan 24, 2022
A Calendar subscribe server for python

cn-holiday-ics-server A calendar subscribe server 直接使用我搭建的服务 订阅节假日:https://cdxy.fun:9999/holiday 订阅调休上班:https://cdxy.fun:9999/workday 节假日和调休上班在一起的版本:h

CD 11 Nov 12, 2022
The Delegate Network: An Interactive Voice Response Delegative Democracy Implementation of Liquid Democracy

The Delegate Network Overview The delegate network is a completely transparent, easy-to-use and understand version of what is sometimes called liquid

James Bowery 2 Feb 25, 2022
This script helps us to add IP, host name entry in hosts file and create directory run nmap scan and directory scan with your favourite tools

A python script to automate your set-up for Hack The Box, It sets up Workspace, Opens TMUX session, connects to OpenVPN, Runs Nmap and many more...

Cognizance 7 Mar 25, 2022
Decentra Network is an open source blockchain that combines speed, security and decentralization.

Decentra Network is an open source blockchain that combines speed, security and decentralization. Decentra Network has very high speeds, scalability, asymptotic security and complete decentralization

Decentra Network 74 Nov 22, 2022
A website to list Shadowsocks proxies and check them periodically

Shadowmere An automatically tested list of Shadowsocks proxies. Motivation Collecting proxies around the internet is fun, but what if they stop workin

Jorge Alberto Díaz Orozco (Akiel) 29 Dec 21, 2022
FIRM-AFL is the first high-throughput greybox fuzzer for IoT firmware.

FIRM-AFL is the first high-throughput greybox fuzzer for IoT firmware. FIRM-AFL addresses two fundamental problems in IoT fuzzing

356 Dec 23, 2022
Typhon is a macOS specific payload aimed at targetting Jamf managed devices.

Typhon is a macOS specific payload aimed at targetting Jamf managed devices. This payload can be used to manipulate macOS devices into communicating with a Mythic instance, which acts as a Jamf serve

Mythic Agents 29 Dec 23, 2022
BlueHawk is an HTTP/1.1 compliant web server developed in python

This project is done as a part of Computer Networks course. It aims at the implementation of the HTTP/1.1 Protocol based on RFC 2616 and includes the basic HTTP methods of GET, POST, PUT, DELETE and

2 Nov 11, 2022
Network Dynaimcs Simulation

A Final Year Project in CUHK, Autumn 2021 Network Dynaimcs Simulation Files param.h edit all the variables & settings here simulate.c the main program

Likchun 0 Mar 28, 2022
Pteronode - Script for managing Pterodactyl nodes

pteronode Script for managing Pterodactyl nodes Pteronode allows you to create s

9 Sep 28, 2022
A repository to spoof ARP table of any devices and successfully establish Man in the Middle(MITM) attack using Python3 in Linux

arp_spoofer A repository to spoof ARP table of any devices and successfully establish Man in the Middle(MITM) attack using Python3 in Linux Usage: git

Surya Das N 1 Oct 30, 2021
Ultimate transformation library that supports validation, contexts and aiohttp.

Trafaret Ultimate transformation library that supports validation, contexts and aiohttp. Trafaret is rigid and powerful lib to work with foreign data,

Mikhail Krivushin 174 Nov 27, 2022
TsuserverMoS - A Python-based server for Attorney Online,

tsuserverMoS A Python-based server for Attorney Online, forked from RealKaiser/tsuserverCC Requires Python 3.7+ and PyYAML. Changes/additions from tsu

1 Dec 30, 2021
GhostVPN - Simple and lightweight TUI application for CyberGhostVPN

GhostVPN Simple and lightweight TUI application for CyberGhostVPN. Screenshot Us

Mehmet Ali KERİMOĞLU 5 Jul 27, 2022
Whoisss is a website information gatharing Tool.

Whoisss Whoisss is a website information gatharing Tool. You can cse it to collect information about website. Usage apt-get update apt-get upgrade pkg

Md. Nur habib 2 Jan 23, 2022
Tripwire monitors ports and icmp to send the admin a message if somebody is scanning a machine that shouldn't be touched

Tripwire monitors ports and icmp to send the admin a message if somebody is scanning a machine that shouldn't be touched

3 Apr 05, 2022
Build custom OSINT tools and APIs (Ping, Traceroute, Scans, Archives, DNS, Scrape, Whois, Metadata & built-in database for more info) with this python package

Build custom OSINT tools and APIs with this python package - It includes different OSINT modules (Ping, Traceroute, Scans, Archives, DNS, Scrape, Whoi

QeeqBox 52 Jan 06, 2023
Easy-to-setup bot, ChatOps project for handling telegram chat logging over docker-compose services, being runned as one of them.

Easy-to-setup bot, ChatOps project for handling telegram chat logging over docker-compose services, being runned as one of them.

Rashid 7 Aug 08, 2022
ServerStatus with node management and monitor

ServerStatus with node management and monitor

lidalao 162 Jan 01, 2023