Yet Another Workflow Parser for SecurityHub

Related tags

Data Analysisyawps
Overview

YAWPS

Yet Another Workflow Parser for SecurityHub

"Screaming pepper" by Rum Bucolic Ape is licensed with CC BY-ND 2.0. To view a copy of this license, visit https://creativecommons.org/licenses/by-nd/2.0/

Purpose

Currently SecurityHub has a ChatBot integration thats a bit lacking. All of securityhub goes to chatbot, which means a singular flooding channel of alerts.

With cloud-custodians recent support for securityhub and organizations we have a good way to send all alerts for an entire org to slack. But that means every account goes to a single channel.

This repo is part of a multi-series talk/demo on how to intelligently route account messages to differing Slack channels.

In the scenario where a team owns an account it would be nice to let cloud-custodian generate meaningful securityhub notifications that go to specific team channels.

For this talk we will simply tag AWS accounts with 2 tags account_name (a human readable name) and slack_channel (a slack channel to direct those security hub notifications to).

A blog post and KubeCon talk will be coming soon with more information

Prerequisites

The only real pre-requisite here is a working multi-account SecurityHub

Configuration

Environment Variable Description
SLACK_FALLBACK_CHANNEL Channel to fallback to if the slack_channel tag is not provided on the account
SLACK_TOKEN the path in SSM to the slack token`
SLACK_TOKEN_SSM_PATH if a SLACK_TOKEN is not found, this is where to grab it from the EC2 Param store
LOGGING_LEVEL the logging level to use. Default is INFO
ENABLE_FORK_COPY_SEVERITY Enable the ability to fork some messages to another channel by severity. Value can be True or False. Default is False
FORK_COPY_SEVERITY_VALUE If ENABLE_FORK_COPY_SEVERITY is True, what severity level to fork by. Should be an integer between 0 and 100. Default is 90
ENABLE_FORK_ONLY_SEVERITY Enable the ability to fork some messages to only another channel by severity. Value can be True or False. Default is False
FORK_ONLY_SEVERITY_VALUE If ENABLE_FORK_ONLY_SEVERITY is True, what severity level to fork by. Should be an integer between 0 and 100. Default is 100

Forking

There are a few use cases for forking.

In general (all defaults) YAWPS will only send to the channel found in the tag or the SLACK_FALLBACK_CHANNEL (because it's required).

This is great until you have rules that you want a second team (lets say security) to also see and follow up with.

Using ENABLE_FORK_COPY_SEVERITY and FORK_COPY_SEVERITY_VALUE lets you also send to that second slack channel. Lets say you set FORK_COPY_SEVERITY_VALUE to 90. This means that anything rated 90 will send to both.

Another use-case exists: not sending team specific alerts. Lets say that an alert is not actionable by the configured team, but is purely for security visibility (like failed IAM logins etc). You can use ENABLE_FORK_ONLY_SEVERITY set to, say 100, in this scenario so that custom rules can set severity to 100 and send it only to security and bypass the primary team. This is good for noise filtration and helping to keep things actionable by a singular source.

Deploy

ServerLess

TODO

Terraform

  1. Download this repository (or a released artifact)
  2. Run make zip to produce a fully deployable s3 artifact
  3. Deploy something similar to this terraform

Testing

$ poetry install
$ poetry run tox
Owner
myoung34
Cloud security engineer, tinkerer, tomato farmer
myoung34
A crude Hy handle on Pandas library

Quickstart Hyenas is a curde Hy handle written on top of Pandas API to allow for more elegant access to data-scientist's powerhouse that is Pandas. In

Peter Výboch 4 Sep 05, 2022
Pandas and Dask test helper methods with beautiful error messages.

beavis Pandas and Dask test helper methods with beautiful error messages. test helpers These test helper methods are meant to be used in test suites.

Matthew Powers 18 Nov 28, 2022
Data Analytics on Genomes and Genetics

Data Analytics performed on On genomes and Genetics dataset to predict genetic disorder and disorder subclass. DONE by TEAM SIGMA!

1 Jan 12, 2022
PipeChain is a utility library for creating functional pipelines.

PipeChain Motivation PipeChain is a utility library for creating functional pipelines. Let's start with a motivating example. We have a list of Austra

Michael Milton 2 Aug 07, 2022
A lightweight, hub-and-spoke dashboard for multi-account Data Science projects

A lightweight, hub-and-spoke dashboard for cross-account Data Science Projects Introduction Modern Data Science environments often involve many indepe

AWS Samples 3 Oct 30, 2021
Hatchet is a Python-based library that allows Pandas dataframes to be indexed by structured tree and graph data.

Hatchet Hatchet is a Python-based library that allows Pandas dataframes to be indexed by structured tree and graph data. It is intended for analyzing

Lawrence Livermore National Laboratory 14 Aug 19, 2022
vartests is a Python library to perform some statistic tests to evaluate Value at Risk (VaR) Models

gg I wasn't satisfied with any of the other available Gemini clients, so I wrote my own. Requires Python 3.9 (maybe older, I haven't checked) and opti

RAFAEL RODRIGUES 5 Jan 03, 2023
Geospatial data-science analysis on reasons behind delay in Grab ride-share services

Grab x Pulis Detailed analysis done to investigate possible reasons for delay in Grab services for NUS Data Analytics Competition 2022, to be found in

Keng Hwee 6 Jun 07, 2022
The Spark Challenge Student Check-In/Out Tracking Script

The Spark Challenge Student Check-In/Out Tracking Script This Python Script uses the Student ID Database to match the entries with the ID Card Swipe a

1 Dec 09, 2021
CleanX is an open source python library for exploring, cleaning and augmenting large datasets of X-rays, or certain other types of radiological images.

cleanX CleanX is an open source python library for exploring, cleaning and augmenting large datasets of X-rays, or certain other types of radiological

Candace Makeda Moore, MD 20 Jan 05, 2023
OpenARB is an open source program aiming to emulate a free market while encouraging players to participate in arbitrage in order to increase working capital.

Overview OpenARB is an open source program aiming to emulate a free market while encouraging players to participate in arbitrage in order to increase

Tom 3 Feb 12, 2022
A simplified prototype for an as-built tracking database with API

Asbuilt_Trax A simplified prototype for an as-built tracking database with API The purpose of this project is to: Model a database that tracks constru

Ryan Pemberton 1 Jan 31, 2022
Stitch together Nanopore tiled amplicon data without polishing a reference

Stitch together Nanopore tiled amplicon data using a reference guided approach Tiled amplicon data, like those produced from primers designed with pri

Amanda Warr 14 Aug 30, 2022
MapReader: A computer vision pipeline for the semantic exploration of maps at scale

MapReader A computer vision pipeline for the semantic exploration of maps at scale MapReader is an end-to-end computer vision (CV) pipeline designed b

Living with Machines 25 Dec 26, 2022
Automated Exploration Data Analysis on a financial dataset

Automated EDA on financial dataset Just a simple way to get automated Exploration Data Analysis from financial dataset (OHLCV) using Streamlit and ta.

Darío López Padial 28 Nov 27, 2022
Powerful, efficient particle trajectory analysis in scientific Python.

freud Overview The freud Python library provides a simple, flexible, powerful set of tools for analyzing trajectories obtained from molecular dynamics

Glotzer Group 195 Dec 20, 2022
Visions provides an extensible suite of tools to support common data analysis operations

Visions And these visions of data types, they kept us up past the dawn. Visions provides an extensible suite of tools to support common data analysis

168 Dec 28, 2022
In this tutorial, raster models of soil depth and soil water holding capacity for the United States will be sampled at random geographic coordinates within the state of Colorado.

Raster_Sampling_Demo (Resulting graph of this demo) Background Sampling values of a raster at specific geographic coordinates can be done with a numbe

2 Dec 13, 2022
nrgpy is the Python package for processing NRG Data Files

nrgpy nrgpy is the Python package for processing NRG Data Files Website and source: https://github.com/nrgpy/nrgpy Documentation: https://nrgpy.github

NRG Tech Services 23 Dec 08, 2022
NumPy and Pandas interface to Big Data

Blaze translates a subset of modified NumPy and Pandas-like syntax to databases and other computing systems. Blaze allows Python users a familiar inte

Blaze 3.1k Jan 05, 2023