Domain Connectivity Analysis Tools to analyze aggregate connectivity patterns across a set of domains during security investigations

Overview

DomainCAT (Domain Connectivity Analysis Tool)

Domain Connectivity Analysis Tool is used to analyze aggregate connectivity patterns across a set of domains during security investigations

This project was a collaborative effort between myself and Matthew Pahl

Introduction

When analyzing pivots during threat hunting, most people approach it from the perspective of “what can a single pivot tell you?” But often actors will set their domains up to use commodity hosting infrastructure, so the number of entities associated with a given pivot are so big they don’t really give you any useful information.

This is where DomainCAT can help. Actors make decisions around domain registration and hosting options when setting up their malicious infrastructure. These can be considered behavioral choices.

  • What registrar(s) do they use?
  • What TLDs do they prefer?
  • What hosting provider(s) do they like?
  • What TLS cert authority do they use?

All of these decisions, together, makeup part of that actor’s infrastructure tools, tactics and procedures (TTPs), and we can analyze them as a whole to look for patterns across a set of domains.

DomainCAT is a tool written in Jupyter Notebooks, a web-based interactive environment that lets you combine text, code, data, and interactive visualizations into your threat hunting toolbelt. The tool analyzes aggregate connectivity patterns across a set of domains looking at every pivot for every domain, asking; what are the shared pivots across these domains, how many shared pivots between each domain, do they have a small pivot count or a really large one? All of these aspects are taken into consideration as it builds out a connectivity graph that models how connected all the domains in an Iris search are to each other.

Example Visualizations:

3D visualization of domain to domain connections based on shared infrastructure, registration and naming patterns

SegmentLocal

2D visualization of domain to domain connection

domain_graph2d.png

DomainCat Tutorial

Click here for the DomainCAT Tutorial documentation

Installation Steps: Docker (recommended)

Note: building the container takes a bit of RAM to compile the resources for the jupyterlab-plotly extension. Bump up your RAM in Docker preferences to around 4Gb while building the container. Then afterwards you can drop it back down to your normal level to run the container

Steps:

Clone the git repository locally

$ git clone https://github.com/DomainTools/DomainCAT.git

Change directory to the domaincat folder

$ cd domaincat

Build the jupyter notebook container

$ docker build --tag domaincat .

Run the jupyter notebook

$ docker run -p 9999:9999 --name domaincat domaincat

Installation Steps: Manual (cross your fingers)

Note: this project uses JupyterLab Widgets, which requires nodejs >= 12.0.0 to be installed...which is on you

Steps:

Clone the git repository locally

$ git clone https://github.com/DomainTools/DomainCAT.git

Change directory to the domaincat folder

$ cd domaincat

Install python libraries

$ pip install -r requirements.txt

JupyterLab widgets extension

$ jupyter labextension install [email protected] --no-build
$ jupyter labextension install @jupyter-widgets/jupyterlab-manager --no-build
$ jupyter labextension install [email protected] --no-build
$ jupyter lab build

Run the jupyter notebook

$ jupyter lab

Plotly Bug: in the 2D visualization of the domain graph there is a weird bug in Plotly Visualization library where if your cursor is directly over the center of a node, the node's tool tip with the domain's name will disappear and if you click the node, it unselects all nodes. So only click on a node if you see it's tool tip

Owner
DomainTools
DomainTools
Learn Basic to advanced level Data visualisation techniques from this Repository

Data visualisation Hey, You can learn Basic to advanced level Data visualisation techniques from this Repository. Data visualization is the graphic re

Shashank dwivedi 16 Jan 03, 2023
An(other) implementation of JSON Schema for Python

jsonschema jsonschema is an implementation of JSON Schema for Python. from jsonschema import validate # A sample schema, like what we'd get f

Julian Berman 4k Jan 04, 2023
Mathematical learnings with Lean, for those of us who wish we knew more of both!

Lean for the Inept Mathematician This repository contains source files for a number of articles or posts aimed at explaining bite-sized mathematical c

Julian Berman 8 Feb 14, 2022
Shaded 😎 quantile plots

shadyquant 😎 This python package allows you to quantile and plot lines where you have multiple samples, typically for visualizing uncertainty. Your d

Mehrad Ansari 13 Sep 29, 2022
Jupyter notebook and datasets from the pandas Q&A video series

Python pandas Q&A video series Read about the series, and view all of the videos on one page: Easier data analysis in Python with pandas. Jupyter Note

Kevin Markham 2k Jan 05, 2023
These data visualizations were created for my introductory computer science course using Python

Homework 2: Matplotlib and Data Visualization Overview These data visualizations were created for my introductory computer science course using Python

Sophia Huang 12 Oct 20, 2022
cqMore is a CadQuery plugin based on CadQuery 2.1.

cqMore (under construction) cqMore is a CadQuery plugin based on CadQuery 2.1. Installation Please use conda to install CadQuery and its dependencies

Justin Lin 36 Dec 21, 2022
A curated list of awesome Dash (plotly) resources

Awesome Dash A curated list of awesome Dash (plotly) resources Dash is a productive Python framework for building web applications. Written on top of

Luke Singham 1.7k Dec 26, 2022
A Scheil-Gulliver simulation tool using pycalphad.

scheil A Scheil-Gulliver simulation tool using pycalphad. import matplotlib.pyplot as plt from pycalphad import Database, variables as v from scheil i

pycalphad 6 Dec 10, 2021
Create SVG drawings from vector geodata files (SHP, geojson, etc).

SVGIS Create SVG drawings from vector geodata files (SHP, geojson, etc). SVGIS is great for: creating small multiples, combining lots of datasets in a

Neil Freeman 78 Dec 09, 2022
With Holoviews, your data visualizes itself.

HoloViews Stop plotting your data - annotate your data and let it visualize itself. HoloViews is an open-source Python library designed to make data a

HoloViz 2.3k Jan 04, 2023
A Jupyter - Leaflet.js bridge

ipyleaflet A Jupyter / Leaflet bridge enabling interactive maps in the Jupyter notebook. Usage Selecting a basemap for a leaflet map: Loading a geojso

Jupyter Widgets 1.3k Dec 27, 2022
Machine learning beginner to Kaggle competitor in 30 days. Non-coders welcome. The program starts Monday, August 2, and lasts four weeks. It's designed for people who want to learn machine learning.

30-Days-of-ML-Kaggle 🔥 About the Hands On Program 💻 Machine learning beginner → Kaggle competitor in 30 days. Non-coders welcome The program starts

Roja Achary 145 Jan 01, 2023
Here I plotted data for the average test scores across schools and class sizes across school districts.

HW_02 Here I plotted data for the average test scores across schools and class sizes across school districts. Average Test Score by Race This graph re

7 Oct 27, 2021
mysql relation charts

sqlcharts 自动生成数据库关联关系图 复制settings.py.example 重命名为settings.py 将数据库配置信息填入settings.DATABASE,目前支持mysql和postgresql 执行 python build.py -b,-b是读取数据库表结构,如果只更新匹

6 Aug 22, 2022
Generate knowledge graphs with interesting geometries, like lattices

Geometric Graphs Generate knowledge graphs with interesting geometries, like lattices. Works on Python 3.9+ because it uses cool new features. Get out

Charles Tapley Hoyt 5 Jan 03, 2022
:bowtie: Create a dashboard with python!

Installation | Documentation | Gitter Chat | Google Group Bowtie Introduction Bowtie is a library for writing dashboards in Python. No need to know we

Jacques Kvam 753 Dec 22, 2022
NW 2022 Hackathon Project by Angelique Clara Hanzel, Aryan Sonik, Damien Fung, Ramit Brata Biswas

Spiral-Data-Visualizer NW 2022 Hackathon Project by Angelique Clara Hanzell, Aryan Sonik, Damien Fung, Ramit Brata Biswas Description This project vis

Damien Fung 2 Jan 16, 2022
Lime: Explaining the predictions of any machine learning classifier

lime This project is about explaining what machine learning classifiers (or models) are doing. At the moment, we support explaining individual predict

Marco Tulio Correia Ribeiro 10.3k Dec 29, 2022
High-level geospatial data visualization library for Python.

geoplot: geospatial data visualization geoplot is a high-level Python geospatial plotting library. It's an extension to cartopy and matplotlib which m

Aleksey Bilogur 1k Jan 01, 2023