Divide full port scan results and use it for targeted Nmap runs

Overview

DivideAndScan

Divide Et Impera And Scan (and also merge the scan results)

version python codacy pypi docker twitter


DivideAndScan is used to efficiently automate port scanning routine by splitting it into 3 phases:

  1. Discover open ports for a bunch of targets.
  2. Run Nmap individually for each target with version grabbing and NSE actions.
  3. Merge the results into a single Nmap report (different formats available).

For the 1st phase a fast port scanner is intended to be used (Masscan / RustScan), whose output is parsed and stored in a database (TinyDB). Next, during the 2nd phase individual Nmap scans are launched for each target with its set of open ports (multiprocessing is supported) according to the database data. Finally, in the 3rd phase separate Nmap outputs are merged into a single report in different formats (XML / HTML / Simple text / Grepable) with nMap_Merger.

Potential use cases:

  • Pentest engagements / red teaming with a large scope to enumerate.
  • Cybersecurity wargames / training CTF labs.
  • OSCP certification exam.

How It Works

how-it-works.png

How to Install

Prerequisites

To successfully divide and scan we need to get some good port scanning tools.

๐Ÿ“‘ Note: if you don't feel like messing with dependecies on your host OS, skip to the Docker part.

Nmap

sudo apt install nmap sudo xsltproc -y
sudo nmap --script-updatedb

Masscan

cd /tmp
git clone https://github.com/robertdavidgraham/masscan.git
cd masscan
make
sudo make install
cd && rm -rf /tmp/masscan

RustScan

cd /tmp

wget -qO- https://api.github.com/repos/RustScan/RustScan/releases/latest \
| grep "browser_download_url.*amd64.deb" \
| cut -d: -f2,3 \
| tr -d \" \
| wget -qO rustscan.deb -i-

sudo dpkg -i rustscan.deb
cd && rm /tmp/rustscan.deb

sudo wget https://gist.github.com/snovvcrash/c7f8223cc27154555496a9cbb4650681/raw/a76a2c658370d8b823a8a38a860e4d88051b417e/rustscan-ports-top1000.toml -O /root/.rustscan.toml

Installation

DivideAndScan is available on PyPI as divideandscan, though I recommend installing it from GitHub with pipx in order to always have the bleeding-edge version:

~$ pipx install -f "git+https://github.com/snovvcrash/DivideAndScan.git"
~$ das

For debbugging purposes you can set up a dev environment with poetry:

~$ git clone https://github.com/snovvcrash/DivideAndScan
~$ cd DivideAndScan
~$ poetry install
~$ poetry run das

๐Ÿ“‘ Note: DivideAndScan uses sudo to run all the port scanners, so it will ask for the password when scanning commands are invoked.

Using from Docker

You can run DivideAndScan in a Docker container as follows:

~$ docker run -it --rm --name das -v `pwd`:/app snovvcrash/divideandscan

Since the tool requires some input data and produces some output data, you should specify your current working directory as the mount point at /app within the container. You may want to set an alias to make the base command shorter:

~$ alias das='docker run -it --rm --name das -v `pwd`:/app snovvcrash/divideandscan'
~$ das

How to Use

how-to-use.png

More examples coming soon at snovvcrash.rocks...

0. Preparations

Make a new directory to start DivideAndScan from. The tool will create subdirectories to store the output, so I recommend launching it from a clean directory to stay organized:

~$ mkdir divideandscan
~$ cd divideandscan

1. Filling the DB

Provide the add module a command for either Masscan or RustScan to discover open ports in a desired range.

โš ๏ธ Warning: please, make sure that you understand what you're doing, because nearly all port scanning tools can damage the system being tested if used improperly.

# Masscan example
~$ das add -db testdb masscan '--rate 1000 -iL hosts.txt --open -p1-65535'
# RustScan example
~$ das add -db testdb rustscan '-b 1000 -t 2000 -u 5000 -a hosts.txt -r 1-65535 -g --no-config'

When the module completes its work, a hidden directory .db is created in CWD containig the database file and raw scan results.

2. Targeted Scanning

Launch targeted Nmap scans with the scan module. You can adjust the scan surface with either -hosts or -ports option:

# Scan by hosts
~$ das scan -db testdb -hosts all -oA report1
~$ das scan -db testdb -hosts 192.168.1.0/24,10.10.13.37 -oA report1
~$ das scan -db testdb -hosts hosts.txt -oA report1
# Scan by ports
~$ das scan -db testdb -ports all -oA report2
~$ das scan -db testdb -ports 22,80,443,445 -oA report2
~$ das scan -db testdb -ports ports.txt -oA report2

To start Nmap simultaneously in multiple processes, specify the -parallel switch and set number of workers with the -proc option (if no value is provided, it will default to the number of processors on the machine):

~$ das scan -db testdb -hosts all -oA report -parallel [-proc 4]

The output format is selected with -oX, -oN, -oG and -oA options for XML+HTML formats, simple text format, grepable format and all formats respectively. When the module completes its work, a hidden directory .nmap is created in CWD containig Nmap raw scan reports.

Also, you can inspect the contents of the database with -show option before actually launching the scans:

~$ das scan -db testdb -hosts all -show

3 (Optional). Merging the Reports

In order to generate a report independently of the scan module, you should use the report module. It will search for Nmap raw scan reports in the .nmap directory and process and merge them based on either -hosts or -ports option:

# Merge outputs by hosts
~$ das report -db testdb -hosts all -oA report1
~$ das report -db testdb -hosts 192.168.1.0/24,10.10.13.37 -oA report1
~$ das report -db testdb -hosts hosts.txt -oA report1
# Merge outputs by ports
~$ das report -db testdb -ports all -oA report2
~$ das report -db testdb -ports 22,80,443,445 -oA report2
~$ das report -db testdb -ports ports.txt -oA report2

๐Ÿ“‘ Note: keep in mind that the report module does not search the DB when processing the -hosts or -ports options, but looks for Nmap raw reports directly in .nmap directory instead; it means that -hosts 127.0.0.1 argument value will be successfully resolved only if .nmap/127-0-0-1.* file exists, and -ports 80 argument value will be successfully resolved only if .nmap/port80.* file exists.

Help

usage: das [-h] {add,scan,report} ...

 -----------------------------------------------------------------------------------------------
|  ________  .__      .__    .___        _____              .____________                       |
|  \______ \ |__|__  _|__| __| _/____   /  _  \   ____    __| _/   _____/ ____ _____    ____    |
|   |    |  \|  \  \/ /  |/ __ |/ __ \ /  /_\  \ /    \  / __ |\_____  \_/ ___\\__  \  /    \   |
|   |    `   \  |\   /|  / /_/ \  ___//    |    \   |  \/ /_/ |/        \  \___ / __ \|   |  \  |
|  /_______  /__| \_/ |__\____ |\___  >____|__  /___|  /\____ /_______  /\___  >____  /___|  /  |
|          \/                 \/    \/        \/     \/      \/       \/     \/     \/     \/   |
|  {@snovvcrash}            {https://github.com/snovvcrash/DivideAndScan}             {vX.Y.Z}  |
 -----------------------------------------------------------------------------------------------

positional arguments:
  {add,scan,report}
    add              run a full port scan {masscan,rustscan} and add the output to DB
    scan             run targeted Nmap scans against hosts and ports from DB
    report           merge separate Nmap outputs into a single report in different formats

optional arguments:
  -h, --help         show this help message and exit

Psst, hey buddy... Wanna do some organized p0r7 5c4nn1n6?

ToDo

Support

If this tool has been useful for you, feel free to buy me a beer coffee!

beer.png

You might also like...
It helps to use fixtures in pytest.mark.parametrize

pytest-lazy-fixture Use your fixtures in @pytest.mark.parametrize. Installation pip install pytest-lazy-fixture Usage import pytest @pytest.fixture(p

An AWS Pentesting tool that lets you use one-liner commands to backdoor an AWS account's resources with a rogue AWS account - or share the resources with the entire internet ๐Ÿ˜ˆ
An AWS Pentesting tool that lets you use one-liner commands to backdoor an AWS account's resources with a rogue AWS account - or share the resources with the entire internet ๐Ÿ˜ˆ

An AWS Pentesting tool that lets you use one-liner commands to backdoor an AWS account's resources with a rogue AWS account - or share the resources with the entire internet ๐Ÿ˜ˆ

This file will contain a series of Python functions that use the Selenium library to search for elements in a web page while logging everything into a file

element_search with Selenium (Now With docstrings ๐Ÿ˜Ž ) Just to mention, I'm a beginner to all this, so it it's very possible to make some mistakes The

WIP SAT benchmarking tooling, written with only my personal use in mind.

SAT Benchmarking Some early work in progress tooling for running benchmarks and keeping track of the results when working on SAT solvers and related t

A command-line tool and Python library and Pytest plugin for automated testing of RESTful APIs, with a simple, concise and flexible YAML-based syntax

1.0 Release See here for details about breaking changes with the upcoming 1.0 release: https://github.com/taverntesting/tavern/issues/495 Easier API t

Turn any OpenAPI2/3 and Postman Collection file into an API server with mocking, transformations and validations.

Prism is a set of packages for API mocking and contract testing with OpenAPI v2 (formerly known as Swagger) and OpenAPI v3.x. Mock Servers: Life-like

To automate the generation and validation tests of COSE/CBOR Codes and it's base45/2D Code representations

To automate the generation and validation tests of COSE/CBOR Codes and it's base45/2D Code representations, a lot of data has to be collected to ensure the variance of the tests. This respository was established to collect a lot of different test data and related test cases of different member states in a standardized manner. Each member state can generate a folder in this section.

Test django schema and data migrations, including migrations' order and best practices.

django-test-migrations Features Allows to test django schema and data migrations Allows to test both forward and rollback migrations Allows to test th

An Instagram bot that can mass text users, receive and read a text, and store it somewhere with user details.
An Instagram bot that can mass text users, receive and read a text, and store it somewhere with user details.

Instagram Bot ๐Ÿค– July 14, 2021 Overview ๐Ÿ‘ A multifunctionality automated instagram bot that can mass text users, receive and read a message and store

Comments
  • Runtime error FileNotFoundError: [Errno 2] No such file or directory: '/root/.das/db/main.json'

    Runtime error FileNotFoundError: [Errno 2] No such file or directory: '/root/.das/db/main.json'

    I just cloned the repo and run the following commands :

    /tmp/DivideAndScan# alias das='docker run -it --rm --name das -v ~/.das:/root/.das -v pwd:/app divideandscan:latest'

    /tmp/DivideAndScan# das scan -hosts 185.165.116.100-185.165.116.200 -oA report1

    and here is the error that I faced :

    Traceback (most recent call last): File "/usr/local/bin/das", line 8, in sys.exit(main()) File "/usr/local/lib/python3.9/dist-packages/das/divideandscan.py", line 158, in main sr = ScanRun(str(P), args.hosts, args.ports) File "/usr/local/lib/python3.9/dist-packages/das/scan.py", line 35, in init self.db = TinyDB(db_path) File "/usr/local/lib/python3.9/dist-packages/tinydb/database.py", line 94, in init self._storage: Storage = storage(*args, **kwargs) File "/usr/local/lib/python3.9/dist-packages/tinydb/storages.py", line 102, in init touch(path, create_dirs=create_dirs) File "/usr/local/lib/python3.9/dist-packages/tinydb/storages.py", line 31, in touch with open(path, 'a'): FileNotFoundError: [Errno 2] No such file or directory: '/root/.das/db/main.json'

    opened by neginsadeghi 1
  • Bump cryptography from 38.0.2 to 38.0.3

    Bump cryptography from 38.0.2 to 38.0.3

    Bumps cryptography from 38.0.2 to 38.0.3.

    Changelog

    Sourced from cryptography's changelog.

    38.0.3 - 2022-11-01

    
    * Updated Windows, macOS, and Linux wheels to be compiled with OpenSSL 3.0.7,
      which resolves *CVE-2022-3602* and *CVE-2022-3786*.
    

    .. _v38-0-2:

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump urllib3 from 1.26.4 to 1.26.5

    Bump urllib3 from 1.26.4 to 1.26.5

    Bumps urllib3 from 1.26.4 to 1.26.5.

    Release notes

    Sourced from urllib3's releases.

    1.26.5

    :warning: IMPORTANT: urllib3 v2.0 will drop support for Python 2: Read more in the v2.0 Roadmap

    • Fixed deprecation warnings emitted in Python 3.10.
    • Updated vendored six library to 1.16.0.
    • Improved performance of URL parser when splitting the authority component.

    If you or your organization rely on urllib3 consider supporting us via GitHub Sponsors

    Changelog

    Sourced from urllib3's changelog.

    1.26.5 (2021-05-26)

    • Fixed deprecation warnings emitted in Python 3.10.
    • Updated vendored six library to 1.16.0.
    • Improved performance of URL parser when splitting the authority component.
    Commits
    • d161647 Release 1.26.5
    • 2d4a3fe Improve performance of sub-authority splitting in URL
    • 2698537 Update vendored six to 1.16.0
    • 07bed79 Fix deprecation warnings for Python 3.10 ssl module
    • d725a9b Add Python 3.10 to GitHub Actions
    • 339ad34 Use pytest==6.2.4 on Python 3.10+
    • f271c9c Apply latest Black formatting
    • 1884878 [1.26] Properly proxy EOF on the SSLTransport test suite
    • See full diff in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
Releases(v0.2.0)
Owner
snovvcrash
Penetration tester
snovvcrash
Pynguin, The PYthoN General UnIt Test geNerator is a test-generation tool for Python

Pynguin, the PYthoN General UnIt test geNerator, is a tool that allows developers to generate unit tests automatically.

Chair of Software Engineering II, Uni Passau 997 Jan 06, 2023
Statistical tests for the sequential locality of graphs

Statistical tests for the sequential locality of graphs You can assess the statistical significance of the sequential locality of an adjacency matrix

2 Nov 23, 2021
The best, free, all in one, multichecking, pentesting utility

The best, free, all in one, multichecking, pentesting utility

Mickey 58 Jan 03, 2023
Rerun pytest when your code changes

A simple watcher for pytest Overview pytest-watcher is a tool to automatically rerun pytest when your code changes. It looks for the following events:

Olzhas Arystanov 74 Dec 29, 2022
buX Course Enrollment Automation

buX automation BRACU - buX course enrollment automation Features: Automatically enroll into multiple courses at a time. Find courses just entering cou

Mohammad Shakib 1 Oct 06, 2022
Whatsapp messages bulk sender using Python Selenium.

Whatsapp Sender Whatsapp Sender automates sending of messages via Whatsapp Web. The tool allows you to send whatsapp messages in bulk. This program re

Yap Yee Qiang 3 Jan 23, 2022
The Social-Engineer Toolkit (SET) repository from TrustedSec - All new versions of SET will be deployed here.

๐Ÿ’ผ The Social-Engineer Toolkit (SET) ๐Ÿ’ผ Copyright 2020 The Social-Engineer Toolkit (SET) Written by: David Kennedy (ReL1K) @HackingDave Company: Trust

trustedsec 8.4k Dec 31, 2022
Compiles python selenium script to be a Window's executable

Problem Statement Setting up a Python project can be frustrating for non-developers. From downloading the right version of python, setting up virtual

Jerry Ng 8 Jan 09, 2023
A small faัade for the standard python mocker library to make it user-friendly

unittest-mocker Inspired by the pytest-mock, but written from scratch for using with unittest and convenient tool - patch_class Installation pip insta

Vertliba V.V. 6 Jun 10, 2022
MultiPy lets you conveniently keep track of your python scripts for personal use or showcase by loading and grouping them into categories. It allows you to either run each script individually or together with just one click.

MultiPy About MultiPy is a graphical user interface built using Dear PyGui Python GUI Framework that lets you conveniently keep track of your python s

56 Oct 29, 2022
A suite of benchmarks for CPU and GPU performance of the most popular high-performance libraries for Python :rocket:

A suite of benchmarks for CPU and GPU performance of the most popular high-performance libraries for Python :rocket:

Dion Hรคfner 255 Jan 04, 2023
A simple serverless create api test repository. Please Ignore.

serverless-create-api-test A simple serverless create api test repository. Please Ignore. Things to remember: Setup workflow Change Name in workflow e

Sarvesh Bhatnagar 1 Jan 18, 2022
Getting the most out of your hobby servo

ServoProject by Adam Bรคckstrรถm Getting the most out of your hobby servo Theory The control system of a regular hobby servo looks something like this:

209 Dec 20, 2022
This repository contnains sample problems with test cases using Cormen-Lib

Cormen Lib Sample Problems Description This repository contnains sample problems with test cases using Cormen-Lib. These problems were made for the pu

Cormen Lib 3 Jun 30, 2022
FFPuppet is a Python module that automates browser process related tasks to aid in fuzzing

FFPuppet FFPuppet is a Python module that automates browser process related tasks to aid in fuzzing. Happy bug hunting! Are you fuzzing the browser? G

Mozilla Fuzzing Security 24 Oct 25, 2022
Mixer -- Is a fixtures replacement. Supported Django, Flask, SqlAlchemy and custom python objects.

The Mixer is a helper to generate instances of Django or SQLAlchemy models. It's useful for testing and fixture replacement. Fast and convenient test-

Kirill Klenov 871 Dec 25, 2022
Travel through time in your tests.

time-machine Travel through time in your tests. A quick example: import datetime as dt

Adam Johnson 373 Dec 27, 2022
A pytest plugin to skip `@pytest.mark.slow` tests by default.

pytest-skip-slow A pytest plugin to skip @pytest.mark.slow tests by default. Include the slow tests with --slow. Installation $ pip install pytest-ski

Brian Okken 19 Jan 04, 2023
Automated mouse clicker script using PyAutoGUI and Typer.

clickpy Automated mouse clicker script using PyAutoGUI and Typer. This app will randomly click your mouse between 1 second and 3 minutes, to prevent y

Joe Fitzgibbons 0 Dec 01, 2021
Hypothesis is a powerful, flexible, and easy to use library for property-based testing.

Hypothesis Hypothesis is a family of testing libraries which let you write tests parametrized by a source of examples. A Hypothesis implementation the

Hypothesis 6.4k Jan 05, 2023