ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete

Overview

ShadowClone

ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete.

You can make full use of the Free Tiers provided by cloud providers and supercharge your mundane cli tools with shadow clone jutsu (Naruto style)!

Installation

Please visit the wiki for installation and intial configuration instructions

Usage

⚡ python shadowclone.py -h
usage: shadowclone.py [-h] -i INPUT [-s SPLITNUM] [-o OUTPUT] -c COMMAND

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
  -s SPLITNUM, --split SPLITNUM
                        Number of lines per chunk of file
  -o OUTPUT, --output OUTPUT
  -c COMMAND, --command COMMAND
                        command to execute

How it works

We create a container image during the initial setup and register it as a runtime for our function in AWS/GCP/Azure whatever. When you execute ShadowClone on your computer, instances of that container are activated automatically and are only active for the duration of its execution. How many instances to activate is dynamically decided at runtime depending on the size of the input file provided and the split factor. The input is then split into chunks and equally distributed between all the instances to execute in parallel. For example, if your input file has 10,000 lines and you set the split factor to 100 lines, then it will be split into 100 chunks of 100 lines each and 100 instances will be run in parallel!

Features

  • Extremely fast
  • No need to maintain a VPS (or a fleet of it :))
  • Costs almost nothing per month
    • Compatible with free tiers of most cloud services
  • Cloud agnostic
    • Same script works with AWS, GCP, Azure etc.
  • Supports upto 1000 parallel invocations
  • Dynamically decide the number of invocations
  • Run any tool in parallel on the cloud
  • Pipe output to other tools

Comparison

This tool was inspired by the awesome Axiom and Fleex projects and goes beyond the concept of VPS for running the tools by using serverless functions and containers.

Features Axiom/Fleex ShadowClone
Instances 10-100s* 1000s
Cost Per instance/per minute Mostly Free**
Startup Time 4-5 minutes 2-3 seconds
Idle Cost $++ Free
On Demand Scalability No

*Most cloud providers do not allow spinning up too many instances by default, so you are limited to around 10-15 instances at max. You have to make a request to the support to increase this number.

** AWS & Azure allow 1 million invocations per month for free. Google allows 2 million invocations per month for free. You will be charged only if you go above these limits

Demo

DNS Bruteforcing using a 43mb file - 34 seconds

asciicast

Running httpx on 94K subdomains in 1 min

asciicast

References

Lithops documentation

Free Tiers

Cloud Provider Free Allowance Link
Google Functions 2 Million invocations, 400,000 GB-seconds per month Google Cloud Free Program
AWS Lambda 1 Million invocations, Up to 3.2 million seconds of compute time per month Free Cloud Computing Services - AWS Free Tier
Azure Functions 1 Million invocations Microsoft Azure Free Services

Obviously, you can make any number of function invocations per month. The table above only shows how many invocations are free.

Similar Tools

Disclaimer

This tool is designed as a proof-of-concept implementation and it's usage is intended for educational purpose only. Usage for attacking targets without prior mutual consent is illegal. It's the end user's responsibility to obey all applicable local, state and federal laws. Developers assume no liability and are not responsible for any misuse or damage caused by this program.

A Simple Telegram Bot To Download And Upload Files

AquaDLBot ➠ I Can Download And Upload files To Telegram DEMO Copyright (C) 2020-2026 by [ema

Asia Argento 8 Feb 15, 2022
Production Ontology Merging (PrOM) Framework

Production Ontology Merging (PrOM) Framework OWL 2 DL ontology merging framework tailored to the production domain Features preprocessing: translation

4 Nov 02, 2022
Discord.py(disnake) selfbot

Zzee selfbot Discord.py selfbot Version: 1.0 ATTENTION! we are not responsible for your discord account! this program violates the ToS discord rules!

1 Jan 10, 2022
Automatically kick deleted accounts

AntiDeletedAccountsBot (ADAB) Automatically kick deleted accounts Based on uniborg, a pluggable asyncio Telegram userbot based on Telethon. Installati

Qwerty-Space 34 Jan 02, 2023
A WhatsApp Crashing Tool for Termux

CrashW A WhatsApp Crashing Tool For Termux Users Installing : apt update && apt upgrade -y pkg install python3 pkg install git git clone git://gith

Gokul Mahato 20 Dec 27, 2022
Example of a discord bot in Python

discordbot.py Example of a discord bot in Python Requirements Python 3.8 or higher Discord Bot Setting Up Clone this repo or download the files Rename

Debert Jamie 1 Oct 23, 2021
BroBot's files, code and tester.

README - BroBOT Made by Rohan Chaturvedi [email protected] DISCLAIMER: Th

1 Jan 09, 2022
GitHub action to deploy serverless functions to YandexCloud

YandexCloud serverless function deploy action Deploy new serverless function version (including function creation if it does not exist). Inputs yc_acc

Много Лосося 4 Apr 10, 2022
wrapper for facebook messenger

pyfacebook pyfacebook library for python. Requirements common Help Got a question? File a GitHub issue. Contributing Bug Reports & Feature Requests Pl

Luis Mayta 3 Nov 12, 2021
Blankly - 🚀 💸 Trade stocks, cryptos, and forex w/ one package. Easily build, backtest, trade, and deploy across exchanges in a few lines of code.

💨 Rapidly build and deploy quantitative models for stocks, crypto, and forex 🚀 View Docs · Our Website · Join Our Newsletter · Getting Started Why B

Blankly Finance 1.4k Jan 03, 2023
A Django-style ORM idea for manipulating Google Datastore entities

No SeiQueLa ORM EM DESENVOLVIMENTO Uma ideia de ORM no estilo do Django para manipular entidades do Google Datastore. Montando seu modelo: from noseiq

Geraldo Castro 16 Nov 01, 2022
A python oriented telegram with API of yobit.net

YoBit-BTC A python oriented telegram bot with API of https://yobit.net/ Developed By @riz4d What is Yobit? ➪ YoBit is a cryptocurrency exchange that w

Muhammed Rizad 6 Apr 02, 2022
ARKHAM X GOD MULTISPAM BOT

ARKHAM-X-GOD-MULTISPAM-BOT 𝗗𝗘𝗣𝗟𝗢𝗬 𝗨𝗣𝗧𝗢 30 𝗕𝗢𝗧𝗦 𝗜𝗡 𝗔 𝗦𝗜𝗡𝗚𝗟?

ArkhamXGod 2 Jan 08, 2022
Data and a Twitter bot for the EPA's DOCUMERICA (1972-1977) program.

documerica This repository holds JSON(L) artifacts and a few scripts related to managing archival data from the EPA's DOCUMERICA program. Contents: Ma

William Woodruff 2 Oct 27, 2021
A repository for 8G server's discord bot

8G Discord-Bot A general-purpose discord bot for the 8G Discord-Server To setup: Create a new file called secrets.py and make it look like this TOKEN=

1 Jan 12, 2022
A minimalistic library designed to provide native access to YNAB data from Python

pYNAB A minimalistic library designed to provide native access to YNAB data from Python. Install The simplest way is to install the latest version fro

Ivan Smirnov 92 Apr 06, 2022
Astro Bot With Golang

Astro-Bot Features: Astronomy Picture of the day Horoscope People In Space How we built it Our team was broken, one person didn't do anything the othe

Vaarun Sinha 1 Nov 21, 2021
Discord-Bot - Bot using nextcord for beginners

Discord-Bot Bot using nextcord for beginners! Requirements: 1 :- Install nextcord by typing "pip install nextcord" Thats it! You can use this code any

INFINITE_. 3 Jan 10, 2022
Drcom-pt-client - Drcom Pt version client with refresh timer

drcom-pt-client Drcom Pt version client with refresh timer Dr.com Pt版本客户端 可用于网页认

4 Nov 16, 2022
A Telegram bot that can stream Telegram files to users over HTTP.

T.ME_FILE_TO_LINK Hi iam a file to link bot....best Bot telegram Telegram File To Link Generation Bot A Telegram bot that can stream Telegram files to

1 Oct 24, 2021