ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete

Overview

ShadowClone

ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete.

You can make full use of the Free Tiers provided by cloud providers and supercharge your mundane cli tools with shadow clone jutsu (Naruto style)!

Installation

Please visit the wiki for installation and intial configuration instructions

Usage

⚡ python shadowclone.py -h
usage: shadowclone.py [-h] -i INPUT [-s SPLITNUM] [-o OUTPUT] -c COMMAND

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
  -s SPLITNUM, --split SPLITNUM
                        Number of lines per chunk of file
  -o OUTPUT, --output OUTPUT
  -c COMMAND, --command COMMAND
                        command to execute

How it works

We create a container image during the initial setup and register it as a runtime for our function in AWS/GCP/Azure whatever. When you execute ShadowClone on your computer, instances of that container are activated automatically and are only active for the duration of its execution. How many instances to activate is dynamically decided at runtime depending on the size of the input file provided and the split factor. The input is then split into chunks and equally distributed between all the instances to execute in parallel. For example, if your input file has 10,000 lines and you set the split factor to 100 lines, then it will be split into 100 chunks of 100 lines each and 100 instances will be run in parallel!

Features

  • Extremely fast
  • No need to maintain a VPS (or a fleet of it :))
  • Costs almost nothing per month
    • Compatible with free tiers of most cloud services
  • Cloud agnostic
    • Same script works with AWS, GCP, Azure etc.
  • Supports upto 1000 parallel invocations
  • Dynamically decide the number of invocations
  • Run any tool in parallel on the cloud
  • Pipe output to other tools

Comparison

This tool was inspired by the awesome Axiom and Fleex projects and goes beyond the concept of VPS for running the tools by using serverless functions and containers.

Features Axiom/Fleex ShadowClone
Instances 10-100s* 1000s
Cost Per instance/per minute Mostly Free**
Startup Time 4-5 minutes 2-3 seconds
Idle Cost $++ Free
On Demand Scalability No

*Most cloud providers do not allow spinning up too many instances by default, so you are limited to around 10-15 instances at max. You have to make a request to the support to increase this number.

** AWS & Azure allow 1 million invocations per month for free. Google allows 2 million invocations per month for free. You will be charged only if you go above these limits

Demo

DNS Bruteforcing using a 43mb file - 34 seconds

asciicast

Running httpx on 94K subdomains in 1 min

asciicast

References

Lithops documentation

Free Tiers

Cloud Provider Free Allowance Link
Google Functions 2 Million invocations, 400,000 GB-seconds per month Google Cloud Free Program
AWS Lambda 1 Million invocations, Up to 3.2 million seconds of compute time per month Free Cloud Computing Services - AWS Free Tier
Azure Functions 1 Million invocations Microsoft Azure Free Services

Obviously, you can make any number of function invocations per month. The table above only shows how many invocations are free.

Similar Tools

Disclaimer

This tool is designed as a proof-of-concept implementation and it's usage is intended for educational purpose only. Usage for attacking targets without prior mutual consent is illegal. It's the end user's responsibility to obey all applicable local, state and federal laws. Developers assume no liability and are not responsible for any misuse or damage caused by this program.

Bot per controllare la disponibilità di appuntamenti per la vaccinazione Covid-19 in Veneto

VaxBot9000 Prerequisites Python 3.9 Poetry latest version of geckodriver Firefox Setup poetry install Copy config.sample.toml to config.toml and edit

Augusto Zanellato 5 Jun 13, 2021
Currency Merger is a simple tool for joining values in different currencies

Currency Merger Description Currency Merger is a simple tool for joining values in different currencies. For example, if I have money in USD, EUR, BRL

Arthur Diniz 1 Feb 08, 2022
Binance leverage futures Hook

Simple binance futures Attention Just use leverage. The fee difference between futures and spot is not considered. For example, funding rate, etc. Onl

Adriance 26 Aug 27, 2022
Python wrapper for GitHub API v3

Squeezeit - Python CSS and Javascript minifier Copyright (C) 2011 Sam Rudge This program is free software: you can redistribute it and/or modify it un

David Medina 207 Oct 24, 2022
Programa de código abierto para probar el API de Bitso, el exchange más importante de América Latina.

Bitso Semiautomático Programa de código abierto para probar el API de Bitso, el exchange más importante de América Latina. Desarrollador Fernando Mire

Fernando Mireles 17 Dec 07, 2022
Yok bentar lagi update Premium :( DI FOLLOW YA GUYS

SIMBF + PREMIUM PRINTAH PENGINSTALAN ON TERMUX $ pkg update && upgrade $ termux-setup-storage $ pkg install python $ pkg install git $ pip install bs4

Jeeck 21 Jan 14, 2022
The Most advanced and User-Friendly Google Collab NoteBook to download Torrent directly to Google Drive with File or Magnet Link support and with added protection of Timeout Preventer.

Torrent To Google Drive (UI Added! 😊 ) A Simple and User-Friendly Google Collab Notebook with UI to download Torrent to Google Drive using (.Torrent)

Dr.Caduceus 33 Aug 16, 2022
Send song lyrics to iMessage users using the Genius lyrics API

pyMessage Send song lyrics to iMessage users using the Genius lyrics API. Setup 1.) Open the main.py file, and add your API key on line 7. 2.) Install

therealkingnull 1 Jan 23, 2022
This automation protect against subdomain takeover on AWS env which also send alerts on slack.

AWS_Subdomain_Takeover_Detector Purpose The purpose of this automation is to detect misconfigured Route53 entries which are vulnerable to subdomain ta

Puneet Kumar Maurya 8 May 18, 2022
A script to forward mass number of media to another group/channel. Heroku deploy

Telegram Forward Script 😇 This is a Script to Forward Large Number of Files to Another Telegram Channel. Star එකක් දාල fork එකක් ගහපියව් 🥴 If You Tr

Anjana Madu 17 Oct 21, 2022
Python script using Twitter API to change user banner to see 100DaysOfCode process.

100DaysOfCode - Automatic Banners 👩‍💻 Adds a number to your twitter banner indicating the number of days you have in the #100DaysOfCode challenge Se

Ingrid Echeverri 10 Jul 06, 2022
聚合空间测绘搜索(Fofa,Zoomeye,Quake,Shodan,Censys,BinaryEdge)

#Search-Tools Search-Tools集合比较常见的网络空间探测引擎 Fofa,Zoomeye,Quake,Shodan,Censys,BinaryEdge 简单说明 ICO搜索目前只有Fofa,Shodan,Quake支持 代理设置是防止在API请求过于频繁,或者在实战中,好多红队打

311 Dec 16, 2022
Library for working with QIWI API.

Library for working with QIWI API.

qxtony 2 Apr 26, 2022
A custom discord bot maker in python

custom-discord-bot-maker Sorry for using Translator. Each description may be inaccurate. how to use 1. Make new application at https://discord.com/dev

2 Nov 29, 2021
A simple test repo created following docker docs.

docker_sampleRepo A simple test repo created following docker docs. Link to docs: https://docs.docker.com/language/python/develop/ Other links: https:

Suraj Verma 2 Sep 16, 2022
Ivan Telegram Userbot with python

Riviani Ramadhan Ivan-Ubot Pada Dasarnya Ivan-Ubot adalah userbot Telegram modular yang berjalan di Python3 dengan database sqlalchemy. Berbasis Paper

1 Oct 29, 2021
An unofficial python wrapper for the comdirect API for private consumers.

An unofficial python wrapper for the comdirect API for private consumers.

Alexander Knittel 18 Nov 30, 2022
Python Bot that attends classes, answers polls, and then again waits for classes to start.

LPU_myclass_Bot LPU_myclass_Bot is a Python bot that waits for class to start, attends class, answers polls, and then again waits for another class to

Saurabh Kumar 6 Apr 07, 2022
An asyncio Python wrapper around the Discord API, forked off of Rapptz's Discord.py.

Novus A modern, easy to use, feature-rich, and async ready API wrapper for Discord written in Python. A full fork of Rapptz's Discord.py library, with

Voxel Fox 60 Jan 03, 2023
An advanced crypto trading bot written in Python

Jesse Jesse is an advanced crypto trading framework which aims to simplify researching and defining trading strategies. Why Jesse? In short, Jesse is

Jesse 4.4k Jan 09, 2023