ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete

Overview

ShadowClone

ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete.

You can make full use of the Free Tiers provided by cloud providers and supercharge your mundane cli tools with shadow clone jutsu (Naruto style)!

Installation

Please visit the wiki for installation and intial configuration instructions

Usage

⚡ python shadowclone.py -h
usage: shadowclone.py [-h] -i INPUT [-s SPLITNUM] [-o OUTPUT] -c COMMAND

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
  -s SPLITNUM, --split SPLITNUM
                        Number of lines per chunk of file
  -o OUTPUT, --output OUTPUT
  -c COMMAND, --command COMMAND
                        command to execute

How it works

We create a container image during the initial setup and register it as a runtime for our function in AWS/GCP/Azure whatever. When you execute ShadowClone on your computer, instances of that container are activated automatically and are only active for the duration of its execution. How many instances to activate is dynamically decided at runtime depending on the size of the input file provided and the split factor. The input is then split into chunks and equally distributed between all the instances to execute in parallel. For example, if your input file has 10,000 lines and you set the split factor to 100 lines, then it will be split into 100 chunks of 100 lines each and 100 instances will be run in parallel!

Features

  • Extremely fast
  • No need to maintain a VPS (or a fleet of it :))
  • Costs almost nothing per month
    • Compatible with free tiers of most cloud services
  • Cloud agnostic
    • Same script works with AWS, GCP, Azure etc.
  • Supports upto 1000 parallel invocations
  • Dynamically decide the number of invocations
  • Run any tool in parallel on the cloud
  • Pipe output to other tools

Comparison

This tool was inspired by the awesome Axiom and Fleex projects and goes beyond the concept of VPS for running the tools by using serverless functions and containers.

Features Axiom/Fleex ShadowClone
Instances 10-100s* 1000s
Cost Per instance/per minute Mostly Free**
Startup Time 4-5 minutes 2-3 seconds
Idle Cost $++ Free
On Demand Scalability No

*Most cloud providers do not allow spinning up too many instances by default, so you are limited to around 10-15 instances at max. You have to make a request to the support to increase this number.

** AWS & Azure allow 1 million invocations per month for free. Google allows 2 million invocations per month for free. You will be charged only if you go above these limits

Demo

DNS Bruteforcing using a 43mb file - 34 seconds

asciicast

Running httpx on 94K subdomains in 1 min

asciicast

References

Lithops documentation

Free Tiers

Cloud Provider Free Allowance Link
Google Functions 2 Million invocations, 400,000 GB-seconds per month Google Cloud Free Program
AWS Lambda 1 Million invocations, Up to 3.2 million seconds of compute time per month Free Cloud Computing Services - AWS Free Tier
Azure Functions 1 Million invocations Microsoft Azure Free Services

Obviously, you can make any number of function invocations per month. The table above only shows how many invocations are free.

Similar Tools

Disclaimer

This tool is designed as a proof-of-concept implementation and it's usage is intended for educational purpose only. Usage for attacking targets without prior mutual consent is illegal. It's the end user's responsibility to obey all applicable local, state and federal laws. Developers assume no liability and are not responsible for any misuse or damage caused by this program.

Framework for Telegram users and chats investigating.

telegram_scan Fantastic and full featured framework for Telegram users and chats investigating. Prerequisites: pip3 install pyrogram; get api_id and a

71 Dec 17, 2022
Python API for British Geological Survey magnetic field calculator

Magnetic field calculator Python API for British Geological Survey magnetic field calculator. Description This project magnetic field calculator. It u

Filip Š 3 Mar 11, 2022
Automatically deploy freqtrade to a remote Docker host and auto update strategies.

Freqtrade Automatically deploy freqtrade to a remote Docker host and auto update strategies. I've been using it to automatically deploy to vultr, but

p-zombie 109 Jan 07, 2023
A python bot that scrapes free udemy coupons and sends them to Telegram.

About: A python telegram bot that scrapes information about fresh free Udemy coupons content from couponscorpion.com and sends it to teleram channel h

Irina Gayday 1 Dec 19, 2021
Say "good morning" on Discord, in batch, one-click.

🌞 gm Good Morning! Usage Simply copy the channel_list to gm.py and fill authorization_list with authorization token(s). Enjoy. Authorization Please r

e 3 Nov 18, 2022
This is a scalable system that reads messages from public Telegram channels using Telethon and stores the data in a PostgreSQL database.

This is a scalable system that reads messages from public Telegram channels using Telethon and stores the data in a PostgreSQL database. Its original intention is to monitor cryptocurrency related ch

Greg 3 Jun 07, 2022
A python package to easy the integration with Direct Online Pay (Mpesa, TigoPesa, AirtelMoney, Card Payments)

A python package to easy the integration with Direct Online Pay (DPO) which easily allow you easily integrate with payment options once without having to deal with each of them individually;

Jordan Kalebu 2 Nov 25, 2021
Discord Rich Presence for Team Fortress 2

TF2 Rich Presence Discord Rich Presence for Team Fortress 2 Detects current game state, queue info, playtime, and more Configurable, reliable, and per

Kataiser 33 Dec 31, 2022
AWS SQS event redrive Lambda

This repository contains the Lambda function to redrive sqs events from source to destination queue while controlling maxRetry per event.

1 Oct 19, 2021
A simple telegram bot to help you to remove forward tag from post from any messages . Maded in python3 using @Pyrogram . Developed by @Kunal-Diwan

Frwd-Tag-Remover Telegram Bot to Remove forward tag from any Post . If you need any more modes in repo or If you find out any bugs, mention in @Develo

Kunal Diwan 2 Oct 14, 2022
A Webhook spammer For Python

Webhooker Optimizations Asynchronous Fast & Efficient Multi Tasked Usage Put high threads/tasks for maximum impact Webhook must be valid Proof of conc

andria 1 Dec 20, 2021
Shred your reddit comment and post history

trasheddit Shred your reddit comment and post history (x89/Shreddit replacement) Usage Simple Example Download trasheddit: git clone https://github.co

Elly 2 Jan 05, 2022
Clash of Clans developer unofficial api Wrapper to generate ip based token

Clash of Clans developer unofficial api Wrapper to generate ip based token

Aryan Vikash 6 Apr 01, 2022
this is a telegram torrent bot

owner of this repo :- AYUSH contact me :- AYUSH Slam Mirror Bot This is a telegram bot writen in python for mirroring files on the internet to our bel

AYUSH 237 Dec 16, 2021
AminoLab Library For AminoApps using aminoapps.com/api

AminoLab AminoLab Api For AminoApps using aminoapps.com/api Installing pip install AminoLab Example #Login import AminoLab client = AminoLab.Client()

10 Sep 26, 2022
Discord.py Bot Series With Python

Discord.py Bot Series YouTube Playlist: https://www.youtube.com/playlist?list=PL9nZZVP3OGOAx2S75YdBkrIbVpiSL5oc5 Installation pip install -r requireme

Step 1 Dec 17, 2021
A hilarious program that translates text to Google's

Bork, bork, bork! A hilarious program that translates text in a file to Google's "bork bork bork' language. How to use Download the main.py file. Chan

The Algorithmic 1 Dec 17, 2021
PyDiscord, a maintained fork of discord.py, is a python wrapper for the Discord API.

discord.py A modern, easy to use, feature-rich, and async ready API wrapper for Discord written in Python. The Future of discord.py Please read the gi

Omkaar 1 Jan 16, 2022
Unofficial instagram API, give you access to ALL instagram features (like, follow, upload photo and video and etc)! Write on python.

Instagram-API-python Unofficial Instagram API to give you access to ALL Instagram features (like, follow, upload photo and video, etc)! Written in Pyt

Vladimir Bezrukov 1 Nov 19, 2021