It's a simple python script to take backup of directories (compressing) then the same to move your mentioned S3 bucket with the help of AWS IAM User.

Overview

Directory Backup Moved to S3 (Pyscript)

Build


Description

Here it's a python script that needs to use this script simply create a directory backup and moved that compressed backup file to a configured S3 bucket with the help of python script and AWS IAM User with S3 full access. So, let's roll down.


Feature

  • Easy to configure for anyone
  • It generates the directory compressed format
  • Which one we entered directory converted to a backup/compressed form to S3
  • All the steps I have provided including AWS IAM User and S3 Bucket Creation

Modules used


Pre-Requests

  • Basic Knowledge of python
  • Basic Knowledge of AWS IAM, S3 service
  • Need to change your IAM user creds and bucket name at var.py

IAM User Creation steps (with screenshot)

  1. log into your AWS account as a root user and go to IAM user
  2. goto Access Managment >> Users alt_txt
  3. Click Add User (top right corner) alt_txt
  4. Enter any username as you like and Choose "Programmatic access" >> Click Next Permissions alt_txt
  5. Set Permissions >> Select "Attach existing policies directly" >> Choose "AmazonS3FullAccess" >> Click Next Tags alt_txt
  6. Add Tags(Optional) >> Enter a key and value as you like either you can leave as blank alt_txt
  7. Review your user details and click "Create User" alt_txt
  8. Store your credentials to your local alt_txt

Reference URL:: IAM User creation article


S3 Bucket Creation (with screenshot)

  1. Go to S3 > Click Create Bucket alt_txt
  2. Any bucket name as you wish and then please enable versioning (that you upload same file multiple times or modified versions uploaded that the S3 stored as a version bases like Git) alt_txt
  3. Click create bucket

alt_txt

Reference URL:: Creating S3 bucket please use this doc and you can secure your bucket with IAM user using S3 bucket policy


Pre-Requested (Dependency packages)

yum install -y git
yum install -y python3
yum install -y python3-pip

How to get

git clone https://github.com/yousafkhamza/backup-to-s3-pyscript.git
cd backup-to-s3-pyscript
pip3 install -r requirements.txt

Change your creds and bucket name in at var.py file

Command to run the script::

python3 to-S3.py Python/test
# Pyhon/test               <------------ Directory to take backup and move the backup to S3

Output be like

$ $ python3 to-S3.py Python/httpd
Start to Upload that the httpd.tar.gz your S3 Bucket yousaf-test
Backup successfully uploaded to your S3 bucket yousaf-test

View of S3 bucket

Screenshot alt_txt


Behind the code

vim to-S3.py

import boto3
import tarfile
import os
import sys
import posixpath
import var
from boto3.s3.transfer import S3Transfer

directory = sys.argv[1]
dirname = os.path.split(directory)[-1]
if directory.endswith ('/'):
    print ('Please remove / after the directory path you have entered')
else:
    if posixpath.isdir(directory):
        tarname = '/tmp/{}.tar.gz'.format(dirname)
        tar = tarfile.open(tarname,'w:gz')
        tar.add(directory)
        tar.close()
        print('Start to Upload that the', dirname+'.tar.gz', 'your S3 Bucket', var.BUCKET_NAME)

# S3 uploading started
        client = boto3.client('s3', aws_access_key_id=var.AWS_ACCESS_KEY_ID,aws_secret_access_key=var.AWS_SECRET_ACCESS_KEY)
        transfer = S3Transfer(client)
        transfer.upload_file(tarname, var.BUCKET_NAME, 'backup/{}.tar.gz'.format(dirname))
        print('Backup succesfully uploaded to your S3 bucket', var.BUCKET_NAME)
    
# Remove temporary backup from local
        os.remove(tarname)
    else:
        print('Please enter a valid directory path')

var.py

AWS_ACCESS_KEY_ID = 'AKTBRI2N5IAT3ND'             <--------------   Replace your acess key
AWS_SECRET_ACCESS_KEY = 'asUNmMPrC99HoiiQPjehetFtVsPv'          <--------- Replace your secret key
BUCKET_NAME = 'yousaf-test'                <----------- Replace your bucket name

Conclusion

It's a simple python script to take backup of directories (compressing) then the same to move your mentioned S3 bucket with the help of AWS IAM User. this script may be helpful who had face issues moving backups to S3 so it might be useful for cloud engineers.

⚙️ Connect with Me


Owner
Yousaf K Hamza
Junior DevOps Enginner
Yousaf K Hamza
Discord-shell - Reverse shell accessible via discord.py bot

Discord-shell reverse shell on discord bot. (STILL IN DEVELOPMENT) Installation

Skultz 6 Oct 21, 2022
Python client for Midea dhumidifier

This is a library that allows communication with Midea dehumidifier appliances via the local area network. midea-beautiful-dehumidifier This library a

Nenad Bogojevic 42 Dec 22, 2022
Discord.py(disnake) selfbot

Zzee selfbot Discord.py selfbot Version: 1.0 ATTENTION! we are not responsible for your discord account! this program violates the ToS discord rules!

1 Jan 10, 2022
A Telegram Bot To Stream Videos in Telegram Voice Chat.

Video Stream X Bot Telegram bot project for streaming video on telegram video chat, powered by tgcalls and pyrogram Deploy to Heroku 👨‍🔧 The easy wa

Mⷨoͦns͛ᴛⷮeͤrͬ Zeͤrͬoͦ 13 Dec 05, 2022
A super awesome Twitter API client for Python.

birdy birdy is a super awesome Twitter API client for Python in just a little under 400 LOC. TL;DR Features Future proof dynamic API with full REST an

Inueni 259 Dec 28, 2022
Simple Bot With Python 3.8+ For Converstaion Your Media

Media-Conversation Simple Bot With Python 3.8+ For Converstaion Your Media

Farzin 2 Dec 06, 2021
A Powerful, Smart And Simple Userbot In Pyrogram.

Eagle-USERBOT 🇮🇳 A Powerful, Smart And Simple Userbot In Pyrogram. Support 🚑 Inspiration & Credits Userge-X Userge Pokurt Pyrogram Code Owners Mast

Masterolic 1 Nov 28, 2021
Fun telegram bot =)

Recolor Bot About Fun telegram bot, that can change your hair color. Preparations Update package lists sudo apt-get update; Make sure Git and docker-c

Just Koala 4 Jul 09, 2022
A results generator and automatic token checker for Yandex Contest

Yandex contest Python checking tools A results generator and automatic token checker for Yandex Contest. Версия на русском языке Installation Clone th

Nikolay Chechulin 9 Dec 14, 2022
Automatically send commands to send Twitch followers to any Twitch account.

Automatically send commands to send Twitch followers to any Twitch account. You just need to be in a Twitch follow bot Discord server!

Thomas Keig 6 Nov 27, 2022
Resources for the AMLD 2022 workshop "DevOps on AWS"

MLOPS on AWS | AMLD 2022 This repository contains all the resources necessary to follow along and reproduce the workshop "MLOps on AWS: a Hands-On Tut

xtream 8 Jun 16, 2022
Working TikTok Username Auto-Claimer/Sniper/Swapper which will autoclaim username if it´s available

TikTok-AutoClaimer Working TikTok Username Auto-Claimer/Sniper/Swapper which will autoclaim username if it´s available Usage Python 3.6 or above is re

Kevin 18 Dec 08, 2022
VALORANT rank yoinker lets you retrieve the ranks and basic informations of everyone in the lobby, regardless of gamemode.

vRY VALORANT rank yoinker Retrieve the rank and basic information of everyone in the lobby, regardless of gamemode. Table of Contents Terms of Use Abo

Isaac Kenyon 270 Dec 30, 2022
Python wrapper for JeyyAPI

Async python wrapper for JeyyAPI

7 Dec 10, 2022
“ Hey there 👋 I'm Daisy „ AI based Advanced Group Management Bot Suit For All Your Needs ❤️.. Source Code of @Daisyxbot

Project still under heavy development Everything will be changed in the release “ Hey there 👋 I'm Daisy „ AI based Advanced telegram Group Management

TeamDaisyX 43 Nov 12, 2022
M3U Playlist for free TV channels

Free TV This is an M3U playlist for free TV channels around the World. Either free locally (over the air): Or free on the Internet: Plex TV Pluto TV P

Free TV 964 Jan 08, 2023
Python script to delete old / embarrassing tweets.

Delete Tweets Do you have hundreds of embarrassing tweets on your Twitter profile, that you tweeted over a decade ago as an innocent high schooler, th

Linda Zheng 9 Nov 26, 2022
This is to notify you via Discord whenever there is a new beacon.

BeaconNotifier-Discord This is to notify you via Discord whenever there is a new beacon. Make sure you have python3 installed Steps: Create a Discord

26 Dec 28, 2022
Declarative assertions for AWS

AWSsert AWSsert is a Python library providing declarative assertions about AWS resources to your tests. Installation Use the package manager pip to in

19 Jan 04, 2022
A napari plugin for visualising and interacting with electron cryotomograms

napari-subboxer A napari plugin for visualising and interacting with electron cryotomograms. Installation You can install napari-subboxer via pip: pip

3 Nov 25, 2021