Dumps to CSV all the resources in an organization's member accounts

Overview

AWS Org Inventory

Dumps to CSV all the resources in an organization's member accounts.

Set your environment's AWS_PROFILE and AWS_DEFAULT_REGION variables.

The AWS_PROFILE should be configured to use a role in the organization management account that can assume OrganizationAccountAccessRole in the member accounts.

Redirect the dumper's output to save the file.

The dumper uses Botocove to query each member account.

Why?

This tool fills in the gaps in AWS Config's inventory.

Sadly AWS Config supports only a subset of all the resource types you may have.

AWS Config's organization aggregators are great, but they may not update instantly with new resources.

Installation

The package is published to PyPy as aws-org-inventory, so you can install it with pip or anything equivalent.

pip install aws-org-inventory

Basic example

Configure environment:

export AWS_PROFILE=OrgMgmtRole
export AWS_DEFAULT_REGION=eu-west-1

Dump inventory of CloudWatch log groups:

aws-org-inventory logs describe_log_groups logGroups

Dump inventory of support cases:

aws-org-inventory support describe_cases cases

Dump inventory of EC2 key pairs:

aws-org-inventory ec2 describe_key_pairs KeyPairs

Try doing those with AWS Config!

General use

To derive arguments for other use cases, check the boto service documentation.

The value passed to the boto3.client method that would instantiate a client for the service is parameter 1.

Find the method of that client that lists or describes the resource type that you want to dump.

The name of the method is parameter 2.

Find in the method's response syntax the top-level key for the list of objects.

The name of the key is parameter 3.

Error output

On stderr you will always see a summary of the botocove result and any exceptions. These exceptions may reveal problems such as an incorrect command invocation, a misconfigured AWS account, or a bug in the program (feel free to report those!)

If Botocove fails to get a session for an account, it will output the ID to stderr like this.

Invalid session Account IDs as list: ['111111111111']

That account's resources will not be included in the main output.

Development

Use Poetry to build and push a new version to PyPI.

poetry build
poetry publish

TODO

TODO: query multiple regions (see aws-boto-multiregion-client for example)

TODO: ensure that org management account is included in results

TODO: give example of how to use AwsOrgInventory class in other applications

TODO: improve CLI

TODO: Use boto's service model to automate the parameters given a resource type

TODO: improve error handling

Comments
  • Fix PyPI package description

    Fix PyPI package description

    "The author of this package has not provided a project description."

    I think I can fix it with the readme option.

    https://github.com/python-poetry/poetry/issues/1979

    https://python-poetry.org/docs/pyproject/#readme

    opened by iainelder 1
  • Bump ipython from 7.28.0 to 7.31.1

    Bump ipython from 7.28.0 to 7.31.1

    Bumps ipython from 7.28.0 to 7.31.1.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Define a config file format

    Define a config file format

    Support configuration for multiple organizations.

    Each organization should have a:

    • user-defined label
    • management account profile
    • list of regions
    • list of organizations units (OUs) to exclude
    opened by iainelder 0
  • Include organization management account in results

    Include organization management account in results

    botocove by default returns results from member accounts, but not the organization management account.

    You can pass a list of account IDs instead.

    This might work, but it feel like it's duplicate what botocove already does internally.

    Contact the botocove maintainers to see if we can update the tool to return results for the management account.

    opened by iainelder 0
  • Derive transformation from boto model

    Derive transformation from boto model

    services = boto3.Session().get_available_services()
    
    def get_shape(service, method):
    
        meta = boto3.client(service).meta
        api = meta.method_to_api_mapping[method]
        output_shape = meta.service_model.operation_model(api).output_shape
        key = next(k for k, v in output_shape.members.items())
        return output_shape.members[key]
    

    See boto-multiregion-client

    Build a table of: key, list / or scalar for top key, scalar or dict for members of list

    opened by iainelder 0
  • Improve compatibility with csvsql

    Improve compatibility with csvsql

    Using aws-org-inventory version 0.2.0.

    cases.csv was created using this command:

    poetry run aws-org-inventory support describe_cases cases > cases.csv
    
    $ csvsql --snifflimit 0 --query 'SELECT 1 FROM cases' -- cases.csv
    /home/isme/.local/pipx/venvs/csvkit/lib/python3.8/site-packages/agate/utils.py:276: UnnamedColumnWarning: Column 0 has no name. Using "a".
    OperationalError: (sqlite3.OperationalError) duplicate column name: status
    [SQL: 
    CREATE TABLE cases (
    	a FLOAT, 
    	"Id" FLOAT, 
    	"Arn" VARCHAR, 
    	"Email" VARCHAR, 
    	"Name" VARCHAR, 
    	"Status" VARCHAR, 
    	"AssumeRoleSuccess" BOOLEAN, 
    	"caseId" VARCHAR, 
    	"displayId" FLOAT, 
    	subject VARCHAR, 
    	status VARCHAR, 
    	"serviceCode" VARCHAR, 
    	"categoryCode" VARCHAR, 
    	"severityCode" VARCHAR, 
    	"submittedBy" VARCHAR, 
    	"timeCreated" TIMESTAMP, 
    	"recentCommunications" VARCHAR, 
    	"ccEmailAddresses" VARCHAR, 
    	language VARCHAR, 
    	CHECK ("AssumeRoleSuccess" IN (0, 1))
    )
    
    ]
    (Background on this error at: http://sqlalche.me/e/13/e3q8)
    

    Several issues to fix here (maybe split them out):

    • Avoid duplicate column names by removing less-useful columns added by botocove (Arn, Email, Name, Status) so that only Id remains
    • Rename Id column to InventoryAccountId (not just AccountId because it may still clash with stack set APIs)
    • Remove or replace the unnamed index column (this forced me to use snifflimit)
      • replace with AccountId and Id of resource (may be hard to determine resource Id in the general case)
      • can you have a data frame without an index, or at least hide it on export?
      • replace with a monotonically increasing id
    opened by iainelder 0
Releases(v0.5.2.post1)
Owner
Iain Samuel McLean Elder
AWS Certified Solutions Architect. Freelance consultant.
Iain Samuel McLean Elder
Check your bot status automatically using userbot, simply and easy

Status Checker Userbot check your bot status automatically using userbot, simply and easy. Mandatory Vars API_ID : Telegram API_ID, get it from my.tel

ALBY 6 Feb 20, 2022
A smart tool to backup members 📈 So you even after a raid/ ban you can easily restore them in seconds 🎲

🤑 Discord-backer 🤑 A open-source Discord member backup and restore tool for your server. This can help you get all your members in 5 Seconds back af

John 29 Dec 21, 2022
𝗖𝝠𝝦𝝩𝝠𝝞𝝥 𝝦𝗥𝝞𝗖𝝽°™️ 🇱🇰 Is An All In One Media Inline Bot Made For Inline Your Media Effectively With Some Advance Security Tools♥️

𝗖𝝠𝝦𝝩𝝠𝝞𝝥 𝝦𝗥𝝞𝗖𝝽° ™️ 🇱🇰 𝗙𝗘𝝠𝝩𝗨𝗥𝗘𝗦 Auto Filter IMDB Admin Commands Broadcast Index IMDB Search Inline Search Random Pics Ids & User I

Kɪꜱᴀʀᴀ Pᴇꜱᴀɴᴊɪᴛʜ Pᴇʀᴇʀᴀ 〄 13 Jun 21, 2022
API which uses discord+mojang api to scrape NameMC searches/droptime/dropping status of minecraft names, and texture links

API which uses discord+mojang api to scrape NameMC searches/droptime/dropping status of minecraft names, and texture links

2 Dec 22, 2021
Assistant made in python to control your spotify via voice

Spotify-Assistant Assistant made in python to control your spotify via voice Overview 🚀 PLAY, PAUSE, NEXT, PREVIOUS, VOLUME COMMANDS 📝 Toast notific

Mauri 6 Jan 18, 2022
Hacktoberfest2021 - Submit Just 4 PRs to earn SWAGS and Tshirts🔥

dont contribute in this repo, contribute only in below mentioned repo Special Note For Everyone ''' always make more then 4 pull request lets you have

Keshav Singh 820 Jan 02, 2023
Handles SDVX EXCEED GEAR result screen photos and attempts to read it.

Handles SDVX EXCEED GEAR result screen photos and attempts to read it.

silverhawke 1 Jan 08, 2022
Personal Discord Python Bot based on Discord.py

Personal Discord bot using the discord.py library by Rapptz

2 Dec 14, 2022
Autov2new - Pro Auto Filter Bot V2

Pro Auto Filter Bot V2 Deploy You can deploy this bot anywhere. Watch Deploying

1 Jan 06, 2022
Script to post multiple status(posts) on twitter

Script to post multiple status on twitter (i.e. TWITTER STORM) This program can post upto maximum limit of twitter(around 300 tweets) within seconds.

Sandeep Kumar 4 Sep 09, 2021
Often discord bots just die, and we hardly find one that is durable

Muitas vezes bots do discord simplesmente morrem, e dificilmente achamos um que seja durável. Então porque não ter um próprio para emergências? Como c

Guilherme Almeida 3 Dec 06, 2022
Powerful and Advance Telegram Bot with soo many features😋🔥❤

Chat-Bot Reach this bot on Telegram Chat Bot New Features 🔥 ✨ Improved Chat Experience ✨ Removed Some Unnecessary Commands ✨ Added Facility to downlo

Sanila Ranatunga 10 Oct 21, 2022
Telegram anime bot that uses Anilist API

Telegram Bot Repo Capable of fetching the following Info via Anilist API inspired from AniFluid and Nepgear Anime Airing Manga Character Scheduled Top

Lucky Jain 71 Jan 03, 2023
批量下载抖音无水印视频

版权说明 本项目fork自Johnserf-Seed TikTokDownload。目的是为了增加个性化的功能,若想体验更多完善的功能请支持原作者的项目。 免责声明 本代码仅用于学习,下载后请勿用于商业用途。 环境要求 请检查宿主机,是否安装了python环境,并且配置了环境变量 pytho

Zhiwu Mao 44 Dec 28, 2022
A anti-repostbot script for reddit, runs u/ThisIsARepostBotBot

ThisIsARepostBotBot So you found a repost bot, now what? This is a bot to reply to all posts of a repost bot with a message urging users to report and

3 May 23, 2022
Module to use some statistics from Spotify API

statify Module to use some statistics from Spotify API To use it you have to import the functions into your own project. You have also to authenticate

Miguel Cózar 2 Jun 02, 2022
Demo to explain how to use AWS Chalice to connect to twitter API and change profile picture at scheduled times.

chalice-twitter-demo Demo to explain how to use AWS Chalice to connect to twitter API and change profile picture at scheduled times. Video Demo Click

Ahmed Mohamed 4 Dec 13, 2021
Catware - A powerful grabber with a built in bot control system

catware A powerful grabber with a built in bot control system PLEASE NOTE THAT I

4 Feb 04, 2022
Telegram 聊天機器人,追蹤momo降價、重新上架

簡介 price-tracker-bot is a telegram bot that can trace the price on momoshop. 功能 降價通知 上架通知 收藏商品 清空已收藏商品 顯示目前已收藏商品 Demo Bot Telegram bot search @momo_pr

92 Dec 28, 2022
DankMemer-Farmer - Autofarm Self-Bot for Discord bot Named Dankmemer.

DankMemer-Farmer Autofarm Self-Bot for Discord bot Named Dankmemer. Warning We are not responsible if you got banned, since "self-bots" outside of the

Mole 16 Dec 14, 2022