Dumps to CSV all the resources in an organization's member accounts

Overview

AWS Org Inventory

Dumps to CSV all the resources in an organization's member accounts.

Set your environment's AWS_PROFILE and AWS_DEFAULT_REGION variables.

The AWS_PROFILE should be configured to use a role in the organization management account that can assume OrganizationAccountAccessRole in the member accounts.

Redirect the dumper's output to save the file.

The dumper uses Botocove to query each member account.

Why?

This tool fills in the gaps in AWS Config's inventory.

Sadly AWS Config supports only a subset of all the resource types you may have.

AWS Config's organization aggregators are great, but they may not update instantly with new resources.

Installation

The package is published to PyPy as aws-org-inventory, so you can install it with pip or anything equivalent.

pip install aws-org-inventory

Basic example

Configure environment:

export AWS_PROFILE=OrgMgmtRole
export AWS_DEFAULT_REGION=eu-west-1

Dump inventory of CloudWatch log groups:

aws-org-inventory logs describe_log_groups logGroups

Dump inventory of support cases:

aws-org-inventory support describe_cases cases

Dump inventory of EC2 key pairs:

aws-org-inventory ec2 describe_key_pairs KeyPairs

Try doing those with AWS Config!

General use

To derive arguments for other use cases, check the boto service documentation.

The value passed to the boto3.client method that would instantiate a client for the service is parameter 1.

Find the method of that client that lists or describes the resource type that you want to dump.

The name of the method is parameter 2.

Find in the method's response syntax the top-level key for the list of objects.

The name of the key is parameter 3.

Error output

On stderr you will always see a summary of the botocove result and any exceptions. These exceptions may reveal problems such as an incorrect command invocation, a misconfigured AWS account, or a bug in the program (feel free to report those!)

If Botocove fails to get a session for an account, it will output the ID to stderr like this.

Invalid session Account IDs as list: ['111111111111']

That account's resources will not be included in the main output.

Development

Use Poetry to build and push a new version to PyPI.

poetry build
poetry publish

TODO

TODO: query multiple regions (see aws-boto-multiregion-client for example)

TODO: ensure that org management account is included in results

TODO: give example of how to use AwsOrgInventory class in other applications

TODO: improve CLI

TODO: Use boto's service model to automate the parameters given a resource type

TODO: improve error handling

Comments
  • Fix PyPI package description

    Fix PyPI package description

    "The author of this package has not provided a project description."

    I think I can fix it with the readme option.

    https://github.com/python-poetry/poetry/issues/1979

    https://python-poetry.org/docs/pyproject/#readme

    opened by iainelder 1
  • Bump ipython from 7.28.0 to 7.31.1

    Bump ipython from 7.28.0 to 7.31.1

    Bumps ipython from 7.28.0 to 7.31.1.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Define a config file format

    Define a config file format

    Support configuration for multiple organizations.

    Each organization should have a:

    • user-defined label
    • management account profile
    • list of regions
    • list of organizations units (OUs) to exclude
    opened by iainelder 0
  • Include organization management account in results

    Include organization management account in results

    botocove by default returns results from member accounts, but not the organization management account.

    You can pass a list of account IDs instead.

    This might work, but it feel like it's duplicate what botocove already does internally.

    Contact the botocove maintainers to see if we can update the tool to return results for the management account.

    opened by iainelder 0
  • Derive transformation from boto model

    Derive transformation from boto model

    services = boto3.Session().get_available_services()
    
    def get_shape(service, method):
    
        meta = boto3.client(service).meta
        api = meta.method_to_api_mapping[method]
        output_shape = meta.service_model.operation_model(api).output_shape
        key = next(k for k, v in output_shape.members.items())
        return output_shape.members[key]
    

    See boto-multiregion-client

    Build a table of: key, list / or scalar for top key, scalar or dict for members of list

    opened by iainelder 0
  • Improve compatibility with csvsql

    Improve compatibility with csvsql

    Using aws-org-inventory version 0.2.0.

    cases.csv was created using this command:

    poetry run aws-org-inventory support describe_cases cases > cases.csv
    
    $ csvsql --snifflimit 0 --query 'SELECT 1 FROM cases' -- cases.csv
    /home/isme/.local/pipx/venvs/csvkit/lib/python3.8/site-packages/agate/utils.py:276: UnnamedColumnWarning: Column 0 has no name. Using "a".
    OperationalError: (sqlite3.OperationalError) duplicate column name: status
    [SQL: 
    CREATE TABLE cases (
    	a FLOAT, 
    	"Id" FLOAT, 
    	"Arn" VARCHAR, 
    	"Email" VARCHAR, 
    	"Name" VARCHAR, 
    	"Status" VARCHAR, 
    	"AssumeRoleSuccess" BOOLEAN, 
    	"caseId" VARCHAR, 
    	"displayId" FLOAT, 
    	subject VARCHAR, 
    	status VARCHAR, 
    	"serviceCode" VARCHAR, 
    	"categoryCode" VARCHAR, 
    	"severityCode" VARCHAR, 
    	"submittedBy" VARCHAR, 
    	"timeCreated" TIMESTAMP, 
    	"recentCommunications" VARCHAR, 
    	"ccEmailAddresses" VARCHAR, 
    	language VARCHAR, 
    	CHECK ("AssumeRoleSuccess" IN (0, 1))
    )
    
    ]
    (Background on this error at: http://sqlalche.me/e/13/e3q8)
    

    Several issues to fix here (maybe split them out):

    • Avoid duplicate column names by removing less-useful columns added by botocove (Arn, Email, Name, Status) so that only Id remains
    • Rename Id column to InventoryAccountId (not just AccountId because it may still clash with stack set APIs)
    • Remove or replace the unnamed index column (this forced me to use snifflimit)
      • replace with AccountId and Id of resource (may be hard to determine resource Id in the general case)
      • can you have a data frame without an index, or at least hide it on export?
      • replace with a monotonically increasing id
    opened by iainelder 0
Releases(v0.5.2.post1)
Owner
Iain Samuel McLean Elder
AWS Certified Solutions Architect. Freelance consultant.
Iain Samuel McLean Elder
Auto file forward bot with python

Auto-File-Forward-Bot Auto file forward bot. Without Admin Permission in FROM_CHANNEL Only Give Permission In your Telegram Personal Channel Please fo

Milas 1 Oct 15, 2021
This Lambda will Pull propagated routes from TGW and update VPC route table

AWS-Transitgateway-Route-Propagation This Lambda will Pull propagated routes from TGW and update VPC route table. Tested on python 3.8 Lambda AWS INST

4 Jan 20, 2022
A Git Alert Bot - Github Integration for Pyrogram & Telethon

Yet Another GitAlertBot Inspired From @Pokurt's GitGram Run Bot: Local Host Git Clone Repo : For Telethon Version : git clone https://github.com/DevsE

DevsExpo 23 Oct 21, 2022
Robocord is a bot created for the Pycord community.

Robocord is a bot created for the community of the Pycord Server. Just a bot created for Pycord Server. You can start pull requests, I will check it and if its good I will add it to the bot. 👍

Bruce 7 Jun 26, 2022
a discord libary that use to make discord bot with low efficiency and bad performance because I don't know how to manage the project

Aircord 🛩️ a discord libary that use to make discord bot with low efficiency and bad performance because I don't know how to manage the project Examp

Aircord 2 Oct 24, 2021
A bot to view Garfield comics directly from Discord and get updates of the comics automatically

Garfield-Bot A bot to view Garfield comics directly from Discord and get updates of the comics automatically. Instructions to use the bot: Invite the

Raghav Sharma 3 Feb 13, 2022
Powerful and Async API for AnimeWorld.tv 🚀

Powerful and Async API for AnimeWorld.tv 🚀

1 Nov 13, 2021
Simple Similarities Service

simsity Simsity is a Super Simple Similarities Service[tm]. It's all about building a neighborhood. Literally! This repository contains simple tools t

vincent d warmerdam 95 Dec 25, 2022
Fetch torrent links from nyaa, according to releases by smoke index.

Nyaa - Smoke's index torrent fetcher Description This script parses the local (or online) anime release index (csv format) made by Big Smoke. And uses

Dinank 21 Jun 08, 2022
PR Changes Matrix Builder

Pr-changes-matrix-builder - A Github Action that will output a variable to be used in a matrix strategy job based on a PR&'s changes

Kyle James Walker (he/him) 21 Oct 04, 2022
Monetize your apps with KivAds using Google AdMob api.

KivAds(WIP) Monetize your apps with KivAds using Google AdMob api. KivAds uses the latest version of Google AdMob sdk(version 20.0.0). KivAds exposes

Guhan Sensam 16 Nov 05, 2022
Many discord bots serving different purposes

Discord_Botlari Farklı amaçlara hizmet eden bir çok discord botu En kapsamlı Bot Game Bottur. bir oyun botudur discord sunucularında kullanılır. (tüm

1 Dec 21, 2021
This bot is created by AJTimePyro and It accepts direct downloading url & then return file as telegram file.

URL Uploader Bot This is the source code of URL Uploader Bot. And the developer of this bot is AJTimePyro, His Telegram Channel & Group. You can use t

Abhijeet 23 Nov 13, 2022
A Python wrapper for the QQ Channel API

A Python wrapper for the QQ Channel API

Fox_white 55 Dec 07, 2022
Telegram bot to trim and download videos from youtube.

Inline-YouTube-Trim-Bot Telegram bot to trim and download youtube videos Deploy You can deploy this bot anywhere. Demo - YouTubeBot Required Variables

SUBIN 56 Dec 11, 2022
Web-music-bot - A telegram bot which get a *site Url* and sends all songs contain in the Url to telegram

web music bot this is a telegram bot which get a site Url and sends all songs co

Arya Shabane 4 Apr 02, 2022
Extend the commitizen tools to create conventional commits and README that link to Jira and GitHub.

cz-github-jira-conventional cz-github-jira-conventional is a plugin for the commitizen tools, a toolset that helps you to create conventional commit m

12 Dec 13, 2022
Simple script to extract useful informations from the combo BloodHound + Neo4j

bloodhound-quickwin Simple script to extract useful informations from the combo BloodHound + Neo4j. Can help to choose a target. Prerequisites python3

140 Dec 21, 2022
ApiMoedas - This API is a extesion of API

🪙 Api Moeda 🪙 Este projeto é uma extensão da API Awesome API. Basicamente, ele mostra todas as moedas que a Awesome API tem e todas as suas conversõ

Abel 4 May 29, 2022
Telegram bot to stream videos in telegram voicechat for both groups and channels. Supports live strams, YouTube videos and telegram media.

Telegram bot to stream videos in telegram voicechat for both groups and channels. Supports live strams, YouTube videos and telegram media.

SUBIN 449 Dec 27, 2022