Terraform Remote State Manager(tfremote)
tf is a python package for managing terraform remote state for: Google(Gcloud), AWS, and Azure. It sets a defined structure for all cloud providers by removing the overheard of configuring and managing the path in storage buckets.
It works with:
❗️ Note Best practice is to make sure buckets are versioned.
Install package
pip install tfremote --upgrade
Environment setup
-
Install Python 3.6+
-
Using virtualenv is strongly recommended:
python3 -m venv <venv name>
- Terraform 0.14.0 and above (download: https://www.terraform.io/downloads.html)
Default log level is WARNING, to change:
export TF_LOG_LEVEL to any of these: 'CRITICAL', 'ERROR', 'WARNING', 'INFO', 'DEBUG'
❗️ Important - Two variables are required for usingtfpackage (used set creat path in remote storage):
- teamid
- prjid
Required variables can be defined using:
- As
inline variablese.g.:-var='teamid=demo-team' -var='prjid=demo-project'- Inside
.tfvarsfile e.g.:-var-file=<tfvars file location>Two optional variables:
workspaceandstate_keycan be defined using:
-w=<workspace_name>. If no workspace is provideddefaultworkspace is used.
s=<state_key name>. If no key is providedterraformis used.Path created in S3 backend:
/<teamid>/<prjid>/<workspace>/<state-key>.tfstateFor more information refer to Terraform documentation
Setup environment variables
Workspace list file location TF_WORKSPACE_FILE_LOCATION
export TF_WORKSPACE_FILE_LOCATION=<workspace yml file location>
Reference file: link
AWS
❗️ Important - s3 bucket for remote state should reside inus-west-2
Set these env variables:
export TF_AWS_BUCKET=<your_remote_state_bucket_name>
export TF_AWS_BUCKET_REGION=us-west-2
One of below environment variable is required:
export TF_AWS_PROFILE=<aws profile to use>
or
export AWS_ACCESS_KEY_ID=<aws access key>
export AWS_SECRET_ACCESS_KEY=<aws secret access key>
Azure
To create storage for remote state there is handy script.
Run scripts/remote_state.sh (fill in the required information)
Set below env variables:
export TF_AZURE_STORAGE_ACCOUNT=<remote state storage account name>
export TF_AZURE_CONTAINER=<remote state container>
export ARM_ACCESS_KEY=<storage account access key>
GCP(gcloud)
https://cloud.google.com/community/tutorials/managing-gcp-projects-with-terraform
Set below env variables:
export TF_GCLOUD_BUCKET=<remote state storage bucket name>
export TF_GCLOUD_CREDENTIALS=json credentials file path>
Usage
For GCP(gcloud):
tf plan -c=gcloud -var=teamid=demo-team -var=prjid=demo-app -w=demo-workspace
The structure in Google Storage Bucket:
For AWS:
tf plan -c=aws -var=teamid=demo-team -var=prjid=demo-app -w=demo-workspace
The structure in AWS S3:
If you need to specify state_key in S3, specify -s=tryme-key
For Azure:
tf plan -c=azure -var=teamid=demo-team -var=prjid=demo-app -w=demo-workspace
The structure in Azure Storage:
For more available options:
tf --help
usage: tf [-h] [-var] [-var-file] [-c] [-w] [-wp] [-s] [-no-color] [-json] [-out] [-f] [-nf] [-v]
Terraform remote state wrapper package
--------------------------------------
Usage: Set below env variables to begin (more information: https://github.com/tomarv2/tfremote):
TF_WORKSPACE_FILE_LOCATION
aws: TF_AWS_BUCKET, TF_AWS_BUCKET_REGION=us-west-2, TF_AWS_PROFILE or AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
azure: TF_AZURE_STORAGE_ACCOUNT, TF_AZURE_CONTAINER, ARM_ACCESS_KEY
gcloud: TF_GCLOUD_BUCKET, TF_GCLOUD_CREDENTIALS
optional arguments:
-h, --help show this help message and exit
-var Set Terraform configuration variable. This flag can be set multiple times
-var-file Set Terraform configuration variables from a file. This flag can be set multiple times
-c Specify cloud provider (default: 'aws'). Supported values: gcloud, aws, or azure
-w Specify existing workspace name(default: 'default')
-wp Overwrite workspace directory path structure
-s File name in remote state (default: 'terraform.tfstate')
-no-color Disables terminal formatting sequences in the output
-json Enables the machine readable JSON UI output
-out Writes the generated plan to the given filename in an opaque file format
-f Enable FIPS endpoints (default: True)
-nf Disable FIPS endpoints
-v show program's version number and exit
21 Jul 25, 2022
1 Jan 24, 2022
6 Jun 4, 2022
311 Dec 21, 2022
8 Sep 20, 2022
6 Mar 14, 2022
28 Dec 29, 2022
37 Dec 28, 2022
2 Aug 18, 2022
32 Dec 25, 2022
2 Oct 23, 2022
147 Sep 18, 2022
11 Jul 28, 2022
11 Mar 19, 2022
2.9k Jan 08, 2023
1 Jan 17, 2022
2 Jul 05, 2022
2 Dec 04, 2021
95 Sep 24, 2021
16 Dec 13, 2022
104 Nov 26, 2022
26 Nov 30, 2022
270 Dec 30, 2022
320 Dec 22, 2022
47 Dec 06, 2022
0 Dec 08, 2021
3 Dec 21, 2021
2 Feb 03, 2022
2 Jun 25, 2022