Infrastructure as Code. Why you need it.

No one likes meaningless duplication

In our approach we use data inheritance from one terraservice to another. It is possible with Terraform data source terraform_remote_state. Through it we can receive any data, outputted in terraservices that have already been applied. As a result, in every new terraservice we should manually define only a few variables that are specific for it.

Divide and rule

According to Terraservices concept, we divide our Terraform code into several groups:

0. terraform state storage infrastructure
1. core infra: VPC, Subnets, routing tables, etc.
2. common resources
    * bastion instance (if needed)
    * RDS
    * network connectivity (if needed)
3. infrastructure for our new service

The last point could contain several separate Terraservices, depending on your target infrastructure:

0. terraform state storage infrastructure (S3 and DynamoDB table)
1. core infra (VPC, Subnets, routing tables, etc.)
2. common resources
    * bastion instance (if needed)
    * RDS
    * network connectivity (if needed)
3. infrastructure for our new service
    * shared resources
    * service's backend
    * service's frontend

Notice

If you want to separate Production and non-Production environments by placing them in different accounts, you should move Terraform backend configuration from *.tf files to the separate *.hcl files. It allows you to choose required back-end on terraform init step:

[user@host ~] $ terraform init -backend-config=/path/to/your/tf_backend_config.hcl

The catalog tree in your repository will look this:

Some readers may ask: "Why do you store tfstate files for 0_terraform_infra in your git repository?" There is an answer: code in 0_terraform_infra performs the creation of S3 for our Terraform backend, and until it doesn’t exist we have no other place to store tfstate files. These files don’t contain any sensitive data, so we don't break git best practices (I mean "never store any secrets in your repository").

Also 0_terraform_infra creates a Terraform backend config file (prod.hcl, dev.hcl), which will be used for all future terraservices. A name of the file will be generated based on the workspace name.

"By the power of Workspaces!"

Alright, we have a Terraform code for our infrastructure. But it should manage several environments, prod and dev, at least. Terraform workspaces are designed right for this! But first, let's agree on the naming convention.

Assumption

Workspace name will contain the environment name and AWS Region name, eg prod-eu-west-1 and dev-us-east-1.

For prod and dev environments we should use different input values, that’s why each environment should have a separate *.tfvars file. Let's name them according to the workspace name to avoid confusion: prod-eu-west-1.tfvars and dev-us-east-1.tfvars.

Setup sequence example for 1_core:

[user@host 1_core] $ terraform init -backend-config=../dev.hcl                  # Initialize backend for dev environment
[user@host 1_core] $ terraform workspace new dev-us-east-1                      # Create new workspace for dev environment
[user@host 1_core] $ terraform apply -var-file=tfvars/dev-us-east-1.tfvars      # Create dev infrastructure by applying Terraform code
[user@host 1_core] $ rm -rf .terraform                                          # Remove backend configuration for dev env
[user@host 1_core] $ terraform init -backend-config=../prod.hcl                 # Initialize backend for production environment
[user@host 1_core] $ terraform workspace new prod-eu-west-1                     # Create new workspace for production environment
[user@host 1_core] $ terraform apply -var-file=tfvars/prod-eu-west-1.tfvars     # Create prod infrastructure by applying Terraform code

"Infrastructure, assemble!"

Using all the described hints, you’ll get flexible control on each level of your environments. Competent separation of your infrastructure code will allow you to update any part of the infrastructure safely, with minimum risks and lowest effect on other parts of the service.