Advanced Terraform Tooling: Terragrunt, Testing, and Best Practices

Advanced Terraform Tooling and Best Practices

While Terraform itself is powerful, the ecosystem around it provides additional tools that can enhance your Infrastructure as Code (IaC) workflow. This post explores these tools and best practices for maintaining enterprise-grade Terraform code.

Terragrunt: Keep Your Terraform Code DRY

Terragrunt is a thin wrapper for Terraform that provides extra tools for working with multiple Terraform modules, remote state, and keeping your configurations DRY (Don’t Repeat Yourself).

Installing Terragrunt

# macOS brew install terragrunt # Linux/Windows with go installed go install github.com/gruntwork-io/terragrunt@latest

Basic Terragrunt Structure

# terragrunt.hcl in root directory remote_state { backend = "s3" config = { bucket = "my-terraform-state" key = "${path_relative_to_include()}/terraform.tfstate" region = "us-west-2" encrypt = true dynamodb_table = "terraform-locks" } } # Common variables for all environments inputs = { company = "MyCompany" owner = "DevOps Team" environment = "${get_env("TF_VAR_environment", "dev")}" } # Generate provider configuration generate "provider" { path = "provider.tf" if_exists = "overwrite_terragrunt" contents = <<EOF provider "aws" { region = "us-west-2" default_tags { tags = { Environment = "${get_env("TF_VAR_environment", "dev")}" Terraform = "true" Owner = "DevOps" } } } EOF }

Project Structure with Terragrunt

infrastructure/ ├── terragrunt.hcl ├── modules/ │ ├── vpc/ │ ├── ecs/ │ └── rds/ └── live/ ├── dev/ │ ├── vpc/ │ │ └── terragrunt.hcl │ └── ecs/ │ └── terragrunt.hcl └── prod/ ├── vpc/ │ └── terragrunt.hcl └── ecs/ └── terragrunt.hcl

Environment-Specific Configuration

# live/dev/vpc/terragrunt.hcl include "root" { path = find_in_parent_folders() } include "env" { path = find_in_parent_folders("env.hcl") } terraform { source = "../../../modules//vpc" } inputs = { vpc_cidr = "10.0.0.0/16" environment = "dev" }

Pre-commit Hooks for Terraform

Pre-commit hooks help maintain code quality by running checks before commits are made.

Setting Up Pre-commit

1. Install pre-commit:
   pip install pre-commit
2. Create .pre-commit-config.yaml: ```yaml repos:
   • repo: https://github.com/antonbabenko/pre-commit-terraform rev: v1.83.5 hooks:
   • id: terraform_fmt
   • id: terraform_docs
   • id: terraform_tflint
   • id: terraform_validate
   • id: terraform_checkov

• repo: https://github.com/pre-commit/pre-commit-hooks rev: v4.5.0 hooks:
  • id: check-merge-conflict
  • id: end-of-file-fixer
  • id: trailing-whitespace
  • id: check-yaml ```

1. Install the hooks:
   pre-commit install

Automated Testing for Terraform

Unit Testing with Terratest

// test/vpc_test.go package test import ( "testing" "github.com/gruntwork-io/terratest/modules/terraform" "github.com/stretchr/testify/assert" ) func TestVPCCreation(t *testing.T) { terraformOptions := &terraform.Options{ TerraformDir: "../modules/vpc", Vars: map[string]interface{}{ "vpc_cidr": "10.0.0.0/16", "environment": "test", }, } defer terraform.Destroy(t, terraformOptions) terraform.InitAndApply(t, terraformOptions) vpcID := terraform.Output(t, terraformOptions, "vpc_id") assert.NotEmpty(t, vpcID) }

Integration Testing

// test/integration_test.go func TestVPCWithSubnets(t *testing.T) { terraformOptions := &terraform.Options{ TerraformDir: "../environments/test", Vars: map[string]interface{}{ "environment": "test", }, } defer terraform.Destroy(t, terraformOptions) terraform.InitAndApply(t, terraformOptions) // Test VPC vpcID := terraform.Output(t, terraformOptions, "vpc_id") // Test Subnets privateSubnetIDs := terraform.OutputList(t, terraformOptions, "private_subnet_ids") assert.Equal(t, 3, len(privateSubnetIDs)) }

Continuous Integration Pipeline

# .github/workflows/terraform.yml name: 'Terraform CI' on: push: branches: [ main ] pull_request: branches: [ main ] jobs: terraform: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Setup Terraform uses: hashicorp/setup-terraform@v2 - name: Setup Go uses: actions/setup-go@v4 with: go-version: '1.20' - name: Terraform Format run: terraform fmt -check - name: Terraform Init run: terraform init - name: Terraform Validate run: terraform validate - name: Run Terratest run: | cd test go test -v ./...

Enterprise Features and Alternatives

Terraform Enterprise Features

1. Remote Operations
   • Remote state management
   • Remote plan and apply
   • Policy as code (Sentinel)
   • Private registry
2. Team Management
   • RBAC
   • SSO integration
   • Audit logging

Open Source Alternatives

1. State Management
   # Using S3 with DynamoDB locking terraform { backend "s3" { bucket = "terraform-state" key = "state/terraform.tfstate" region = "us-west-2" encrypt = true dynamodb_table = "terraform-locks" } }
2. Policy Enforcement ```hcl

   Using Open Policy Agent (OPA)

   package terraform

deny[msg] { resource := input.planned_values.root_module.resources[_] resource.type == “aws_instance” not resource.values.tags.Environment

msg = sprintf("EC2 instance %v must have Environment tag", [resource.address]) } ```

1. CI/CD Integration ```yaml

   GitLab CI example

   terraform_plan: stage: plan script:

   • terraform init
   • terraform plan -out=plan.tfplan artifacts: paths:
     • plan.tfplan

terraform_apply: stage: apply script: - terraform apply plan.tfplan when: manual only: - main ```

Best Practices

1. Module Organization
   • Keep modules small and focused
   • Use consistent interface patterns
   • Version your modules
2. State Management
   • Use remote state
   • Enable state locking
   • Implement backup strategies
3. Security
   • Use IAM roles
   • Encrypt sensitive data
   • Implement least privilege
4. Testing
   • Write unit tests
   • Implement integration tests
   • Use policy checks

Conclusion

By implementing these tools and practices, you can create a robust and maintainable Terraform codebase that scales with your organization’s needs. Remember to:

• Use Terragrunt for DRY configurations
• Implement pre-commit hooks
• Write automated tests
• Consider enterprise features or alternatives
• Follow security best practices

Additional Resources

• Terragrunt Documentation
• Pre-commit Terraform Hooks
• Terratest Documentation
• Open Policy Agent

Stay tuned for more advanced Terraform topics and best practices!