ADR-004: Testing Strategy (Native .tftest.hcl vs Terratest Go vs Both)

Status: Accepted Date: 2026-02-25 Deciders: @cloud-architect, @product-owner, HITL/Manager

Context

Two testing frameworks exist for Terraform:

Native Terraform Test Framework (.tftest.hcl, GA in Terraform 1.6+):

• Built into Terraform CLI; zero additional dependencies
• Supports command = plan (no credentials) and command = apply (requires credentials)
• Mock providers in 1.7+ (mock_provider {}) for pure unit tests
• Parallel test runs with run {} blocks
• Source: https://developer.hashicorp.com/terraform/language/testing

Terratest (Go-based, HashiCorp-endorsed but not built-in):

• Full Go test ecosystem: assertions, retries, HTTP helpers, SSH helpers
• terraform.InitAndApply() → validate → terraform.Destroy()
• Supports LocalStack via terraform.Options{EnvVars: {"LOCALSTACK_HOST": "..."}}
• Required for stateful assertions: "did the S3 bucket actually become publicly-blocked?"
• Source: https://github.com/gruntwork-io/terratest

Existing codebase analysis (from source module inspection):

• iam-identity-center/tests/: .tftest.hcl files only (Tier 1 plan tests, Tier 2-3 apply tests)
• aws-ecs-fargate/tests/: .tftest.hcl files only (Tier 1 minimal)
• The ADLC skill (terraform-patterns.md) references BOTH Go Terratest AND native tests
• The manager brief specifies "3-tier testing": Tier 1 (static/snapshot), Tier 2 (LocalStack), Tier 3 (AWS Sandbox)

Framework comparison for this context:

Criterion	Native .tftest.hcl	Terratest Go
Setup complexity	Zero (built-in)	Go toolchain required
Mock providers	YES (1.7+)	NO (hits real/local endpoints)
LocalStack integration	Via provider config	Via terraform.Options{}
HTTP/API assertions post-apply	NO	YES
Retry logic	NO (manual)	YES (built-in retry.DoWithRetry)
Parallel execution	YES (run {} blocks)	YES (Go t.Parallel())
CI speed	Fast (no Go compile)	Slower (Go build cache helps)
Registry module testing standard	Native preferred	Both used by major publishers
Cross-module integration tests	Limited	Strong

Decision

Adopt BOTH frameworks with clear tier assignment:

Tier	Framework	When	Cost
Tier 1: Static	Native .tftest.hcl with command = plan	Every PR, no credentials	$0
Tier 1: Mock	Native .tftest.hcl with mock_provider {}	Every PR, no credentials	$0
Tier 2: LocalStack	Native .tftest.hcl with command = apply + LocalStack provider	Every PR, LocalStack container	$0
Tier 3a: Sandbox Apply	Native .tftest.hcl with command = apply (real AWS)	On-demand, HITL approval	~$5-50/run
Tier 3b: Integration	Terratest Go	Weekly / pre-release only	~$20-100/run

Rationale for primary native .tftest.hcl:

1. The existing source modules already use .tftest.hcl — consistency is non-negotiable for DRY
2. Mock providers (Terraform 1.7+) eliminate the "need real credentials for unit tests" problem
3. Native tests are executable by ANY consumer without Go toolchain
4. Registry community expectation: published modules provide runnable examples, not just Go tests

Rationale for Terratest in Tier 3b only:

1. HTTP endpoint validation after ECS service deploy cannot be done in native tests
2. Aurora cluster failover time measurement requires retry loops (Terratest strength)
3. Cross-module fullstack integration (WAF + CloudFront + ECS + Aurora in sequence) is easier in Go
4. Limit Terratest to weekly pre-release runs to control cost

File naming convention:

tests/
├── 00_tier1_plan.tftest.hcl          # Plan-only, no credentials
├── 01_tier1_mock.tftest.hcl          # Mock provider, no credentials
├── 02_tier2_localstack.tftest.hcl    # LocalStack apply
└── tier3/
    ├── 03_tier3_apply.tftest.hcl     # Real AWS apply (HITL gate)
    └── integration_test.go           # Terratest (pre-release only)

Consequences

Positive

• Every PR gets Tier 1 + Tier 2 coverage at $0 (LocalStack container in CI)
• Consumers can run terraform test without Go toolchain
• Terratest covers integration scenarios that native tests cannot
• ADLC Constitution Principle III (Evaluation-First) satisfied: 100% plan coverage + mock coverage

Negative

• Maintaining TWO test frameworks increases cognitive overhead for contributors
• Terratest Go tests require a Go developer to maintain; if team skill atrophies, tests rot

Blocking Risks

Risk	Probability	Impact	Mitigation
Terratest tests become stale/ignored when Go expertise is unavailable	Medium	Medium	Terratest tests run in weekly CI; any failure blocks the weekly pre-release tag; explicit ownership assigned in CODEOWNERS

Alternatives Considered

1. Native only: Rejected — cannot validate post-apply HTTP endpoints, retry logic, or cross-module integration scenarios that require Go
2. Terratest only: Rejected — breaks the existing .tftest.hcl investment in the source modules (3,342 LOC of Identity Center has 11 existing test files); forces Go toolchain on all consumers; violates DRY
3. Both with Terratest as primary: Rejected — reverses the build vs buy decision; native is sufficient for 85% of scenarios

Related ADRs

• ADR-003: Version constraints define mock_provider availability (requires Terraform >= 1.7+)
• ADR-005: Example naming convention (mvp-, poc-, production-) maps directly to test tiers (Tier 1, Tier 2, Tier 3)

References

• Terraform Test Framework: https://developer.hashicorp.com/terraform/language/testing
• Mock Providers (1.7+): https://developer.hashicorp.com/terraform/language/testing/mocking
• Terratest: https://terratest.gruntwork.io/
• LocalStack + Terraform: https://docs.localstack.cloud/user-guide/integrations/terraform/

Coordination Evidence

Consolidated from .adlc/projects/terraform-aws/ as part of ADR-001→019 SSOT consolidation (2026-02-27).