Ensemble Docs
Self-Hosting

Terraform Reference

The AWS and GCP Terraform stacks that provision the supporting cloud resources, their modules, variables, and outputs.

Two Terraform stacks provision the supporting resources the platform needs. They are intentionally scoped to the surrounding cloud services and do not create the Kubernetes cluster, the database, or Temporal.

StackPathCloud
AWSinfrastructure/terraformAmazon Web Services
GCPinfrastructure/terraform-gcpGoogle Cloud Platform

What both stacks share

  • Requirements: Terraform >= 1.5.
  • A workloads identity: an IAM role (AWS) or service account (GCP) that your Kubernetes workloads assume, scoped to exactly the resources they use.
  • A secrets placeholder: one secret (app-secrets) seeded with empty keys and ignore_changes, so Terraform never overwrites the real values you set later.
  • A Helm bridge output: helm_config_values, a map of the non-secret environment variables the chart needs (bucket names, KMS references, region, scheduler wiring, auth pool IDs). Feed it into the chart's *.config.
  • A service account annotation output: service_account_annotation, the exact annotation to put on the Kubernetes service account for IRSA (AWS) or Workload Identity (GCP).

Neither stack creates the cluster (EKS/GKE), the database, or Temporal. Provision those first (see Prerequisites). On AWS you must create the EKS cluster and its OIDC provider, then pass the OIDC ARN back into Terraform so the workloads role can use IRSA. Without it, the role falls back to an EC2 trust relationship.


AWS stack (infrastructure/terraform)

What it creates

ModuleEnabled by defaultCreates
s3YesThree buckets: user-files, documents, tenant-migrations (SSE-S3, public access blocked, versioning + noncurrent expiry)
kmsYesA KMS key + alias for secret encryption (rotation on), or reuses an existing key
iamYesThe workloads IAM role (IRSA) with scoped S3, KMS, Secrets, Scheduler, Cognito, and Bedrock policies; plus the scheduler role
secretsYesA Secrets Manager secret <prefix>/app-secrets seeded with empty keys
cognitoYesA Cognito user pool, SPA client, and hosted-UI domain; optional Google IdP and scheduler OAuth client
schedulerYesEventBridge Scheduler group, event bus, and (if an endpoint is set) an API destination with auth
ecrNoOne repository each for server, web, worker, migration, chart, with lifecycle policies

The ECR module is off by default because most customers pull images from the Marketplace registry rather than mirroring them. When enabled, its lifecycle policy keeps the last N -dev and untagged images and keeps immutable release tags forever.

Key root variables

VariableDefaultDescription
aws_regionus-west-2Region
environmentdevEnvironment name (prefixes resources)
project_nameworkflowsResource name prefix
eks_oidc_provider_arn""EKS OIDC provider ARN, required for IRSA
eks_namespaceworkflowsNamespace the workloads run in
eks_service_account_nameworkflows-saKubernetes service account name
enable_cognitotrueProvision Cognito
enable_schedulertrueProvision EventBridge Scheduler
enable_ecrfalseProvision ECR repositories
cognito_callback_urls, cognito_logout_urlslocalhostAllowed auth redirect URLs
s3_versioning_enabledtrueBucket versioning
s3_noncurrent_version_expiration_days90Noncurrent version retention
existing_*_bucket, existing_kms_key_arn""Reuse existing resources instead of creating

Scheduler and Cognito have many additional variables for callback endpoints, auth type (API_KEY, BASIC, OAUTH_CLIENT_CREDENTIALS), and Google IdP. See variables.tf in the stack for the full list.

Key outputs

workloads_role_arn, service_account_annotation (the eks.amazonaws.com/role-arn annotation), the three bucket names and ARNs, kms_key_arn, app_secrets_arn/app_secrets_name, the Cognito pool/client/domain IDs, the scheduler group/bus/role ARNs, and helm_config_values.

State and provider

  • Provider hashicorp/aws ~> 5.0.
  • The remote S3 backend is present but commented out (default is local state). Enable it for team use.

GCP stack (infrastructure/terraform-gcp)

What it creates

ModuleEnabled by defaultCreates
iamYesA Google service account (Workload Identity target) bound to the Kubernetes service account, with Vertex AI and logging roles
kmsYesA key ring plus three crypto keys (secrets, connections, environment), 90-day rotation, prevent_destroy
gcsYesThree buckets: user-files, documents, tenant-migrations (uniform access, versioning, noncurrent expiry)
secretsYesA Secret Manager secret <prefix>-app-secrets seeded with empty keys
firebase-authYesIdentity Platform (Firebase Auth) config with email/password and optional Google IdP
artifact-registryYesOne Docker repository holding all images, with a keep-recent cleanup policy
schedulerNoA Cloud Scheduler invoker service account and IAM (jobs are created by the app at runtime)

The stack also enables the required Google APIs on the project (KMS, Storage, Secret Manager, IAM, Vertex AI, and conditionally Artifact Registry, Identity Platform, and Cloud Scheduler) when enable_apis is true.

Key root variables

VariableDefaultDescription
project_id(required)GCP project ID (one project per environment recommended)
regionus-west1Region for regional resources
environment(required)Environment name
gke_namespaceworkflowsNamespace the workloads run in
gke_service_account_nameworkflows-saKubernetes service account name
enable_firebase_authtrueProvision Identity Platform
enable_artifact_registrytrueProvision the Docker repository
enable_schedulerfalseProvision Cloud Scheduler wiring
firebase_authorized_domains["localhost"]Auth redirect domains
firebase_google_client_id / _secret""Optional Google sign-in
gcs_versioning_enabledtrueBucket versioning
artifact_registry_keep_count10Image versions to keep per package

Key outputs

workloads_service_account_email, service_account_annotation (the iam.gke.io/gcp-service-account annotation), gcs_bucket_names, kms_key_ring and kms_key_names, app_secret_id, artifact_registry_repo_url, and helm_config_values (with CLOUD_PROVIDER=gcp, AUTH_PROVIDER=firebase, project, region, bucket names, and KMS references).

State and provider

  • Providers hashicorp/google and hashicorp/google-beta, both ~> 6.0 (google-beta is required for Identity Platform).
  • Remote state uses a GCS backend, configured per environment with -backend-config. Create the state bucket once before terraform init.

Manual steps not covered by Terraform

  • Enable Identity Platform in the console once (a product toggle), and register a Firebase web app to obtain FIREBASE_API_KEY and FIREBASE_AUTH_DOMAIN.
  • Grant the GKE node pool's service account artifactregistry.reader if image pulls authenticate as the node service account.

Applying a stack

The pattern is the same for both, differing only in the backend and variable file.

cd infrastructure/terraform          # or terraform-gcp
terraform init                       # add -backend-config for remote state
terraform plan  -var-file=environments/prod.tfvars
terraform apply -var-file=environments/prod.tfvars
terraform output                     # capture helm_config_values and the SA annotation

Never put secret values in .tfvars files committed to Git. Pass sensitive variables (OAuth client secrets, API keys) via TF_VAR_* environment variables at apply time. The app-secrets secret is created with empty placeholders; write the real values out-of-band (console, CLI, or your CI secrets pipeline).

The cloud-specific deployment walkthroughs put these steps in order end to end: Deploy on AWS and Deploy on GCP.

On this page