# Repo map — Infrastructure as Code

Ground truth for the course. Paths from the repo root; IaC lives under
<span class="filepath">deployments/terraform/</span>. Line numbers drift. Terraform **≥ 1.9**
(Atlantis pins **1.9.8**); providers `google` + `google-beta` (+ cloudflare, github).

---

## 0. Two facts to note up front

1. **The Terragrunt root config is `root.hcl`** (not the conventional `terragrunt.hcl`):
   <span class="filepath">deployments/terraform/live/root.hcl</span>.
2. **Not everything is Terraform-managed.** The TF state bucket, CI cache, and platform buckets
   are TF — but Thanos's <code>manabie-thanos</code> bucket (Course 6) is **not** in this tree
   (externally/manually provisioned). A useful "the map isn't the whole territory" caveat.

---

## 1. Layout — Terragrunt, live/modules *(Lessons 5–6)*
- **Split:** <span class="filepath">deployments/terraform/modules/</span> (**34** modules,
  reusable, no state) + <span class="filepath">deployments/terraform/live/</span> (per-cell
  configs). **180** `terragrunt.hcl` leaf components across all env×org; 27 top-level `live/` dirs.
- **Root config:** <span class="filepath">live/root.hcl</span> — `remote_state` (`:11-23`),
  `generate "provider"` (`:25-34`), `generate "versions"` (`:39-55`), `inputs = merge(env, common)`
  (`:57-60`).
- **A leaf** (e.g. <span class="filepath">live/stag-manabie/platforms/terragrunt.hcl</span>):
  `include { path = find_in_parent_folders("root.hcl") }` (`:1-3`); `terraform { source =
  "../../..//modules/<component>" }` (`:5-7`); `dependency "<x>"` blocks wiring cross-component
  outputs (`:9-11`); a second `include "env"` pulling a shared component config from
  <span class="filepath">live/_env/<component>.hcl</span> with `expose = true` (`:13-16`).
- **env×org expressed 3 ways:** (1) per-cell dir + <span class="filepath">live/<env>-<org>/env.hcl</span>
  locals (`project_id`, `region`, `env`, `service_account_prefix`, state `bucket`); (2) shared
  component logic in <span class="filepath">live/_env/*.hcl</span> (`platforms.hcl`,
  `stag-apps.hcl`, `project-roles.hcl`, …); (3) global non-env-org dirs (`live/cloudflare`,
  `live/github-oidc`, `live/kms-key`, `live/terraform-state-bucket`, `live/monitoring`).

## 2. State backend *(Lesson 3)*
- GCS backend, set once in <span class="filepath">live/root.hcl:11-23</span> (`backend = "gcs"`,
  `prefix = "${path_relative_to_include()}/"`, generates `backend.tf`) — inherited by every leaf.
- Bucket: default `manabie-terraform-state-2` (`root.hcl:6-7`), per-env override (e.g.
  `manabie-staging-terraform-state`, `live/stag-manabie/env.hcl:12`). **GCS-native locking**
  (no separate lock table). Per-component prefix = isolated state.

## 3. Module inventory (~34) *(Lesson 7)*
| Module | Purpose |
|---|---|
| `platforms/` | the big env platform: GKE (`gke.tf`), Cloud SQL (`postgresql.tf`), GCS (`storage.tf`), KMS (`kms.tf`), BigQuery, PubSub, Artifact Registry |
| `apps/{stag,uat,prod}/` | per-service **GSAs + Workload Identity bindings** + KMS grants + GKE RBAC |
| `vpc/` | VPC, subnets, PSC, Cloud NAT, private service access, VPC connectors |
| `postgresql/` | DBs/schemas/users/extensions *inside* an existing Cloud SQL instance |
| `kms-key/` | KMS keyring + keys + decrypter IAM (the **SOPS** keys) |
| `binary-authorization-attestor/` | binauthz attestors + KMS attestor keys + admission policy |
| `github-oidc/` | GitHub Actions **WIF pool + providers + CI bot SAs** |
| `simple-bucket/` | thin wrapper over the cloud-storage simple_bucket module |
| `cloudflare-dns/` | DNS records + certificate packs + custom hostnames |
| `project-roles/`,`project-grant/`,`project-services/` | custom IAM roles, grants, enabled APIs |
| `postgresql-roles/`,`postgresql-grant/` | DB-level RBAC |
| `oncall/`,`kubeip/`,`deploy-bot/`,`lib/gke-identity-namespaces/` | OnCall API objects; egress IP; automation SAs; ns mapping |

## 4. GKE module *(Lesson 8)*
<span class="filepath">modules/platforms/gke.tf</span> wraps
`terraform-google-modules/kubernetes-engine//modules/beta-public-cluster` (`:4-5`).
- **`enable_binary_authorization = var.gke.enable_binary_authorization`** at
  <span class="filepath">gke.tf:111</span> (Course 5); `true` in `stag-manabie/env.hcl:680`.
- Release channel `gke.tf:14` (`REGULAR`, K8s 1.30.x). **Workload Identity** enabled by the
  upstream module; output `gke_identity_namespace` = `PROJECT.svc.id.goog`.
- **Node pools** from `var.gke.node_pools` (`gke.tf:50-80`); the pool list is in
  <span class="filepath">live/stag-manabie/env.hcl:694-1109</span> — **`spot = true/false`** is the
  spot-vs-on-demand switch (Course 2 L10): on-demand `n2d-highmem-4` (`spot=false`), spot
  `n2d-highmem-4-spot`, `*-runners-spot`, GPU `*-gpu-spot`. `enable_secure_boot`, per-pool
  autoscaling, VPA, KMS `database_encryption`, `network_policy`, `remove_default_node_pool`.

## 5. Cloud SQL *(Lesson 11)*
- Instances created in <span class="filepath">modules/platforms/postgresql.tf:1-8</span>
  (`for_each` over `var.postgresql`, wraps `GoogleCloudPlatform/sql-db`). Stag has 5 service DB
  instances: **lms, common, auth, lmsdwh, aitutor** (`live/stag-manabie/platforms/terragrunt.hcl`
  `inputs.postgresql`) — mostly `POSTGRES_14` (aitutor 17), IAM auth + pgaudit + pg_cron,
  `deletion_protection`, `ssl_mode: ENCRYPTED_ONLY`, private network from the VPC dependency.
- `modules/postgresql/` (singular) manages DBs/users *on* an existing instance (data source).
  The Grafana OnCall DB is a Cloud SQL Postgres (Course 6); `modules/oncall/` manages OnCall's
  API objects (escalation chains, schedules), not the instance.

## 6. Networking & Cloudflare *(Lesson 11)*
- <span class="filepath">modules/vpc/main.tf</span> wraps `terraform-google-modules/network`
  (subnets + secondary ranges `:1-12`; private service access `:14-27`; PSC subnets `:29-47`;
  **Cloud NAT** + static IPs + router `:67-110`; VPC access connectors `:114-127`).
- <span class="filepath">modules/cloudflare-dns/</span>: `records.tf:6`
  `cloudflare_record.manabieio_records` (for_each), `certificate_pack.tf:1-15`
  `cloudflare_certificate_pack` — the **front-end/edge certs** Istio's Cloudflare gateway relies
  on (Course 4). DNS data in `live/cloudflare/manabie-io/dns/records.hcl` (e.g. `thanos` records
  `:376-384`).

## 7. IAM & Workload Identity — the cross-course payoff *(Lesson 9)*
**(a) Per-service GSA + WI binding** (backs `iam.gke.io/gcp-service-account`, Course 3/6):
- GSA: <span class="filepath">modules/apps/stag/sa.tf:58-66</span>
  (`google_service_account` for_each over `var.service_accounts`).
- **WI binding:** <span class="filepath">modules/apps/stag/sa_identity_namespaces.tf:41-54</span>
  — `google_service_account_iam_member`, `role = "roles/iam.workloadIdentityUser"` (`:52`),
  `member = serviceAccount:PROJECT.svc.id.goog[<ns>/<ksa>]` (`:53`). *This is the Terraform behind
  the annotation.* `gke_identity_namespace` fed from the GKE output (`_env/stag-apps.hcl:938`).
  Service list generated from `deployments/decl/<env>-defs.yaml` (`_env/stag-apps.hcl:42-43`).

**(b) GitHub Actions WIF** (Course 5):
- Pool: <span class="filepath">modules/github-oidc/main.tf:6-13</span>
  (`google_iam_workload_identity_pool`), id **`gh-action-pool`** (`live/github-oidc/env.hcl:6`).
- Per-bot providers (build/deploy/admin/integration-test/…); binding
  `member = principalSet://…/attribute.repository/<repo>`, role `workloadIdentityUser`.
- CI bot SAs: <span class="filepath">live/github-oidc/student-coach-e1e95/terragrunt.hcl</span> —
  `prod-build-bot` (`:61`, artifactregistry.writer, workflow-allowlisted), `prod-deploy-bot`
  (`:13`), `integration-test-bot` (`:97`), etc.

## 8. KMS *(Lesson 10)*
- **SOPS keys** (Course 3): <span class="filepath">modules/kms-key/kms.tf</span> — keyring
  (`:21-25`), per-service keys (`:27-37`), decrypter IAM to each service GSA (`:87-96`). Live
  keyring name **`backend-services`** (`live/kms-key/staging-manabie-online/terragrunt.hcl:68-71`),
  keys per `stag-<svc>`/`uat-<svc>` from the decl YAML.
- **Binauthz attestor keys** (Course 5): keyring **`binauthz-attestors`** from
  `modules/binary-authorization-attestor`; live
  <span class="filepath">live/stag-manabie/binary-authorization-attestor/terragrunt.hcl:16-34</span>
  (attestor `stag-deploy`, `EC_SIGN_P256_SHA256`, signer `prod-build-bot`).

## 9. Atlantis — the apply path *(Lesson 12)*
**Terraform is applied via Atlantis PR automation, NOT GitHub Actions** (CI `paths:` filters skip
`deployments/terraform/**`).
- <span class="filepath">deployments/helm/platforms/atlantis/production-values.yaml</span>:
  atlantis v0.32.0, `atlantis.manabie.io`, runs as GSA `atlantis@student-coach-e1e95` (WI, `:125-127`).
- `repoConfig` (`:26-107`): `apply_requirements: [approved]` (`:32`); custom **terragrunt**
  workflow — plan = `terragrunt init` + `terragrunt plan -out=$PLANFILE` (`:46-51`), apply =
  `terragrunt apply $PLANFILE` (`:52-64`).
- **Team allowlist** `dev:plan, func-platform:unlock, terraform-approvers:apply` (`:135`) — only
  `terraform-approvers` may `apply`. TF version pinned `1.9.8` (`:138-139`).

## 10. GCS buckets *(Lessons 3, 10)*
- **TF state:** `modules/simple-bucket` → `manabie-staging-terraform-state`
  (`live/terraform-state-bucket/staging/terragrunt.hcl:11`, `public_access_prevention: enforced`),
  prod sibling.
- **CI cache** `stag-manabie-cache`: platform `gcs_buckets` (`live/stag-manabie/env.hcl:592-617`).
- Other platform buckets in the same list (`env.hcl:512-653`), rendered by `platforms/storage.tf`.
- **`manabie-thanos` (Course 6): NOT Terraform-managed here** — only referenced by Helm values +
  Cloudflare DNS. Externally/manually provisioned.

---

## Anchors cheat-list

| Thing | Where |
|---|---|
| Terragrunt root | `live/root.hcl:11` (state), `:25` (provider), `:39` (versions) |
| A live leaf | `live/stag-manabie/platforms/terragrunt.hcl:1-16` |
| State bucket | `live/root.hcl:6-7`; `live/terraform-state-bucket/staging/…:11` |
| GKE module + binauthz | `modules/platforms/gke.tf:4,111`; node pools `env.hcl:694-1109` (`spot`) |
| Cloud SQL instances | `modules/platforms/postgresql.tf:1-8`; `stag-manabie/platforms/terragrunt.hcl` |
| **WI binding (the payoff)** | `modules/apps/stag/sa_identity_namespaces.tf:52-53` |
| GitHub WIF pool + bots | `modules/github-oidc/main.tf:6`; `live/github-oidc/student-coach-e1e95/terragrunt.hcl:61` |
| SOPS KMS keyring | `modules/kms-key/kms.tf:21`; `live/kms-key/staging-manabie-online/…:68` (`backend-services`) |
| binauthz keyring | `live/stag-manabie/binary-authorization-attestor/terragrunt.hcl:16` |
| Atlantis apply | `platforms/atlantis/production-values.yaml:32,52-64,135` |
| Thanos bucket (NOT TF) | referenced only in Helm/Cloudflare — externally provisioned |