KSE-0182 Amendment A: workload identity plane and agenix -> OpenBao cutover #39

Merged

founder merged 2 commits from kse/0182-amendment-a-workload-identity into main

2026-08-16 08:29:57 +00:00

founder commented

2026-08-16 08:10:56 +00:00

Owner

Summary

Founder direction 2026-08-16: move secrets into OpenBao, WIF for everything, deployed with OpenTofu. This records the decisions and a phased plan as Amendment A of KSE-0182 (plus a scope note on KSE-0174). Proposal only — no runtime changes.

What it fixes in the record

OpenBao is already on main and enabled on the forge but has never come up (activating (auto-restart-queued) since 2026-06-14). Reproduced first blocker: Postgres socket 0770 postgres:postgres vs DynamicUser=openbao with no SupplementaryGroups=postgres; behind that, the gcpckms seal has no Google credential — the same gap that keeps the Garage GCS archive leg off (PR #36).
Forgejo 15.0.2 cannot mint job-scoped OIDC tokens (verified in the binary), so "Forgejo WIF" is anchored on Authentik as the machine-identity issuer (already trusted by the Google WIF pool and by infra/openbao's jwt auth), one client-credentials provider per workload class.
One forge-host Google credential (google-wif-credentials.service → /run/google-wif/credentials.json, external_account) serves the OpenBao seal, the rclone GCS leg (env_auth), and cloud-tofu.
agenix shrinks to the bootstrap set; runtime secrets live in OpenBao (bao-agent + LoadCredential on the forge; jwt/AppRole login in CI; decrypt-secrets → openbao-secrets; then delete the AGE_FORGE_SSH_KEY Actions secret used by 22 workflows).
OpenTofu declares WIF/IAM (infra/identity) and auth roles/policies/mounts (infra/openbao); never secret values.
Phases 0–4 with exit criteria; Phase 0 = PR #36 (landed). Phase 3 gated on the Phase 2 unseal-recovery drill.

Decisions to confirm or change on review

Authentik as the sole machine issuer (vs. X.509 WIF for the host / patching Forgejo) — accepted boot-order risk: OpenBao auto-unseal needs Authentik reachable; mitigations listed in A.2.
The bootstrap set that stays in agenix (A.3).
Direct-resource IAM on WIF principals (no service-account impersonation) unless a Google API refuses federated principals.

🤖 Generated with Claude Code

## Summary Founder direction 2026-08-16: **move secrets into OpenBao, WIF for everything, deployed with OpenTofu.** This records the decisions and a phased plan as **Amendment A of KSE-0182** (plus a scope note on KSE-0174). Proposal only — no runtime changes. ## What it fixes in the record - OpenBao is already on `main` and enabled on the forge but has **never come up** (`activating (auto-restart-queued)` since 2026-06-14). Reproduced first blocker: Postgres socket `0770 postgres:postgres` vs `DynamicUser=openbao` with no `SupplementaryGroups=postgres`; behind that, the `gcpckms` seal has no Google credential — the same gap that keeps the Garage GCS archive leg off (PR #36). - **Forgejo 15.0.2 cannot mint job-scoped OIDC tokens** (verified in the binary), so "Forgejo WIF" is anchored on **Authentik as the machine-identity issuer** (already trusted by the Google WIF pool and by `infra/openbao`'s jwt auth), one client-credentials provider per workload class. - One forge-host Google credential (`google-wif-credentials.service` → `/run/google-wif/credentials.json`, `external_account`) serves the OpenBao seal, the rclone GCS leg (`env_auth`), and cloud-tofu. - agenix shrinks to the bootstrap set; runtime secrets live in OpenBao (bao-agent + `LoadCredential` on the forge; `jwt`/AppRole login in CI; `decrypt-secrets` → `openbao-secrets`; then delete the `AGE_FORGE_SSH_KEY` Actions secret used by 22 workflows). - OpenTofu declares WIF/IAM (`infra/identity`) and auth roles/policies/mounts (`infra/openbao`); never secret values. - Phases 0–4 with exit criteria; Phase 0 = PR #36 (landed). Phase 3 gated on the Phase 2 unseal-recovery drill. ## Decisions to confirm or change on review 1. Authentik as the sole machine issuer (vs. X.509 WIF for the host / patching Forgejo) — accepted boot-order risk: OpenBao auto-unseal needs Authentik reachable; mitigations listed in A.2. 2. The bootstrap set that stays in agenix (A.3). 3. Direct-resource IAM on WIF principals (no service-account impersonation) unless a Google API refuses federated principals. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

founder added 1 commit

2026-08-16 08:10:56 +00:00

KSE-0182 Amendment A: workload identity plane and agenix -> OpenBao cutover 5e31f6bce6

Founder direction 2026-08-16: move secrets into OpenBao, use workload
identity federation for everything, deploy it with OpenTofu.

Records what was established while fixing the forge deploy status
(PR #36) and turns it into decisions + phases:

- OpenBao is already on main and enabled on the forge but has never come
  up (activating/auto-restart-queued since 2026-06-14): Postgres socket is
  0770 postgres:postgres and the DynamicUser unit has no
  SupplementaryGroups=postgres; behind that the gcpckms seal has no Google
  credential -- the same gap that keeps the Garage GCS archive leg off.
- Forgejo 15.0.2 cannot mint job-scoped OIDC tokens (verified), so
  "Forgejo WIF" is anchored on Authentik as the machine-identity issuer:
  one OAuth2 client-credentials provider per workload class, trusted by
  Google WIF, Namespace, and OpenBao's jwt auth.
- One forge-host Google credential (google-wif-credentials.service ->
  /run/google-wif/credentials.json, external_account) feeds the OpenBao
  seal, the rclone GCS leg (env_auth) and cloud-tofu.
- agenix shrinks to the bootstrap set; everything else lives in OpenBao,
  materialised via bao-agent + LoadCredential for forge services and via
  jwt/AppRole login for CI (decrypt-secrets -> openbao-secrets, then
  delete the AGE_FORGE_SSH_KEY Actions secret).
- OpenTofu declares WIF/IAM (infra/identity), auth roles/policies/mounts
  (infra/openbao); never secret values.
- Phases 0-4 with exit criteria; Phase 0 = PR #36 (landed).

KSE-0174 gets a scope note pointing here (its runtime-token bootstrap
fallback is superseded by A.3).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

founder requested reviews from billing, cmo, cdo, runner, infra, devex, manager, windows, android, web, assistant, release-runner, security-agent, ios, linux, forge, agent

2026-08-16 08:10:56 +00:00

founder commented

2026-08-16 08:26:14 +00:00

Author

Owner

Revision pushed (2nd commit) — correction + design change.

The first draft claimed Forgejo 15.0.2 cannot mint job-scoped OIDC tokens, "verified in the binary". That was wrong: the grep ran against the 1 KB nix wrapper at bin/forgejo, not bin/.forgejo-wrapped. Against the real binary, the v15.0 source and the live instance, the feature is already live on the forge: issuer https://git.key.store/api/actions, discovery/JWKS at /api/actions/.well-known/{openid-configuration,keys} (200 today), RS256, sub = repo:<owner/repo>:ref:<ref> (or :pull_request), claims repository/ref/workflow_ref/event_name/actor/run_id/…, gated per job by permissions: id-token: write; forgejo-runner 12.10.2 (deployed) exposes ACTIONS_ID_TOKEN_REQUEST_URL/TOKEN. Forgejo 16 hardens it (#12030, #11194, #12275) — upgrade is a founder call since it leaves the 15 LTS line.

Design now: Forgejo issues identity for every Actions job on every runner class (Google/Azure WIF providers + an OpenBao jwt auth backend forgejo-actions bound on repository/ref/workflow_ref; PR runs get no role; no AppRole or runner-side credential for CI). Authentik issues for the forge host (seal, mirrors), KSE-0181 residencies, and humans. Phase 1 adds Forgejo issuer hygiene (explicit signing-key path + backup, 600 s TTL) and makes cloud-tofu.yml the first job-token consumer.

**Revision pushed (2nd commit) — correction + design change.** The first draft claimed Forgejo 15.0.2 cannot mint job-scoped OIDC tokens, "verified in the binary". That was wrong: the grep ran against the 1 KB nix wrapper at `bin/forgejo`, not `bin/.forgejo-wrapped`. Against the real binary, the v15.0 source and the live instance, the feature is **already live on the forge**: issuer `https://git.key.store/api/actions`, discovery/JWKS at `/api/actions/.well-known/{openid-configuration,keys}` (200 today), RS256, `sub = repo:<owner/repo>:ref:<ref>` (or `:pull_request`), claims `repository`/`ref`/`workflow_ref`/`event_name`/`actor`/`run_id`/…, gated per job by `permissions: id-token: write`; forgejo-runner 12.10.2 (deployed) exposes `ACTIONS_ID_TOKEN_REQUEST_URL/TOKEN`. Forgejo 16 hardens it (#12030, #11194, #12275) — upgrade is a founder call since it leaves the 15 LTS line. Design now: **Forgejo issues identity for every Actions job on every runner class** (Google/Azure WIF providers + an OpenBao `jwt` auth backend `forgejo-actions` bound on `repository`/`ref`/`workflow_ref`; PR runs get no role; no AppRole or runner-side credential for CI). **Authentik** issues for the forge host (seal, mirrors), KSE-0181 residencies, and humans. Phase 1 adds Forgejo issuer hygiene (explicit signing-key path + backup, 600 s TTL) and makes `cloud-tofu.yml` the first job-token consumer.

founder added 1 commit

2026-08-16 08:26:16 +00:00

KSE-0182 Amendment A: Forgejo is the CI issuer -- 15.0.2 already mints job tokens 80e10221b4

Correction: the first draft said Forgejo 15.0.2 cannot mint job-scoped OIDC
tokens "verified in the binary". That grep ran against the 1 KB nix wrapper
at bin/forgejo, not bin/.forgejo-wrapped. Against the real binary, the v15.0
source (routers/api/actions/{oidc,id_token}.go) and the live instance:

- issuer https://git.key.store/api/actions, discovery + JWKS at
  /api/actions/.well-known/{openid-configuration,keys} (200 today), RS256,
  TTL [actions] ID_TOKEN_EXPIRATION_TIME (3600 s default), key auto-generated
  at <AppDataPath>/actions_id_token/private.pem
- job token via /_apis/pipelines/workflows/{run_id}/idtoken?audience=...
  with the runtime token when the job has permissions: id-token: write;
  forgejo-runner 12.10.2 (deployed) exposes ACTIONS_ID_TOKEN_REQUEST_*
- sub = repo:<owner/repo>:ref:<ref> | ...:pull_request; claims repository,
  ref, workflow_ref, event_name, actor, run_id, sha, ...
- Forgejo 16 hardens it (#12030 authorisation, #11194 key config, #12275
  JWKS caching); upgrade is a founder call (leaves the 15 LTS line).

Design change: two issuers. Forgejo issues identity for every Actions job on
every runner class (Google/Azure WIF providers + an OpenBao jwt auth backend
"forgejo-actions" bound on repository/ref/workflow_ref; PR runs get no
role). Authentik issues for the forge host (seal, mirrors), KSE-0181
residencies, and humans. No AppRole/runner-side credential for CI. Phase 1
adds Forgejo issuer hygiene (explicit signing-key path + backup, 600 s TTL)
and cloud-tofu as the first job-token consumer. Non-goal "patching Forgejo"
removed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>