docs: audit of the 29 empty-payload .age files under secrets/ (dead / escrow-trap / degraded) #38
No reviewers
Labels
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
key-store/key.store!38
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "audit/empty-age-payloads"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Substitute for a Forgejo issue (issues are disabled on this repo by
infra/forgejo/repository.tf). The full report is the file in this PR; the same text follows so it is readable here.Nothing was minted, restored, modified, or deleted — this is a classification only. The (a) deletions and (c) re-provisioning are decisions for Conrad.
Empty
.agepayload audit — 2026-08-1629 files under
secrets/are valid age files with zero-byte plaintext (header + recipient stanzas + MAC, no payload). agenix andage -d"succeed" on them and materialise 0-byte files, so consumers degrade silently instead of failing.How they got that way. Three commits:
4f53e0f2(gpt-5.2-codex) "rekey release secrets for forge"age -dfailed, empty stdin was piped intoage -e). Most were later restored; 8 of today's empties trace to it: finances/{credit-cards,gnosis-safe,usdc-settlement}, forge/minio-gateway, gcp/service-account, identity/agent-ssh, tuyo/bearer-token (+ dkim/staging-2025 arrived empty in the parent6976b553).ebc1c6c0(Founder) "Update nix cache, identities, and governance terms"65bc708d(gpt-5.3-codex) "rekey encrypted payloads to current recipients"The remaining empties were born empty (placeholders that were never filled): grafana/planetscale-prometheus-, identity/{forge,founder,runner,release-runner}-ssh, identity/-openpgp, nexmail/encryption-key, planetscale/*-database-url, roles/jmap-bearer, stalwart/dkim/staging-2025, forwardemail/imap-password.
Detection (all stanzas
-> ssh-ed25519; size == 102 + 110·recipients ⇒ empty):Live evidence gathered on the forge (2026-08-16):
/run/agenix/{gcpServiceAccount,minioGateway,rolesJmapBearer,grafanaPlanetscalePrometheusUrl,grafanaPlanetscalePrometheusToken}are all 0 bytes; theaccounting-key-storecontainer holds 0-byte/app/records/private/*.pdfand/app/notes/private/business-metadata.nix;key-store-garage-replicate.servicefails every hour on the missing GCS credential; Stalwart's config (stalwart.tomland its Postgres config store) contains nosignature.*/DKIM settings and nostaging-2025string anywhere — outbound mail is DKIM-signed by Forward Email under selectorfe-99839b68ec(per the FE API'ssmtp_dns_records).Recoverability. Every file that once had a real payload still has its last-good blob in git, encrypted to
founderLegacy(Conrad's~/.ssh/id_ed25519); I decrypted each towc -conly (nothing written, nothing restored) — see the byte counts below. Nothing has been minted, restored, modified, or deleted by this audit.(a) Dead — no live consumer → delete the file +
secrets.nixentry (+ listed code refs)grafana/planetscale-prometheus-url.age1a01fb8f)age.secrets.grafanaPlanetscalePrometheusUrl(nixos/keystore-forge.nix:3966) — buthosts/keystore-forge/common.nix:2290setsplanetscale.prometheusUrlFile = null; Grafana's "PlanetScale (forge)" datasource is the local scraper127.0.0.1:9090fed byplanetscale/{id,token}age.secretsblocks (keystore-forge.nix:3966-3977)grafana/planetscale-prometheus-token.ageplanetscale/forgejo-database-url.age38d1c6fd)Scripts/update-planetscale-secrets.sh,Scripts/update-forgejo-planetscale-secret.sh, Makefileforgejo-database-url,secrets/README.md §PlanetScale. Forgejo runs on local Postgres over the socket (config/databases/forgejo-socket.env); the Fly DSN path (Scripts/update-planetscale-secret.sh, singular) writes straight toflyctl secretsand never touches these filesplanetscale/stalwart-database-url.ageupdate-stalwart-planetscale-secret.sh,update-stalwart-database-secret.sh, Makefilestalwart-database-url); Stalwart is on local Postgres (stalwart-db-env.service)planetscale/plane-database-url.agesecrets.nixat all; Plane retired in88474a8d(2026-03-09)roles/jmap-bearer.agedd21d351, renamed fromcodex/)age.secrets.rolesJmapBearer(forgekeystore-forge.nix:3804, staging) → exported asCODEX_JMAP_BEARER_TOKEN_FILE(common.nix:1565, staging:122). No reader in key.store or the pinneddispatchrepo (agent-dispatcher only forwards the env var). JMAP auth is OAuth / per-identity mailbox passwords (KEYSTORE_MCP_LOCAL_AGENT_TOKEN_FILES) since9a99190aage.secretsblock + the two env lines; drop the stalecodex/jmap-bearerfromMakefileSECRETSnexmail/encryption-key.agef68e8037)nixos/modules/services/nexmailexists but is not imported/enabled on any host; KSE-0053/0054 are Draft;services/mail/nexmail.mdmake secret name=nexmail/encryption-keyonly if NexMail ever ships)stalwart/dkim/staging-2025.age6976b553)signature.*instalwart.toml, none in its DB config store, no key on disk); all outbound is relayed tosmtp.forwardemail.net, which signs withfe-99839b68ec._domainkey. The publishedstaging-2025._domainkey.key.storeTXT is an orphan (public half only) andScripts/update-mail-dns.shwill keep re-publishing it fromservices/mail/dkim/staging-2025.txtservices/mail/dkim/staging-2025.txt; then remove the orphan DNS TXT (harmless but misleading). The premise "staging-2025 signs mail today" did not hold up.forwardemail/imap-password.ageebc1c6c0)docs/pgp-wkd.md §10,secrets/README.md §Forward Email, Makefile list). The imapsync mirrors use per-identity passwords (secrets/identity/<slug>-password.age), not thismake secret name=forwardemail/imap-password) if a governance-only IMAP password should be escrowedfinances/mercury-token.ageebc1c6c0(90 B, a realsecret-token:…Mercury API token)SECRETS+ secrets.nix; no other "mercury" reference in the repo262b28adwith founderLegacy if the finance program wants it back; consider rotating it at Mercury since it sat in history)finances/credit-cards.age4f53e0f2— but the last payload (76 B) was literallyTODO:finances/README.md,skills/finance/README.mdstep 2, KSE-0004)make secret name=finances/credit-cardsif the finance skill is livefinances/gnosis-safe.ageTODO:)finances/README.md;Scripts/finances/provision-safe.shreadsgnosis-safe.template.json, not thisfinances/usdc-settlement.ageTODO:)finances/README.mdidentity/founder-openpgp.age,identity/forge-openpgp.age,identity/agent-openpgp.ageebc1c6c0)Scripts/export-openpgp-secret.sh;docs/wkd-workbook.md(every checkbox still unchecked); no reader — WKD publishes public keys onlyScripts/export-openpgp-secret.sh --scope <tier> <identity>)forge/minio-gateway.age4f53e0f2(180 B, realMINIO_ROOT_USER/PASSWORDenv)secrets.nix(a plainagenix -rnever touches it);age.secrets.minioGatewayinkeystore-forge.nix:3888materialises an unused 0-byte file on prod; the only reader isservices.key.store.minioGateway.credentialsFileon the staging host def (hosts/keystore-forge-staging/default.nix:104-114). Prod blob storage moved to Garage (e69a8d6c); no minio unit on prodage.secretsblock. If staging is still meant to run the MinIO→GCS gateway (KSE-0007/0009 both Draft) it needs a fresh env file (make secret name=forge/minio-gatewayafter adding it tosecrets.nix) — but that host'sgcpServiceAccountFileis also empty, so it cannot work today regardless(b) Consumed-by-convention but working anyway — real value lives elsewhere; the empty file is a latent trap
These are escrows of the very SSH keys that are the agenix recipients / git signing identities (
config/git-identities/*.pub). Nothing decrypts them at runtime;secrets/README.md §Git Signing IdentitiesandScripts/git-identity.sh generatedescribe them as the backup. Concrete symptom already observed: this workstation's~/.ssh/key.store/{agent,founder}are 0-byte files dated 2026-07-17 — someone materialised identities from the empty escrows and got nothing.identity/forge-ssh.ageforge@key.store= agenix recipientforge= trusted commit signer = runner age identity/root/.ssh/forge-ed25519on the forge (also/var/lib/forgejo-runner-agent/age_keystore,/var/lib/forgejo-nsc/age_keystore, HQ, and the Forgejo repo secretAGE_FORGE_SSH_KEY)forgeAutomation/releasesecret → full rekey from the founder key. Recipients (founder founderLegacy forge runner) make this a founder-readable escrow, so it is worth fillingmake secret-file name=identity/forge-ssh file=<copy of forge-ed25519>from a host that has itidentity/founder-ssh.age…IA4grzc0)docs/ssh-devshell.md)make secret-file name=identity/founder-ssh file=…identity/agent-ssh.ageagent@key.store(recipient onmodelAgents/financeDisclosuresecrets)/root/.ssh, dispatcher/codex/runner dirs are all the forge key), not on this workstation (0-byte). Recovered blob6976b553:secrets/identity/agent-ssh.age(411 B) decrypts and its embedded pubkey matches the currentagentrecipientagentrecipient is decorative — everything scopedmodelAgents/financeDisclosurewithoutforgeAutomation(finances/*, tuyo, agent-openpgp) is readable only by the founder keys, never by an agentgit show 6976b553:secrets/identity/agent-ssh.age | age -d -i ~/.ssh/id_ed25519 > /tmp/agent && make secret-file name=identity/agent-ssh file=/tmp/agent— or rotate viaScripts/git-identity.sh generate agent(then updatesecrets.nix, allowed_signers, rekey)identity/runner-ssh.age,identity/release-runner-ssh.agerunner,releaseRunnerrecipients (release ++ [runner]on cloud-/azure-signing etc.)ageIdentityPath = /root/.ssh/forge-ed25519for bothforgejo-runnerandforgejo-nsc)Scripts/git-identity.sh generate runner/release-runner) so the escrow is real, or drop the two recipients fromsecrets.nixand delete the escrows(c) Consumed and actually broken / silently degraded — needs re-provisioning by Conrad
gcp/service-account.age4f53e0f2(2,347 B; SArelease@key-d0t-store.iam.gserviceaccount.com, key idb629a27f…)key-store-garage-replicate(hourly) → fails every run: "GCS archival mirror never ran (missing /run/agenix/gcpServiceAccount)". PR #36 (fix/garage-replicate-hetzner-only) downgrades this to a warning (garageGcsArchiveEnabled = false). Also stagingminioGateway.gcpServiceAccountFilemake secret-file name=gcp/service-account file=sa.jsonwithroles/storage.objectAdminonkey-store-garage-archive. Do not restore the historical key blindly — check whether keyb629a27f…still exists/was revoked in IAM firstrecords/2026-01-11_Formations.pdf.age(22.5 KB),records/2026-01-12_Formation_Approved.pdf.age(531 KB),records/2026-01-14_CP_575_Letter.pdf.age(12.5 KB, contains EIN),records/2026-01-14_Operating_Agreement.pdf.age(89.9 KB)ebc1c6c0accounting-key-storecontainer (accounting.key.store) decryptssecrets/records/*.ageat boot with/root/.ssh/forge-ed25519→ 0-byte PDFs in/app/records/private;finances/records.beancountdocumentdirectives point at them (Fava shows empty documents);Scripts/decrypt-records.shsame5770331d(blobs decrypt with founderLegacy, all valid%PDF-):git show 5770331d:secrets/records/<f>.age | age -d -i ~/.ssh/id_ed25519 > <f>; make secret-file name=records/<f> file=<f>(scopefinanceRecords)notes/business-metadata.nix.ageebc1c6c0(887 B: legalName, EIN, banking, addresses…)/app/notes/private/business-metadata.nix;notes/public/README.md, KSE-0023a63b62b3the same way, or re-fill fromnotes/templates/business-metadata.nixtuyo/bearer-token.age4f53e0f2(323 B JWT)finances/Makefile tuyo-snapshot/finances/scripts/tuyo_snapshot.py(TUYO_BEARER). Last snapshot data 2025-12-24; human-run onlymake secret name=tuyo/bearer-token(or move to (a) if the Tuyo feed is abandoned)Follow-ups (not done here)
Scripts/agenix-load-file.sh/ any rekey path refuse zero-length plaintext) so a lossy rekey can never land again.secrets.nix(forge/minio-gateway.age,planetscale/plane-database-url.age);Makefile SECRETSstill lists the pre-renamecodex/jmap-bearer.keystore-mail-architecturenotes: DKIM is Forward Email'sfe-99839b68ec, notstaging-2025.key-store/key.storebyinfra/forgejo/repository.tf(has_issues = false), so this landed as a PR instead of an issue.fa00406bfea360944467