office: back up the world, bridge chat to Zulip, move Codex to API-key auth #15

Merged

founder merged 1 commit from office-backups-bridge-and-codex-apikey into main

2026-07-26 21:03:47 +00:00

founder commented

2026-07-26 21:03:42 +00:00

Owner

The resident agent fleet has been down since 2026-06-16. The ChatGPT auth seed expired that day; the supervisor has been spinning on Failed to refresh token while ~740 steers piled up (609 SMS, 255 mail) — routed correctly, never answered.

Codex auth → API key. The existing API-key branch was unreachable: install_auth_seed_if_newer succeeds whenever the seed merely exists, and an api-key auth.json has no last_refresh, so the expired seed always compared "newer" and would clobber a working login on the next activation. Now behind codexPreferApiKey.
Model ids move with auth mode. ChatGPT served gpt-5.4; the API key's /v1/models does not. Every role resolving through organization.toml would fail model_not_found even once authenticated.
World backups. The office world had no backup — not in the postgres set, not replicated. Daily snapshot via save-off/save-all flush over the existing stdin FIFO (no RCON, no plugin), waits for Paper's real "Saved the game" line, restores saving via trap. Weekly restore-verification job.
Chat bridge. In-game chat/presence → Zulip #office; @assistant messages → signed dispatcher webhook, so office questions become resident steers on the same queue as mail/Zulip.
Pinned the office web-proxy image by digest (was :latest).

Verified live: backup flush 2s + upload size-checked; restore verified twice (local, and Garage-only with the local copy deleted); bridge confirmed posting to Zulip and enqueuing an assistant steer.

Still blocked: the OpenAI key returns insufficient_quota on every model, so no turn can run until billing is topped up. Supervisor is intentionally left stopped so ~740 queued steers don't all fire at once.

🤖 Generated with Claude Code

**The resident agent fleet has been down since 2026-06-16.** The ChatGPT auth seed expired that day; the supervisor has been spinning on `Failed to refresh token` while ~740 steers piled up (609 SMS, 255 mail) — routed correctly, never answered. - **Codex auth → API key.** The existing API-key branch was *unreachable*: `install_auth_seed_if_newer` succeeds whenever the seed merely exists, and an api-key `auth.json` has no `last_refresh`, so the expired seed always compared "newer" and would clobber a working login on the next activation. Now behind `codexPreferApiKey`. - **Model ids move with auth mode.** ChatGPT served `gpt-5.4`; the API key's `/v1/models` does not. Every role resolving through `organization.toml` would fail `model_not_found` even once authenticated. - **World backups.** The office world had *no* backup — not in the postgres set, not replicated. Daily snapshot via `save-off`/`save-all flush` over the existing stdin FIFO (no RCON, no plugin), waits for Paper's real "Saved the game" line, restores saving via trap. Weekly restore-verification job. - **Chat bridge.** In-game chat/presence → Zulip `#office`; `@assistant` messages → signed dispatcher webhook, so office questions become resident steers on the same queue as mail/Zulip. - **Pinned** the office web-proxy image by digest (was `:latest`). **Verified live:** backup flush 2s + upload size-checked; restore verified twice (local, and Garage-only with the local copy deleted); bridge confirmed posting to Zulip and enqueuing an assistant steer. **Still blocked:** the OpenAI key returns `insufficient_quota` on every model, so no turn can run until billing is topped up. Supervisor is intentionally left stopped so ~740 queued steers don't all fire at once. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

founder added 1 commit

2026-07-26 21:03:42 +00:00

office: back up the world, bridge chat to Zulip, move Codex to API-key auth

Namespace: Build Images / Refresh macOS runner lane (push) Successful in 6s

Details

Release: If Needed / Check (Release Needed) (push) Successful in 6s

Details

Namespace: Build Images / Refresh Linux runner image (push) Successful in 1m28s

Details

Deploy: API / Deploy (API) (push) Successful in 2m59s

Details

Deploy: Web / Deploy (Web) (push) Successful in 6m22s

Details

Build: Apple / Check (SwiftLint) (pull_request) Failing after 20s

Details

Build: Apple / Build (macOS) (pull_request) Has been skipped

Details

Build: Apple / Build (visionOS) (pull_request) Has been skipped

Details

Build: Apple / Build (iOS) (pull_request) Has been skipped

Details

e5211d1948

Three findings, in increasing order of severity.

The office world had no backup of any kind. It is not in the postgres backup
set and nothing replicated /var/lib/minecraft, so the only copy of `office-web`
was the live one. Add a daily snapshot that drives `save-off` / `save-all flush`
over the stdin FIFO minecraft-server.socket already exposes -- no RCON, no
plugin -- waits for Paper's actual "Saved the game" line rather than a guessed
sleep, and restores saving via trap so an abort cannot leave the world unable to
save. Archives go to the Garage backups bucket beside the database dumps, with a
weekly job that extracts the newest archive and asserts level.dat plus region
files are really there. Verified live: flush in 2s, upload size-checked, and
restore verified twice -- once locally, once with the local copy deleted so it
had to come back from Garage.

Nothing carried in-game chat anywhere. Add a bridge that mirrors chat and
presence into Zulip #office and hands @assistant-addressed messages to the
dispatcher as a signed webhook, so office questions become resident-identity
steers on the same queue as mail and Zulip instead of a side-channel chatbot.
Verified end to end against the live forge: Zulip received the messages and the
dispatcher enqueued a steer routed to the assistant identity.

The resident fleet has been dead since 2026-06-16. The ChatGPT auth seed expired
that day and the supervisor has been spinning on "Failed to refresh token" ever
since, while ~740 steers (609 SMS, 255 mail) piled up unprocessed -- routed
correctly, never answered. Switch to API-key auth. The existing API-key branch in
key-store-codex-login was unreachable: install_auth_seed_if_newer succeeds
whenever the seed merely exists, so the fallback only ran if the secret was
missing entirely, and an api-key auth.json carries no `last_refresh`, so the
expired seed always compared "newer" and would clobber a working login on the
next activation. Prefer the API key explicitly behind codexPreferApiKey.

Model ids move with the auth mode: the ChatGPT account served `gpt-5.4`, which
the API key's /v1/models does not, so every role resolving through
organization.toml would have failed with model_not_found even once authenticated.

Also pin the office web-proxy image by digest; it was :latest, the only unpinned
image in a path every office session depends on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>