key.store agent-dispatcher

Unified intake service for inbound operations events, role routing, and resident identity steers.

This directory is the formal dispatcher codebase for key.store. Webhook topology is source-controlled in topology.toml; Nix passes that file to the API service and local callers can inspect the active, secret-free view at GET /topology.

Inbound webhook paths

• POST /webhook/redmine
• POST /webhook/forgejo
• POST /webhook/zulip
• POST /webhook/forwardemail
• POST /webhook/sms
• POST /webhook/stable
• POST /webhook/minecraft
• POST /webhook/grafana
• GET /healthz
• GET /topology (loopback only)

All webhook traffic is expected to come through https://dispatch.key.store.

Communications routing

• Forward Email aliases can post directly to /webhook/forwardemail; the dispatcher dedupes on identity + message-id.
• With --forwardemail-wake-command configured (the forge), /webhook/forwardemail does not enqueue the posted payload: Stalwart stays canonical, so the webhook wakes the alias's imapsync mirror and then a targeted agent-dispatcher mail --once --message-id <id> intake against Stalwart. Without a wake command the webhook enqueues directly.
• Email intake (agent-dispatcher mail) can still poll IMAP, sharing the same dedupe path so webhook-first and mailbox-fetch flows do not double-enqueue. The targeted (--message-id) intake asks the server for SEARCH HEADER Message-ID first but does not trust an empty answer: Stalwart returns nothing for HEADER searches on unindexed fields, so the intake then matches the header client-side over the unseen messages plus the newest 50.
• Mail intake first enqueues a resident-thread steer. The receiving identity must absorb the steer before any side effect; mail intake has no SMS sender.
• Redmine should be searched before creating duplicate todos from mail, Forgejo, SMS, or wake notifications.
• Mail sender policy is fail-closed for execution. Trusted senders can steer the owning role; untrusted mail routes only to the configured review role identity, keeps the requested role in payload context, and must not trigger repository, secret, DNS, or production changes without later human approval.
• Zulip, native: every resident identity is a real Zulip user, and the supervisor long-polls a Zulip event queue as each identity it serves (zulip_listener.go, --zulip-listen, default on). A message that mentions the identity (@**Web** ..., personal or group mention — @all is ignored) or a direct message to it becomes a zulip steer for that identity with the webhook payload shape, so the completed-turn relay answers in the same channel/topic or DM as the identity; the listener also keeps the identity marked present. Identities can address each other this way (Zulip never fires outgoing webhooks for bot-sent messages, which is why the accounts stay users rather than bots) — but a message from another resident never starts a turn: every steer carries kick (humans and systems kick; a notification whose actor is a resident identity — Zulip sender, Forgejo sender, Redmine author, mail from an identity mailbox — does not). Non-kicking steers queue until a human kicks that identity, then ride into that turn (or steer straight into a turn that is already running). Residents can talk to each other; only humans start lifetimes, so there is nothing to rate-limit. The shared Agent Dispatcher outgoing-webhook bot remains as the alias-routed fallback (@Agent Dispatcher @ios ...), and /webhook/zulip also honours bot_email if an identity is ever run as a bot.
• Zulip replies are relayed from completed resident-thread output. The identity absorbs the steer first, then the supervisor posts the final assistant answer back to the originating stream/topic or direct message as the mapped Zulip identity.
• SMS/calls (/webhook/sms) and physical mail (stable) can be delivered via signed webhooks.
• SMS-compatible sources default to the assistant role and assistant identity, and stable defaults to the coo role and manager identity, unless an explicit mention overrides routing.
• SMS does not use a completion relay. The assistant must decide in the resident thread whether the inbound text needs a response and call communications_sms_send before ending the turn when it should text back.
• Forgejo webhooks route explicit role mentions and identity-specific issue assignments or pull request review requests. Assigned/review-requested identities wake their resident role thread even when the Forgejo body has no @role/... or alias mention.
• Grafana Alerting posts to /webhook/grafana through the provisioned key.store dispatcher contact point. Grafana signs the raw body with HMAC-SHA256 and includes the configured timestamp header; labels or annotations can route with key_store_role, role_id, key_store_identity, or role aliases such as team=infra. Alerts default to the infra identity.
• The resident-agent shape is one long-running thread per identity, served by one of two runtimes chosen in org/organization.toml (agent.runtime on the role, runtime on a tier, or agent_runtime.default_runtime; default codex): the Codex app-server, or Claude Code driven headless (claude -p --input-format stream-json --output-format stream-json, one process per thread bound to a stable --session-id, resumed after crashes, --append-system-prompt for the identity instructions, --effort from the role's reasoning effort, the forge control-plane MCP hub via --mcp-config with the bearer token expanded from the task environment). Incoming notifications become steers into that identity thread; the dispatcher no longer exposes a separate task executor path. Steers that arrive mid-turn are injected into the running turn on both runtimes (--claude-steering=false makes Claude finish the turn first).

Resident Steer Execution

• Resident identity threads use the configured role model, reasoning effort, sandbox, and approval policy. Supervisor flags for the Claude runtime: --claude <path> (empty disables it), --claude-config-dir <root> (per-identity CLAUDE_CONFIG_DIR, sessions/settings/auto-memory), --anthropic-api-key-file <file> (exported as ANTHROPIC_API_KEY to claude processes only), --claude-steering (default true). Claude runs with --permission-mode acceptEdits and an explicit tool allow list because a resident has no approval channel; bypassPermissions is refused as root.
• Queue state lives in forge-local PostgreSQL via AGENT_DISPATCHER_DATABASE_URL; legacy sqlite/task imports are no longer part of the live path.
• Mail-to-work links live beside the steer queue: mail_redmine_links records JMAP/RFC message ids, dispatcher event keys, steer ids, and Redmine issue ids without duplicating message bodies.
• Resident environments receive role-scoped Redmine, Forgejo, JMAP, Zulip, communications, physical-mail, and MCP credentials so primary-session tools can read/update the same event the dispatcher routed from.
• Resident environments project org/organization.toml agent runtime policy into the app-server thread. The supervisor prewarms every mapped identity for an identity-mode role; thread persistence, app-server clients, identity-bound credentials, and multi-identity work roots are scoped by role and identity so identities that share a role, such as agent and assistant, do not replace each other or inherit each other's tokens. Role-specific environment variables still override model and provider defaults for recovery changes.
• Resident environments also project the per-identity workspace placement from org/organization.toml. Every identity has an explicit host/surface/profile/tooling contract: iOS on macOS Namespace, Android on the Android Namespace image, Windows on the Windows lane, assistant on HQ, and the remaining identities on appropriate Linux Namespace or forge surfaces.
• Minecraft and office presence are primary-session MCP tools. The dispatcher no longer requires a dedicated Minecraft sub-agent.

MCP and search posture

Checkpoints preserve ongoing work (KSE-0197)

The OpenRouter harness treats its tool-round limit as a checkpoint, not a successful task outcome. It resumes the same session with tools available again, preserves late-arriving steers in the append-only journal, and never reruns calls returned during a tool-free checkpoint. Completed effects remain in the transcript; the continuation asks the resident to inspect state, finish remaining work, and synchronize its own signed memory before finishing.

There is no arbitrary limit on the number of checkpoints. The resident finishes only after a sole complete_task call with outcome: completed or blocked and an accurate summary, or when the turn is cancelled, or a provider/budget error stops execution. A checkpoint alone never marks its steers turn_completed. Prose updates keep the task active. A structured blocker is recorded as blocked, never successful completion or a transient retry. Completion closes atomically against new steers; an accepted late steer keeps the same task active. Each provider request still uses the existing usage recorder and budget policy; the founder's $50 monthly cap remains unchanged. Restart recovery keeps the existing durable steer and journal behavior; this change does not remove HQ's forge queue/MCP dependencies or guarantee a model-authored memory commit. Transient failures use the existing supervisor recovery path, separate from checkpoint continuation.

• dispatch.key.store/mcp is the shared Authentik-backed MCP hub for contributor clients and the per-identity loopback bearer hub for forge-resident agents.
• The public plugin metadata should mention the broad tool surface so Codex tool search can discover the hub for mail, Redmine, Forgejo, SMS, office/Minecraft, and observability work.
• Stalwart stores message data, blobs, lookup, and full-text search in PostgreSQL. The stalwart and agent_dispatcher databases are part of the daily PostgreSQL backup set.
• The mail MCP namespace exposes JMAP-native mailbox state, search, snippets, fetch, thread context, draft/submit/send, calendar event search/create, contact search, and readiness tools rather than inventing a separate message-body index first.
• Meeting-backed calendar events use the Stalwart/JMAP calendar create tool with meet.key.store Jitsi URLs embedded in the event metadata; Stalwart remains the source of truth for calendar state and scheduling mail.
• JMAP search, fetch, and thread results are enriched with dispatcher steer links and stored Redmine issue links. Use mail_jmap_link_redmine_issue after an identity decides that an email needs durable Redmine follow-up.

Resident commit signing (KSE-0192)

• key-store-kms-sshsig is git's gpg.ssh.program in resident workspaces: an ssh-keygen -Y sign drop-in whose private key is a Google Cloud KMS ASYMMETRIC_SIGN key (EC_SIGN_P256_SHA256 → ecdsa-sha2-nistp256, or EC_SIGN_ED25519 → ssh-ed25519), reached through Application Default Credentials (GOOGLE_APPLICATION_CREDENTIALS = the forge's WIF external_account file; KEY_STORE_KMS_ACCESS_TOKEN overrides for tests/ops). It writes standard sshsig blocks that ssh-keygen -Y verify and the forge's pre-receive hook accept, and delegates every non-sign invocation (-Y verify, -Y find-principals) to the real ssh-keygen.
• key-store-kms-sshsig pubkey --key <cryptoKey|cryptoKeyVersion> --comment <principal> prints the OpenSSH public key line (newest ENABLED version) with a kms:<version> comment; user.signingkey points at a file holding that line and the signer reads the KMS resource back from it.

Helper CLIs

• key-store-redmine whoami
• redmine_issue_search is available through MCP for duplicate checks before creating durable work.
• key-store-redmine issue show <id>
• key-store-redmine issue create --subject "..." --description "..."
• key-store-redmine issue note <id> --message "..." or note text on stdin
• key-store-redmine issue update <id> --status-id <n> [--assigned-to-id <n>] [--notes "..."]
• key-store-zulip send --message "..." for office-channel updates.
• communications_sms_send, communications_sms_messages, communications_phone_numbers, and communications_voice_calls are available through MCP for assistant-owned communications.
• stable_mail_items is available through MCP for COO-owned physical-mail lookup.

Provider scripts

• Scripts/communications-fetch-events.sh
• Scripts/communications-send-sms.sh
• Scripts/communications-provision-webhook.sh
• Scripts/stable-fetch-mail.sh
• Scripts/stable-provision-webhook.sh

These scripts assume communications and Stable API keys are managed with agenix:

• secrets/communications/api-key.age
• secrets/stable/api-key.age

Current provider defaults:

• SMS/voice provider API: https://api.openphone.com
• Stable API: https://api.usestable.com
• Stable webhook endpoints are configured in the Stable Dashboard; the local helper validates API access and prints the target/events for manual dashboard setup.

Smoke checks

• services/redmine/redmine-smoke.sh
• services/redmine/redmine-api-smoke.sh
• dispatcher-smoke.sh
• identity-routing-smoke.sh
• identity-placement-smoke.sh
• services/mcp-hub/mcp-hub-smoke.sh

dispatcher-smoke.sh proves the dispatcher HTTP surface, including the Zulip webhook path. identity-placement-smoke.sh fails if forge regains a dedicated iOS supervisor or stops excluding ios and assistant from the forge-owned supervisor.

identity-routing-smoke.sh <api base url> <database url> posts one synthetic, no-action event per webhook source under a stable namespace and asserts that the resulting steer reaches the expected source/role/identity and is delivered (set KEYSTORE_ROUTING_SMOKE_EVENT_SUFFIX to force fresh events). The forge runs it from redmine-dispatcher-smoke.service.

Its mail legs (mail-trusted-billing-cfo, mail-trusted-billing-assistant, mail-trusted-security, mail-review-billing) go through the wake path described above, so the synthetic message must already be in Stalwart when the webhook fires. Before each webhook the smoke therefore APPENDs the same message (Message-ID, From, To, Subject, no-action body) into the target account's INBOX over IMAP, keyed by the Forward Email alias it is addressed to:

• KEYSTORE_ROUTING_SMOKE_MAIL_ACCOUNTS — <alias>=<stalwart account>:<password file>[,...], e.g. billing=billing:/run/agenix/billingPassword,security=security-agent:/run/agenix/securityAgentPassword. This is the same alias → account mapping the key-store-forwardemail-wake@ unit uses; the forge derives both from forwardEmailMirrorMailboxes in nixos/keystore-forge.nix. Unset it for deployments without a wake command, where the webhook enqueues directly and no seeding is needed.
• KEYSTORE_ROUTING_SMOKE_IMAP_ADDR — Stalwart IMAP address (default 127.0.0.1:993; IMAPS on 993 or STARTTLS otherwise, certificate verification skipped on loopback like the dispatcher's own intake).

Seeding is idempotent: the smoke reuses stable Message-IDs across runs and leaves an existing copy alone, and the intake dedupes on identity + message-id, so repeated runs neither pile up mailbox copies nor create new steers. Recipients are the real Forward Email aliases (security@key.store, not the security-agent Stalwart login) because the wake command only mirrors those.

HQ assistant resident supervisor

• nixos/keystore-hq.nix runs the assistant resident supervisor on HQ. It keeps the Codex app-server and communications-capable assistant runtime on the failsafe node while using an SSH tunnel to forge PostgreSQL for the shared steer queue.
• Forge excludes assistant, so signed SMS and HQ watchdog steers have one canonical assistant consumer instead of competing forge and HQ sessions.

macOS iOS resident supervisor

• Scripts/run-ios-macos-agent-supervisor.sh runs the tech_lead_apple / ios supervisor locally on macOS. It starts a local Codex app-server and uses an SSH tunnel to forge PostgreSQL only for queue access when AGENT_DISPATCHER_DATABASE_URL is not set.
• Scripts/install-ios-macos-agent-launchagent.sh installs the no-sudo per-user LaunchAgent for the local Mac.
• Scripts/install-ios-macos-agent-launchd.sh installs the system LaunchDaemon variant for managed macOS hosts where root installation is available.