dispatcher: requeue-safe teardown, macOS TTL enforcement, cache volumes on macOS/Windows create (KSE-0186 D8) #2

Merged

founder merged 1 commit from claude/d8-dispatcher-fixes into main 2026-08-17 00:06:11 +00:00

Conversation 0 Commits 1 Files changed 12 +2087 -92

founder commented 2026-08-17 00:06:11 +00:00

Owner

Copy link

See commit message. go test ./... green (darwin + linux/amd64). Adversarially reviewed in key.store session; -final includes the reviewer fixes.

See commit message. go test ./... green (darwin + linux/amd64). Adversarially reviewed in key.store session; -final includes the reviewer fixes.

founder added 1 commit 2026-08-17 00:06:11 +00:00

dispatcher: requeue-safe teardown, macOS TTL enforcement, cache volumes on macOS/Windows create ... 92cd5bb0d7

Three verified defects (KSE-0186 D8), measured on the forge on 2026-08-16:

1. Forced destroys were never checked against Forgejo. The phase-aware stop
   monitor (82e3da07) already stopped destroying fresh Linux instances (post-
   deploy census: 1 of 38 launches destroyed <15 s, that one a genuine 2 s
   container exit), but a monitor timeout or a cut macOS session still ended
   in `nsc destroy --force` with no idea whether the runner held a task.
   Options.RunnerBusy (wired in main.go via the Forgejo runners API) now
   defers such destroys while Forgejo shows the runner active, bounded by
   stopMonitorConfig.BusyGrace (30 m, poll 30 s). Every describe payload is
   logged at debug (FORGEJO_NSC_LOG_LEVEL=debug); the first per runner stays
   at info.

2. macOS instances outlived their TTL: runMacOSComputeSSHScript held
   session.Run() with no context deadline and Namespace does not end a macOS
   instance at --duration while a Compute SSH session is attached (60 m/90 m
   6x14 and 12x28 daemons were still online 3.5-7 h later, pinning 30 of the
   48 vCPU cap). The SSH session is now bound to a deadline at launch+TTL
   (the transport is closed, Run()/Wait() return errRunnerTTLExpired), the
   instance is destroyed and forgotten, and the macOS bootstrap gains the
   same TTL drain contract as Linux (FORGEJO_RUNNER_TTL_DEADLINE_EPOCH /
   FORGEJO_RUNNER_DRAIN_SECONDS, runner.shutdown_timeout, SIGTERM at
   deadline-drain) so the daemon stops taking jobs before the cut. The macOS
   launch path also stops mutating the caller's ExtraEnv map, which
   app.Service shares between the concurrent launches of one dispatch.

3. NSC_CACHE_VOLUMES were dropped on macOS ("macos nsc cache volumes
   disabled", 13x/day) and never passed on Windows create. Both paths now
   forward `--volume` (nsc 0.0.516..0.0.557 accept the flag for every
   machine type; the CLI parser wants exactly {kind}:{tag}:{mount}[:{size}],
   so drive-letter mounts such as `C:\cache\nscloud` are logged and skipped
   until the config uses a colon-free mount), retry the create once without
   volumes when Namespace rejects the volume request, and only then
   advertise the mounts to the runner: NSC_CACHE_PATH /
   KEYSTORE_NSC_CACHE_VOLUME_ROOT from the mount named "nscloud"
   (/Volumes/nscloud on macOS), KEYSTORE_BAZEL_STORAGE_CACHE from "bazel",
   via the wrapper exports on macOS and $env: in the Windows bootstrap.

Also: the Forgejo client decodes runner labels in both the Gitea object form
and the Forgejo 15 string form (the old struct failed to decode Forgejo's
response), treats status "active" as busy and "idle"/"active" as online, and
paginates the runners list (page/limit + X-Total-Count).

Tests: go test ./... on darwin/arm64 and on the forge (linux/amd64,
go1.26.3, CGO_ENABLED=0), including the linux-only dispatcher tests; the new
tests drive an in-process fake sshd + Compute API to prove the TTL cut and
to read the streamed wrapper env, and fake nsc binaries to check the create
args and the volume-less retry.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

founder merged commit 92cd5bb0d7 into main 2026-08-17 00:06:11 +00:00

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

key-store/forgejo-nsc!2

No description provided.