feat(cloud-workers): session placement, dispatch, and worker turn routing (#106332)

* feat(gateway-protocol): add session placement schema

Closed state discriminator for session execution placement (local/requested/provisioning/syncing/starting/active/draining/reconciling/reclaimed/failed), sessions.dispatch params, and worker-admission transcript/live cursor extensions. Swift protocol models mirror the schema.

* feat(state): add worker session placement table

worker_session_placements rows carry placement state, transition generation, worker ownership metadata, ACK cursors, and the turn claim columns used for atomic admission.

* feat(cloud-workers): add durable placement state machine store

SQLite-backed placement store split by concern: state table (placement-state), discriminated record types + shape invariants (placement-record), row codec + CAS transition values (placement-row-codec), atomic turn-claim admission/release/waiters (placement-turn-claims), and lifecycle CAS transitions (placement-store).

* feat(cloud-workers): sync workspaces and attach sessions to worker environments

Environment service session attachment + turn credentials, tunnel workspace commands over a dedicated SSH runner, and git/plain workspace sync into $HOME/.openclaw-worker/workspaces with an immutable manifest. Symlink escapes are rejected locally before transfer (macOS openrsync stat-fails them opaquely) and again by the remote manifest guard.

* feat(worker): run one-shot embedded turns from launch descriptors

Worker runtime executes a single embedded turn from a stdin launch descriptor and reports completed/failed/fenced on stdout for the gateway launcher. Terminal lifecycle live events are deferred past the final transcript flush; transcript projection helpers are shared via transcript-message instead of duplicated in the runtime.

* feat(cloud-workers): dispatch placements and route worker turns

Dispatch service drives local->requested->provisioning->syncing->starting->active with failure teardown (placement-dispatch-failure) and restart/runtime recovery incl. lost-worker reclaim (placement-dispatch-recovery). Worker turn launcher claims the placement turn atomically, builds a windowed launch descriptor (worker-turn-payload), runs the remote one-shot worker, and reconciles the committed transcript; agent runners route turns through the session placement admission provider.

* feat(gateway): expose session placement RPCs and startup reconciliation

sessions.dispatch RPC with lifecycle admission barriers, operator-facing placement projection on session listings, placement-aware session reset guard, and startup/interval reconciliation wiring for worker placements.
This commit is contained in:
Peter Steinberger
2026-07-13 08:03:53 -07:00
committed by GitHub
parent 06b27b9e1d
commit e98c7dfbcb
92 changed files with 13318 additions and 963 deletions
+4 -2
View File
@@ -46,8 +46,10 @@ Worker mode does not start channels, Gateway HTTP surfaces, or plugin auto-start
beyond the assigned session toolset. It uses a throwaway state directory and has
no standing provider or forge credentials.
Worker-to-worker session dispatch is not exposed in this mode. Agent dispatch
and placement remain gateway-owned milestone-3 surfaces.
Worker-to-worker session dispatch is not exposed in this mode. Placement and
dispatch remain gateway-owned: an operator can dispatch an existing local,
managed-worktree session through the Gateway, while a worker process cannot
dispatch itself or another worker.
The prepared assignment carries the transcript context, accepted base leaf,
commit sequence, and live-event cursor. On a tunnel reconnect, the process
+2 -1
View File
@@ -515,13 +515,14 @@ methods. Treat this as feature discovery, not a full enumeration of
</Accordion>
<Accordion title="Session control">
- `sessions.list` returns the current session index, including per-row `agentRuntime` metadata when an agent runtime backend is configured.
- `sessions.list` returns the current session index, including per-row `agentRuntime` metadata when an agent runtime backend is configured. When cloud-worker placement is enabled or durable recovery state exists, session rows also include a closed `placement` state (`local`, `requested`, `provisioning`, `syncing`, `starting`, `active`, `draining`, `reconciling`, `reclaimed`, or `failed`) plus state-specific environment, owner-epoch, workspace, bundle, ACK-cursor, or recovery fields.
- `sessions.subscribe` and `sessions.unsubscribe` toggle session change event subscriptions for the current WS client.
- `sessions.messages.subscribe` and `sessions.messages.unsubscribe` toggle transcript/message event subscriptions for one session. Pass `includeApprovals: true` to also receive sanitized `session.approval` lifecycle events for approvals whose persisted audience includes that exact session and whose reviewer binding authorizes the subscribing client. The subscribe response then includes a bounded pending `approvalReplay`; it is authoritative when `truncated` is false. The opt-in is per subscribe call, not sticky: re-subscribing to the same session without `includeApprovals: true` removes an existing approval subscription. In addition to normal session-read authority, this opt-in requires `operator.admin`, or `operator.approvals` on a paired device.
- `sessions.preview` returns bounded transcript previews for specific session keys.
- `sessions.describe` returns one gateway session row for an exact session key.
- `sessions.resolve` resolves or canonicalizes a session target.
- `sessions.create` creates a new session entry. `worktree: true` provisions a managed worktree; optional `worktreeBaseRef`/`worktreeName` select the base ref and branch name, and `execNode` (`operator.admin`) binds session exec to a node host. The created worktree is echoed in the result and persisted on the session row (`worktree: { id, branch, repoRoot }`). When the entry is created but its nested initial `chat.send` is rejected, the successful result includes `runStarted: false` and `runError`; clients can preserve the prompt and retry against the returned session key.
- `sessions.dispatch` (`operator.admin`) moves an existing local OpenClaw session with a session-owned managed worktree to a configured cloud-worker profile. Pass `{ key, profileId, agentId? }`. The method is absent when no worker profile is configured, closes local turn admission before draining active work, and returns only after placement reaches `active` worker ownership. Dispatch is one-way; worker-to-local pull-back is not part of this RPC.
- `sessions.groups.list`, `sessions.groups.put`, `sessions.groups.rename`, and `sessions.groups.delete` manage the gateway-owned custom session group catalog (names + display order). Membership stays on each session's `category` field; rename and delete update member sessions server-side.
- `sessions.send` sends a message into an existing session.
- `sessions.steer` is the interrupt-and-steer variant for an active session.