openclaw/docs/plan/runners.md

---
summary: Everything is a node — one placement model where paired machines and cloud boxes host sessions through the worker admission path; clients attach to sessions, never to runners.
title: Runners plan
read_when:
  - Designing or reviewing where sessions run (gateway, device, cloud)
  - Changing the Where picker, device pairing, node onboarding, or worker dispatch surfaces
  - Naming anything around sessions, devices, nodes, or placement
---

## Status

Proposal, revision 2. Supersedes revision 1 in place (2026-08-11, operator
decision). Implementation in progress; update this table in every PR that
advances a milestone.

| #   | Milestone                                                  | Status      | PRs              |
| --- | ---------------------------------------------------------- | ----------- | ---------------- |
| 0   | This plan (revision 2)                                     | landed      | #122454          |
| 1a  | Naming: session copy revert                                | landed      | #120667          |
| 1b  | Naming: devices consolidation                              | landed      | #120689          |
| 1c  | Cleanup: node-pairing → device-pairing merge               | landed      | #120726          |
| 2   | `openclaw resume` + web Continue in terminal               | in progress | #120664          |
| 3   | `openclaw connect` one-paste onboarding + `/j/` join route | in progress | #120768, #122499 |
| 4   | Picker: grouping, placement, liveness, enrichment          | in progress | #120804, #122531 |
| F   | Real-wire session boundary harness                         | landed      | #121212          |
| 5   | Public worker ingress path                                 | in progress | #122578          |
| 6   | Node worker provider (device runners)                      | not started | —                |
| 7   | Bundle push consent + runner updates                       | not started | —                |
| 8   | Stop-and-continue moves                                    | not started | —                |
| 9   | Deletions (ssh sandbox, openshell, exec-host clones, …)    | not started | —                |
| 10  | Cloud convergence (provisioners run `openclaw connect`)    | not started | —                |

Revision history: revision 1 (2026-08-08) established the session/runner
vocabulary, the naming rulings, and the milestone skeleton after a
code-evidence investigation and three adversarial reviews. Revision 2
(2026-08-11) follows a second round of deep code reads (worker admission,
tunnel, sync, node channel, scope model), an industry survey (GitHub/GitLab/
Buildkite/CircleCI runners, Tailscale, VS Code tunnels, Coder, Gitpod Flex,
Amp, Cursor/Claude/Codex cloud), a static teardown of Amp's runner transport,
and a fresh adversarial review of this revision. The operator decisions that
changed the plan:

- **Nodes host sessions.** Revision 1's "no turn loops on the node role"
  non-goal is overridden as a conclusion while its facts stand: the node
  _connection_ is still not an authority boundary, so session-hosting
  authority lives in the dispatch layer (worker admission, per-dispatch
  credentials, turn claims, owner epochs) — relocated, not removed.
- **`openclaw worker` becomes a node-supervised child.** One machine concept:
  a paired node can run everything a cloud worker runs today.
- **SSH is not the device transport.** The gateway never dials devices; the
  device always dials out. Revision 1's "ship sshd first" for device runners
  is deleted — it cannot reach a NAT'd machine and no surveyed product uses
  SSH as control transport. SSH remains only as the legacy cloud-lease
  transport until milestone 10 retires it.

## Problem

Unchanged from revision 1 in substance: OpenClaw has disconnected answers to
"where does work run." Nodes receive forwarded `exec host=node` calls only; a
user's always-on workstation is less capable as a session host than a
throwaway cloud lease. Cloud workers host full sessions with a durable
placement state machine, but only against ephemeral SSH-provisioned leases.
The ssh sandbox backend is a third remote-execution path. Placement is chosen
once from a flat list mixing ontologies, then becomes invisible; onboarding a
new machine takes flags, env vars, and two manual approvals.

The bar, stated as product: an admin clicks "Connect a machine…" in the web
picker, pastes one command on any machine, and seconds later that machine is
visible in the picker for the whole team and can host full agent sessions.

## Model and vocabulary

```
Session   gateway-owned: transcript, identity, placement, managed worktree.
          Clients (web, TUI, macOS app, channels) attach to sessions,
          never to runners. One noun, everywhere: session.
Node      a paired machine holding an outbound connection to the gateway
          (Ed25519 device identity). Protocol/internal vocabulary; user-facing
          copy says "device". EVERY remote machine is a node — personal
          workstations, servers, cloud leases. Phones are nodes that never
          advertise session hosting.
Runner    anything that can host a session's turn loop: the gateway itself,
          or a session-capable node. "Runner" is internal/docs vocabulary;
          UI copy says "Runs on …".
Worker    the per-turn child process (`openclaw worker`) that hosts a
          session's loop under worker admission. On cloud leases it is
          launched over SSH today; on nodes it is a supervised child of the
          node host. Same admission, same protocol, either way.
Isolation a property OF the runner (none | docker | podman), not a place.
Project   repo identity: normalized remote.origin.url, with the existing
          16-char repo fingerprint as the no-remote fallback. Derived,
          never registered.
Checkout  project × runner = { runnerId, path }.
Turn      one prompt-to-response work attempt inside a session.
```

Naming rulings (operator-decided, carried from revision 1): **session** is the
only product noun for a conversation; **devices** is the user-facing word for
paired hardware; new CLI ergonomics ship as **verbs** (`openclaw resume`,
`openclaw connect`); "runner" never appears in UI copy. Milestone 1c (nodes →
devices route/i18n consolidation) lands before any new placement copy ships.

## Architecture

### The two-connection shape

Every surveyed production system (GitHub Actions runners, GitLab, Buildkite,
CircleCI, Tailscale, VS Code tunnels, Coder, Gitpod, Amp) uses outbound-only
connections from the machine to the control plane, and the mature ones split
a persistent presence/control channel from per-job work channels. OpenClaw
already has both halves; this plan connects them:

1. **Node connection** (exists): the outbound gateway WebSocket. Carries
   identity, presence, capability manifest, and bounded command invocation
   (`node.invoke`). This is the control channel: registration, liveness, and
   the transport for workspace operations.
2. **Worker connection** (exists): the per-dispatch WebSocket speaking the
   closed worker protocol (heartbeat, transcript CAS commits, resumable live
   events, gateway-proxied inference, gateway-side session tools). Admission
   is store-backed and transport-free: per-dispatch 32-byte credential
   (10-minute TTL, hashed at rest), environment binding, owner epochs, exact
   bundle hash, per-RPC identity revalidation. On a node runner the worker
   child dials the gateway's public TLS endpoint directly — a connected node
   proves the outbound path exists.

What is deliberately NOT the transport: `node.invoke` as a byte pipe for the
worker connection. Measured constraints (16 KiB string chunks, an awaited RPC
round-trip per chunk, no idempotency dedupe, reconnect kills in-flight
invokes, one-session-per-nodeId eviction, 50 MB buffer hard-close) make it
unsuitable for hours-long streams. It stays what it is: a bounded command
channel.

### Worker ingress on the public endpoint (milestone 5)

Today the worker ingress is a dedicated loopback-only listener reached via
`ssh -R`; the main ingress rejects worker frames. For node runners the same
admission is exposed on a path-tagged upgrade route on the public TLS
endpoint (`connectionKind = "worker"` forced by route instead of listener).
The loopback listener stays for SSH-provisioned cloud workers until
milestone 10.

Hardening that ships with the exposure, not after it:

- Admission failures collapse into one opaque reason. The current
  `invalid-credential` vs `environment-mismatch` distinction is an
  environment-id enumeration oracle and must not be publicly observable.
- The worker path shares the gateway's preauth budgets and rate limits;
  a pre-credential connection gets the same cheap rejection as any other
  unauthenticated client.
- Credential strength is already sufficient (32 random bytes, constant-time
  hashed compare, 10-minute TTL, single environment binding).

### Node worker provider (milestone 6)

`WorkerLease` grows a union: `{ ssh: … } | { node: { deviceId } }`. The
admission/placement machinery (environment store, credential broker,
placement state machine, turn claims, transcript/live-event/inference
protocols) is reused unchanged — that is the hard-won part. What is net-new,
stated honestly (revision 1 undersold this):

- **Node tunnel handle.** A second `WorkerTunnelHandle` implementation:
  `runWorkspaceCommand` maps to a bounded node command (argv + stdin →
  SpawnResult; the remote-side sync/manifest/quiesce scripts already ship in
  the bundle and are transport-agnostic). `remoteSocketPath` is replaced by
  the descriptor carrying the gateway worker URL.
- **Durable launch.** In the SSH flow the launch exec stream _is_ the worker
  lifetime and its death destroys the environment. On a node, launch is a
  supervised node-host command: the node host spawns the worker child
  decoupled from the invoke lifetime, persists the one-line result, and the
  gateway re-collects it idempotently. A node WS blip must not kill a turn.
- **Credential delivery.** The launch descriptor (including the per-turn
  credential) travels over the authenticated node channel instead of SSH
  stdin. Same trust domain: the node host is the machine-side agent either
  way.
- **Workspace sync without rsync.** Manifest-driven delta blob transfer over
  authenticated HTTPS against the gateway (the manifest machinery already
  computes exact changed-blob lists; rsync was only the carrier), with
  git-mode base fetch from origin when the project has one. Existing bounds
  (inventory entries, manifest bytes, reconcile caps) carry over. Nodes with
  an advertised local checkout skip gateway push entirely (the Amp model:
  runner identity = host + workdir + repo).
- **Persistent-machine lifecycle.** `destroy` = logical lease release.
  Provider `inspect` is tri-state against pairing + presence: _present_,
  _dormant_ (paired but offline, within a dormancy ceiling — must NOT be
  driven to `orphaned` by the reconcile sweep), _gone_ (unpaired or ceiling
  elapsed → normal orphan/reap path). A device-environment reaper keyed on
  unpair/dormancy — not on provider teardown proof — cleans rows,
  credentials, and staged refs. Device-side GC of per-session workspace dirs
  and superseded bundles is a milestone exit gate, not an open question:
  persistent machines otherwise leak the user's own disk.
- **Placement `runner-offline`.** Heartbeat/presence loss marks the placement
  with a recorded, operator-visible reason; staged results are preserved by
  the existing fence machinery; the session offers "continue on gateway"
  (reclaim) or "wait for device". Never a silent non-outcome.
- **Dispatch target union.** `sessions.dispatch` accepts
  `{ profileId } | { deviceId }`; the device → environment mapping resolves
  server-side. Devices are not smuggled through synthesized
  `cloudWorkers.profiles` entries.
- **Concurrency slots.** A node declares a session-slot count (default small);
  the picker shows busy state; a dispatch that no live runner can satisfy
  fails visibly after a bounded wait instead of queuing forever.
- **Multi-gateway safety.** The worker install/workspace root on a node is
  namespaced by gateway identity so two gateways pairing one machine cannot
  corrupt each other's state.

Isolation on node runners: optional worker-in-docker/podman, same sandbox
axis as gateway-local sessions. Cloud leases keep full-permission-within-the-
box (the machine is the boundary).

### Trust model (operator-decided, v1)

Cloud workers run full-permission because the box is disposable and
credential-free. A paired personal machine is neither. The v1 resolution:

- **Only admins pair nodes** (already enforced: `role: node` device approval
  requires `operator.admin`; the join-code mint is admin-scoped). Pairing a
  node is the admin declaring it **shared team infrastructure** — a server,
  a build box, a dedicated workstation. That is the consent boundary for
  "everyone on the gateway may dispatch to it and session content lands on
  it."
- **Personal-device runners are out of scope for v1.** They arrive together
  with per-person node ownership (visibility + dispatch policy keyed on a
  recorded owner), not before. Approver identity is recorded at pairing time
  from day one as **provenance, never authorization** (additive nullable
  column), so the later policy has data to stand on.
- **Phones and low-trust devices never advertise session hosting.**
  Capability gating, not ontology: the picker never offers them.
- Non-interactive approval side doors (trusted-CIDR, SSH-verify,
  trusted-proxy browser auto-approve) remain scoped to their current
  presence-level grants and are reviewed for the hosted-gateway class; none
  may mint a session-capable node without an admin.
- Inference stays gateway-proxied; provider keys never reach nodes. If nodes
  ever fetch private repos from origin directly, the gateway mints
  short-lived scoped git credentials per dispatch; no standing PATs on nodes.

### Onboarding (milestone 3)

Copying the industry-standard split (short-lived enrollment secret →
long-lived device identity; GitLab deprecated reusable registration tokens to
get here, Tailscale's key/device revocation split is the documented model):

- Admin mints a **single-use, ~10-minute join code** (≥128-bit entropy) from
  the picker's "Connect a machine…" foot or `openclaw devices` CLI. The
  existing `device.pair.setupCode` RPC and `node` bootstrap profile are the
  substrate; the code pre-approves exactly the node role with zero operator
  scopes.
- The pasted one-liner is `npx openclaw connect <url-or-code>` (top-level
  verb; `openclaw node run` stays as the plumbing command). It accepts the
  full `oc-pair://` payload (offline form, carries gateway URL + bootstrap
  token + optional TLS pin for self-signed gateways) or an
  `https://<gateway-host>/j/<shortcode>` URL whose payload is fetched over
  TLS. `--service` installs the OS service instead of running foreground.
  A curl installer wrapper on the public website installs the CLI and execs
  the same verb; the public site never sees tokens.
- The gateway serves `/j/<shortcode>` (reserved prefix in Control UI routing,
  single-use burn, strict per-IP rate limiting).
- Revocation split, documented: revoking a join code never unpairs nodes;
  removing/banning a node is a first-class devices-page action that also
  fences in-flight placements. Node auto-cleanup after a long dead period
  mirrors runner-industry practice.

### Bundle and updates (milestone 7)

Exact-hash admission stays. The pinned, content-hashed bundle is pushed to
the node over the already-authenticated paired channel. Consent is split so
it cannot rot into approval fatigue or silent surprise:

- **Consent to be a runner**: one-time, per-device, at pairing/enablement.
- **Consent to run a build**: satisfied by the channel — bundles arrive only
  from the gateway this admin paired, and updates on dispatch are the normal
  managed-runner behavior (GitHub runners self-update the same way). The
  devices page shows the installed runner version; the gateway refuses
  dispatch to stale nodes with a doctor-style hint instead of failing
  silently.

### Projects read model (milestone 4 foundation)

OpenClaw already computes project identity twice without naming it: the
worktree service derives `originUrl` + a 16-char repo fingerprint
(`src/agents/worktrees/service.ts:199-205`), and the sessions catalog groups
Codex/Claude rows by project folder, folding `.claude/worktrees/<name>` into
its origin repo. This component promotes that to a first-class observed read
model alongside the registered projects already returned by `projects.list`,
following the same computed pattern as `environments.list`:

- **`projects.list.observedProjects` read model** (computed for
  write-capable callers, no new store): group known checkouts by repo fingerprint → `{ name, originUrl, checkouts:
[{runnerId, path}], lastUsedAt }`. Sources: session rows
  (`execCwd`/`execNode`) and the managed-worktree registry. The observed
  paths and sanitized origins are returned only to `operator.write` callers;
  read-only callers keep the registered project catalog and project-only
  recents. Device-advertised checkouts remain milestone 6 work.

### UI (milestone 4)

Revision 1's design rule stands: normal state is silent; only exceptions
speak. Additions:

- **Use the existing environment type discriminant** for picker grouping:
  local gateway, connected execution-capable nodes, worker environments, and
  the separate cloud profiles list. `sessionHost` is deferred to milestone 6,
  where device runners introduce the capability fact that needs it.
- **Where picker regrouped** (`ui/src/pages/new-session/place-picker.ts`):
  sections "This gateway" / "Devices" / "Cloud". Device rows intersect the
  environment catalog with connected, execution-capable nodes; cloud
  profiles remain their separate list. Folder and destination stay
  orthogonal.
- **Placement chip** on the session header: shows quiet current placement;
  active cloud placements reclaim through `sessions.reclaim` with "Bring
  home". Stop-and-continue moves arrive with milestone 8.
- **Remaining milestone work**: live presence and pairing subscriptions, the
  admin-gated "Connect a machine…" foot, busy and never-connected states,
  and additive `EnvironmentSummary` platform, session-host, trust, and runner
  version facts. `runner-offline` then shows a banner with the recorded reason
  and its recovery verbs.

### Cloud convergence (milestone 10)

A cloud provider's job collapses to: boot box, run
`openclaw connect <one-shot code> --ephemeral` in setup. Ephemeral enrollment
(industry: GitHub `--ephemeral`/JIT, Buildkite `--acquire-job`, Tailscale
ephemeral keys) auto-deregisters after the run and auto-purges the node
record when it goes offline. `destroy` = release lease. After soak, the SSH
reverse-tunnel stack, `PreparedWorkerSsh`, and the rsync transport are
deleted; cloud leases and paired machines become the same runner with
different lifecycles.

## What the adversarial reviews killed or reshaped

Carried forward from revision 1 (still true): no Places registry
(`environments.list` stays the read model, enriched additively); no dispatch
into a live checkout without exclusive ownership; `exec host=node` stays
untouched (different product, different policy domain); no sandbox-as-a-place
picker row; no fake mobility verbs; no live migration; no multi-gateway
federation; no phones as runners.

Revised or new in revision 2:

- Revision 1's "device runners are the existing worker stack with essentially
  no changes" was **overstated**: admission, placement, claims, stores, and
  the worker protocols are reused; transport, credential delivery, sync
  carrier, and launch durability are net-new. Scope milestone 6 accordingly.
- Revision 1's "ship sshd first" transport is **deleted** (unreachable target
  machines; industry-divergent).
- "Everyone dispatches" is **bounded by the trust model above** — shared
  infrastructure only, until per-person ownership ships.
- The `node.invoke` byte-pipe idea (this revision's own first draft) was
  killed by measured protocol constraints; the direct-dial worker connection
  replaced it.

## Prior art (what we copy, what we skip)

- **Amp** (verified by static CLI teardown + manual): outbound WSS only via
  actor framework; per-user control channel carries registration, heartbeat,
  presence, and dispatch intents in heartbeat responses; per-thread WS for
  live sessions; agent loop local on the runner in an existing checkout (no
  file sync; identity = host + workdir + repo URL); inference centralized
  server-side; per-workdir PID claim prevents double-serving. We copy the
  two-channel shape, dispatch-over-control-channel, and checkout
  advertisement; we keep inference gateway-proxied (their centralization is
  a billing choice, not architecture); we scope enrollment tighter than
  their single long-lived API key.
- **GitHub Actions runners**: registration token → device keypair; JIT/
  ephemeral single-job runners; self-update with a staleness ceiling and
  dispatch refusal; blunt security docs about persistent runners running
  untrusted code. All copied in spirit above.
- **Tailscale**: auth-key vs node-key split and the revocation split warning.
  Copied, documented.
- **VS Code tunnels**: the gold-standard enrollment UX (run one command,
  browser confirms); device-code-style confirmation is a candidate
  alternative to pasted codes later. Their 10-tunnel account cap validates
  bounded per-gateway node counts.
- **Coder / Gitpod Flex**: control/data plane split with customer-side
  execution and orchestration-only control plane — the closest analog to
  "inference on gateway, execution on node," validating it as a coherent
  residency story. Gitpod's ~30s registration renewal is the liveness-lease
  reference if presence needs tightening.
- **Cursor / Claude Code / Codex cloud**: managed-VM-only execution with
  git-based handoff; Claude Code's proxy-minted scoped git credentials
  inform the scoped-git-token rule above; teleport-style continuation
  validates attach-only sessions (which OpenClaw gets for free).

## Milestones

Independently mergeable PR series; 3–5 can interleave after 1c.

1. **1c naming cleanup**: finish nodes → devices in route ids, i18n keys,
   labels; `node-pairing.ts` facade merge. Before any new placement copy.
2. **Continuation ergonomics** (in progress): `openclaw resume`, web
   "Continue in terminal".
3. **`openclaw connect`**: verb + `oc-pair://` decoder + TLS pin in payload +
   `/j/<shortcode>` join route (reserved prefix, single-use, rate-limited) +
   shortcode mint + curl wrapper on the public site. Exit: a fresh machine
   pairs against a remote gateway with one pasted command and one admin
   click, no manual approval steps.
4. **Picker** (in progress): regrouped sections, quiet placement + reclaim,
   and the observed projects read model land first; live presence subscription,
   the admin-gated "Connect a machine…" foot, additive `EnvironmentSummary`
   enrichment, and never-connected vs lost states complete the milestone.
5. **Public worker ingress**: path-tagged worker upgrade on the main TLS
   endpoint; opaque admission failure; shared preauth budgets. Exit: a worker
   process on any internet host with a valid dispatch credential completes
   admission; invalid attempts are cheap and unenumerable.
6. **Node worker provider**: lease union, dispatch target union, node tunnel
   handle, durable supervised launch, HTTPS delta sync + origin fetch,
   tri-state inspect + reaper + GC, concurrency slots, `runner-offline`
   placement semantics, gateway-namespaced install root, approver-provenance
   column. Fault-injection tests gate exit: device sleep mid-turn, node WS
   blip mid-turn (turn survives), gateway restart with offline device,
   credential expiry, slot saturation, dispatch-with-no-live-runner timeout.
7. **Bundle push + updates**: consent split, push over paired channel,
   version surfacing, stale-node dispatch refusal.
8. **Stop-and-continue moves**: drain + reclaim + re-dispatch to another
   runner, reusing the migration barrier.
9. **Deletions**: ssh sandbox backend + remote-fs bridge (~2.35k LOC),
   openshell overlap (~3.4k LOC, verify usage first), exec-host structural
   clones (~3k of ~5k LOC), one-shot `agent.cli.claude.run` node path
   (superseded by full session hosting), node/device pairing merge remainder.
   Each gated on its replacement, each its own PR with proof.
10. **Cloud convergence**: `--ephemeral` enrollment, provisioners run
    `openclaw connect`, then delete the SSH tunnel/rsync transport stack.

Net production LOC across the plan is targeted negative: milestones 3–5 are
small additions, 6–7 are mostly a provider + one transport implementation
against reused machinery, and 9–10 delete more than everything before them
adds.

## Open questions

- Dormancy ceiling default (how long a sleeping device stays `dormant`
  before its environments reap) — proposal: 14 days, config-free, revisit
  with usage.
- Slot count default for node runners — proposal: 2 for interactive-class
  devices, higher for server-class; needs a capability signal or a connect
  flag.
- Device-code-style browser confirmation (VS Code model) as an alternative
  to pasted codes — later, once `/j/` exists.
- Repo-owned environment setup (devcontainer.json) for worker profiles —
  unchanged from revision 1: adopt the spec if/when it lands, separate plan.
- Forge integration (repo lists, clone-anywhere, PR status) — explicitly out,
  follow-up once the derived project model has usage.