Tight computer-use loops (click, screenshot, click, screenshot) carried a
full ~1200px image every step even when the screen never changed. Only
age-based pruning existed, so those duplicate bytes sat in model context
for three completed turns each.
When a new capture's sanitized image identity matches the frame identity
already tracked for the same node and screen in the current context epoch,
the tool now returns its normal text metadata plus an explicit "screen
unchanged since previous frame (frameId ...)" note and omits the image
block. The result is append-only; no prior transcript message is rewritten.
Coordinate-frame ownership is preserved: a dedup hit deliberately leaves
contextEpoch.frameToolCallId/frameImageIdentity pointing at the original
image-bearing tool result, so invalidateComputerFrameIfMissing keeps
guarding the pixels the model actually holds. Once history pruning drops
that image, the epoch bumps, coordinates expire as before, and the next
screenshot delivers full pixels again. Only the node-issued displayFrameId
advances on a hit. Dedup is therefore gated on context-epoch tracking:
without it, an unchanged-screen note could reference an already-pruned
image, so those screenshots always deliver pixels.
Screenshots now stay bound until replacement pixels are verified, so
capture failures, act rejections, and failed follow-up screenshots demote
the frame explicitly instead of relying on an unconditional pre-capture
reset.
Provider observation images (window/browser observations) and the browser
plugin's screenshot tool are intentionally excluded: neither has
context-presence tracking.
* fix(gateway): stop a plugin dangerous flag from revoking desktop computer.act
`resolveNodeCommandAllowlistInternal` subtracted every plugin-declared
dangerous command from the composed allowlist, including core's own
`PLATFORM_DEFAULTS` entries and including the pairing allowlist, where core's
dangerous defaults are exempted. `computer.act` is both a desktop platform
default (grant = node-local enablement + pairing approval) and a command that
`registerComputerUseProvider` marks dangerous. Since `cua-computer` became
enabled-by-default on darwin, every macOS-hosted Gateway stripped `computer.act`
from both allowlists, so `normalizeDeclaredNodeCommands` dropped it from the
node's declaration with no pairing upgrade, no prompt, and no record. The
`computer` capability survived because caps were never allowlist-filtered, and
the `computerUse` descriptor was then dropped for the missing command.
Scope the plugin-dangerous subtraction to commands outside the platform-default
base: the flag still keeps a plugin's own surface behind an explicit allow and
still forces a registered invoke policy, but it no longer revokes a command core
declares itself.
Also make cap-without-command unrepresentable. `retainFulfilledNodeCapabilities`
drops a capability when policy withheld commands from its family and admitted
none, and the reconciler records the withheld commands so a refused declaration
is never silent.
* test(gateway): type the computer-use fixture against its contract
* fix(computer-use): unblock the macOS live-rig proof flow
The rig ran its operator CLI and its proof runner from one state dir, so both
shared one device identity. A paired operator device is pinned to the scopes of
its first connect, and `nodes list` connects first for `node.pair.list`
(operator.pairing); the proof runner then needs operator.write, which is a scope
upgrade the gateway never approves silently and which no rig client can approve
for itself. The proof runner is a GATEWAY_CLIENT/BACKEND client, so on a
loopback auth-none gateway it is admitted unpaired with the scopes it asks for:
giving the CLI its own `cli-state` identity is enough, and `agent-state` now
never accumulates a pairing row.
`nodes list` also read `node.list` through the plain CLI client while
`nodes status`/`describe` used the diagnostics ladder. On any gateway where the
CLI must pair, the unfiltered list silently dropped connected/commands/
computerUse and `--connected` failed outright, so the documented rig gate could
not confirm the node. Both call sites now use `callNodeDiagnosticsGatewayCli`.
Docs drop the `devices approve <requestId>` instruction, which was circular:
that invocation is its own new device identity.
* test(cli): share the runtime-log formatter across nodes CLI e2e files
The extracted diagnostics-auth file stringified captured log arguments directly, which the type-aware core lint stripe rejects (no-base-to-string). Move the existing formatter into the shared node test helpers instead of duplicating it.
* fix(cua-computer): prove Linux X11 live vertical
* test(computer-use): authenticate isolated Linux rig
* fix(gateway): refresh computer use after node approval
* refactor(cua-computer): resolve the plugin manifest by static import
* fix(gateway): break plugin runtime import cycle
* fix(computer-use): bind live rig to committed helpers
* feat(cua-computer): add browser action family
* refactor(cua-computer): split browser action files
* refactor(cua-computer): move the shared act-params type to the leaf module
* fix(cua-computer): carry the contract params import with the moved type
* feat(cua-computer): add experimental Windows/Linux computer-use fulfiller
Bundled plugin that fulfills the capability-based computer.act + screen.snapshot
node contract on Windows and Linux by supervising a pinned cua-driver 0.10.x
daemon over MCP stdio. macOS keeps the Peekaboo fulfiller; this plugin is
disabled by default and never available on darwin.
Grounded in cua-driver 0.10.0 source (tool schemas, refusal codes, coordinate
spaces, session/daemon lifecycle). Notable safety and correctness properties:
- Deny-by-default env allowlist so OpenClaw secrets (provider/channel tokens,
CUA_API_KEY) never reach the separately installed daemon; telemetry and
update checks forced off.
- Version-gated handshake (exact-minor pin + capability/schema version),
time-bounded so a corrected driver recovers without a node restart.
- Robust daemon supervision: full readiness-budget polling, startup-race
tolerance, signal-death and spawn-error recovery, shared-daemon lifecycle
(never killed on dispose).
- Frame authorization preserved within upstream limits (generation + full live
geometry; capture refused when screen and screenshot geometry diverge).
- Action mapping refuses inputs cua-driver cannot faithfully deliver:
layout-shifted keys, modifier-held drag/scroll, Linux modifier clicks,
hold_key/mouse down-up, non-positive scroll; drag duration clamped.
* fix(cua-computer): satisfy lint, test-types, dead-code, and docs-map gates
Peter Steinberger