The computer.act v1 wire contract is gone, but the naming that survived it
still described a version split instead of the real one: screen-coordinate
execution versus window/element-scoped execution. Both are live rungs of the
same ladder.
- Extract the screen-coordinate half of the 1334-line ComputerActionService
into ComputerScreenActionExecutor (dispatch, typing, scroll, coordinate
mapping, button-hold watchdog, raw CoreGraphics primitives). Moved code is
unchanged apart from threading the queue authority check as a parameter
instead of reaching back into the queue.
- ComputerActionService keeps its name and becomes the coordinator that owns
the execution queue, the permission probe, and the shared error vocabulary.
- Rename ComputerActionServiceV2 to ComputerWindowActionExecutor, isV2Request
to isWindowScopedRequest, isComputerActV2Only to isWindowScopedOnly, and
ComputerActionError.invalidV2Request to .invalidRequest. The emitted
COMPUTER_INVALID_REQUEST: prefix is unchanged.
- cua-computer: v2-actions.ts becomes window-actions.ts, handleV2Act becomes
handleWindowAct, and the stale v1Params local in handleDesktopAct becomes
desktopParams.
- Note at the computer.act idempotency key that its v1 prefix versions the key
composition, not the wire contract.
Behavior-neutral: no logic edits, no new branches, no changed error strings.
* fix(computer-use): converge provider ref lifecycles
* test(computer-use): expect stale lifecycle code
* fix(computer-use): key native window refs on stable window identity
Peekaboo matched an existing window ref on the whole WindowMutationIdentity,
which embeds capturedBounds and isMinimized. Any move, resize, or minimize
therefore minted a new ref for the same live window, never evicted the old
one, and let the old ref keep resolving to a stale ServiceWindowInfo whose
identity and bounds were then used as the per-action expectations, so the
operator saw a downstream mismatch instead of COMPUTER_STALE_OBSERVATION.
Window refs now key on stable identity only (WindowServer id plus the owner
process generation that guards pid reuse) and refresh the stored target in
place on every discovery, so one live window keeps one ref for the whole
lifecycle generation and later checks compare against current data. The
one-instantiation generic reference store is folded back into the service as
plain state plus small concrete helpers.
The shared case table gains window_moved, which both providers must satisfy,
and the Swift side now drives the real ComputerActionServiceV2 with real
Peekaboo values instead of a toy store with an equality matcher. Its
in-flight generation case runs a real perform() whose lifecycle is revoked
mid-action rather than throwing the error it asserted.
* test(cua-computer): drive the real in-flight generation-rotation path
* fix(cua-computer): prove Linux X11 live vertical
* test(computer-use): authenticate isolated Linux rig
* fix(gateway): refresh computer use after node approval
* refactor(cua-computer): resolve the plugin manifest by static import
* fix(gateway): break plugin runtime import cycle
* fix(computer-use): bind live rig to committed helpers
* feat(cua-computer): add browser action family
* refactor(cua-computer): split browser action files
* refactor(cua-computer): move the shared act-params type to the leaf module
* fix(cua-computer): carry the contract params import with the moved type
* feat(computer-use): computer.act v2 action contract with capability-filtered tool
* fix(computer-use): break contract import cycle, bound model-visible elements, regen swift protocol
* test(computer-use): satisfy curly rule in schema-cap helpers
* fix(computer-use): satisfy type-aware lint on contract and tool
* test(gateway-protocol): keep connect-params suite under the line cap
* test(cua-computer): freeze provider parity fixtures
* test(cua-computer): keep parity fixture type aliases file-local until a consumer exists
* test(cua-computer): split parity fixtures under the line cap and fix strict test types
* feat(cua-computer): add experimental Windows/Linux computer-use fulfiller
Bundled plugin that fulfills the capability-based computer.act + screen.snapshot
node contract on Windows and Linux by supervising a pinned cua-driver 0.10.x
daemon over MCP stdio. macOS keeps the Peekaboo fulfiller; this plugin is
disabled by default and never available on darwin.
Grounded in cua-driver 0.10.0 source (tool schemas, refusal codes, coordinate
spaces, session/daemon lifecycle). Notable safety and correctness properties:
- Deny-by-default env allowlist so OpenClaw secrets (provider/channel tokens,
CUA_API_KEY) never reach the separately installed daemon; telemetry and
update checks forced off.
- Version-gated handshake (exact-minor pin + capability/schema version),
time-bounded so a corrected driver recovers without a node restart.
- Robust daemon supervision: full readiness-budget polling, startup-race
tolerance, signal-death and spawn-error recovery, shared-daemon lifecycle
(never killed on dispose).
- Frame authorization preserved within upstream limits (generation + full live
geometry; capture refused when screen and screenshot geometry diverge).
- Action mapping refuses inputs cua-driver cannot faithfully deliver:
layout-shifted keys, modifier-held drag/scroll, Linux modifier clicks,
hold_key/mouse down-up, non-positive scroll; drag duration clamped.
* fix(cua-computer): satisfy lint, test-types, dead-code, and docs-map gates
Peter Steinberger