Commit Graph

23 Commits

Author SHA1 Message Date
Peter Steinberger 22ab8e3c2c fix(gateway): stop a plugin dangerous flag from revoking desktop computer.act (#124863)
* fix(gateway): stop a plugin dangerous flag from revoking desktop computer.act

`resolveNodeCommandAllowlistInternal` subtracted every plugin-declared
dangerous command from the composed allowlist, including core's own
`PLATFORM_DEFAULTS` entries and including the pairing allowlist, where core's
dangerous defaults are exempted. `computer.act` is both a desktop platform
default (grant = node-local enablement + pairing approval) and a command that
`registerComputerUseProvider` marks dangerous. Since `cua-computer` became
enabled-by-default on darwin, every macOS-hosted Gateway stripped `computer.act`
from both allowlists, so `normalizeDeclaredNodeCommands` dropped it from the
node's declaration with no pairing upgrade, no prompt, and no record. The
`computer` capability survived because caps were never allowlist-filtered, and
the `computerUse` descriptor was then dropped for the missing command.

Scope the plugin-dangerous subtraction to commands outside the platform-default
base: the flag still keeps a plugin's own surface behind an explicit allow and
still forces a registered invoke policy, but it no longer revokes a command core
declares itself.

Also make cap-without-command unrepresentable. `retainFulfilledNodeCapabilities`
drops a capability when policy withheld commands from its family and admitted
none, and the reconciler records the withheld commands so a refused declaration
is never silent.

* test(gateway): type the computer-use fixture against its contract
2026-08-16 16:27:45 -07:00
Peter Steinberger b07c6b2b8b fix(computer-use): macOS live-proof rig deadlocks on operator device approval (#124536)
* fix(computer-use): unblock the macOS live-rig proof flow

The rig ran its operator CLI and its proof runner from one state dir, so both
shared one device identity. A paired operator device is pinned to the scopes of
its first connect, and `nodes list` connects first for `node.pair.list`
(operator.pairing); the proof runner then needs operator.write, which is a scope
upgrade the gateway never approves silently and which no rig client can approve
for itself. The proof runner is a GATEWAY_CLIENT/BACKEND client, so on a
loopback auth-none gateway it is admitted unpaired with the scopes it asks for:
giving the CLI its own `cli-state` identity is enough, and `agent-state` now
never accumulates a pairing row.

`nodes list` also read `node.list` through the plain CLI client while
`nodes status`/`describe` used the diagnostics ladder. On any gateway where the
CLI must pair, the unfiltered list silently dropped connected/commands/
computerUse and `--connected` failed outright, so the documented rig gate could
not confirm the node. Both call sites now use `callNodeDiagnosticsGatewayCli`.

Docs drop the `devices approve <requestId>` instruction, which was circular:
that invocation is its own new device identity.

* test(cli): share the runtime-log formatter across nodes CLI e2e files

The extracted diagnostics-auth file stringified captured log arguments directly, which the type-aware core lint stripe rejects (no-base-to-string). Move the existing formatter into the shared node test helpers instead of duplicating it.
2026-08-16 04:25:03 -07:00
Peter Steinberger 289a9cabc8 docs(computer-use): state the authorization chokepoint explicitly (#124355) 2026-08-15 19:30:29 -07:00
Peter Steinberger abf0ef6513 fix(computer-use): repair artifact verification and post-approval descriptor found by the Linux gate (#124128)
* fix(cua-computer): prove Linux X11 live vertical

* test(computer-use): authenticate isolated Linux rig

* fix(gateway): refresh computer use after node approval

* refactor(cua-computer): resolve the plugin manifest by static import

* fix(gateway): break plugin runtime import cycle

* fix(computer-use): bind live rig to committed helpers
2026-08-15 03:53:23 -07:00
Peter Steinberger af99d1ac7b test(computer-use): security closure across the v2 action surface (#124112) 2026-08-15 01:54:06 -07:00
Peter Steinberger 079bb34196 feat(cua-computer): recording family with host-owned resource handles (#124035)
* feat(cua-computer): add recording resource handles

* test(agents): split computer tool coverage
2026-08-15 00:14:13 -07:00
Peter Steinberger a8f1a0d345 test(computer-use): live end-to-end gate for both macOS providers (#123991)
* fix(computer-use): preserve semantic click observations

* fix(cua-computer): isolate desktop and window sessions

* fix(cua-computer): route cursor reads through desktop scope

* test(computer-use): add isolated macOS live proof rig

* style(cua-computer): format driver sessions

* fix(cua-computer): forward lazy desktop tools

* fix(computer-use): narrow live proof artifact path

* test(computer-use): split schema contract coverage

* refactor(cua-computer): unify session start state

* build(computer-use): register live proof entrypoint

* style(computer-use): satisfy full lint contract

* fix(computer-use): harden live proof authority

* fix(cua-computer): keep window authority immutable

* fix(computer-use): close partial sessions safely
2026-08-14 23:37:53 -07:00
Peter Steinberger a32a7ae9af feat(cua-computer): verify the managed driver artifact on Windows and Linux node hosts (#123986) 2026-08-14 21:08:13 -07:00
Peter Steinberger aa7a2fbe0d feat(cua-computer): browser action family over v2 (#123960)
* feat(cua-computer): add browser action family

* refactor(cua-computer): split browser action files

* refactor(cua-computer): move the shared act-params type to the leaf module

* fix(cua-computer): carry the contract params import with the moved type
2026-08-14 20:59:46 -07:00
Peter Steinberger 19ace6830b feat(macos): add embedded CUA computer provider (#123635)
* feat(macos): embed CUA computer provider

* fix(macos): clarify embedded CUA trust posture

* fix(macos): contain embedded CUA daemon lifecycle

* fix(macos): reap orphaned CUA daemons

* fix(macos): record the spawned CUA daemon pid so reaping can terminate orphans

* chore(macos): refresh native i18n baseline for the computer control provider picker

* style(macos): satisfy swiftlint on the embedded CUA host and connect params

* refactor(gateway): move optional connect params to GatewayConnectOptions
2026-08-14 10:24:08 -07:00
Dallin Romney 1d45930162 chore(cua-computer): upgrade CUA Driver to 0.19.3 (#122191)
* chore(cua-computer): upgrade CUA Driver to 0.19.3

* test(cua-computer): verify CUA Driver enum contract
2026-08-13 15:19:35 +08:00
Dallin Romney 1574723997 fix(cua-computer): load the ESM driver on node hosts (#120169)
* fix(cua-computer): load ESM driver asynchronously

* fix(cua-computer): register computer invoke policy

* test(gateway): cover CUA command allowlist

---------

Co-authored-by: Francesco Bonacci <f@trycua.com>
2026-08-07 15:15:48 +08:00
Dallin Romney e9b25aa21d refactor(cua-computer): replace daemon MCP with CUA Driver SDK (#117205)
* chore(cua-computer): freeze CUA Driver 0.14.1 fixtures

* refactor(cua-computer): use CUA Driver SDK directly

* fix(cua-computer): repair CUA CI checks

* style(cua-computer): format driver cleanup

* fix(cua-computer): migrate retired driver path

* fix(cua-computer): start direct driver sessions
2026-08-02 20:03:24 +08:00
Peter Steinberger 5c6bccff4a feat(macos): enable Computer Control by default and diagnose stale Accessibility grants (#115280)
* feat(macos): enable and diagnose computer control

* docs(macos): explain computer control permissions

* chore: refresh macOS i18n and docs indexes

* test: update macOS computer control i18n fixture
2026-07-28 13:00:32 -04:00
Peter Steinberger 938749180e refactor(nodes)!: retire arming leases for durable capability enablement (#114892)
* refactor(nodes): retire phone control leases

* fix(doctor): clean retired phone control state

* docs(nodes): document durable capability enablement

* docs(changelog): defer node enablement note

* fix(doctor): harden retired phone control cleanup

* fix(doctor): fail closed on unreadable lease state

* fix(doctor): validate retired lease journals

* fix(doctor): preserve deny-shadowed consent

* docs: refresh node authorization map

* fix(doctor): keep retirement helpers private
2026-07-27 23:51:30 -04:00
Peter Steinberger c1a313676b docs(computer-use): add cua-computer troubleshooting/error-code reference and fix stale Codex macOS claim (#112502) 2026-07-21 21:07:01 -07:00
Peter Steinberger edecdbd05e refactor(config): config-surface reduction tranche 3 — product consolidations (review request) (#111527)
* refactor(config): consolidate media model lists

* refactor(config): unify memory configuration

* refactor(config): consolidate TTS ownership

* refactor(config): move typing policy to agents

* refactor(config): retire product-level config surfaces

* refactor(config): share scoped tool policy type

* chore(config): refresh generated baselines

* fix(config): honor agent typing overrides

* fix(config): migrate sibling config consumers

* refactor(infra): keep base64url decoder private

* fix(config): strip invalid legacy TTS values

* chore(config): refresh rebased baseline hash

* fix(doctor): route legacy messages.tts.realtime voice to talk during tts move

* refactor(config): polish final layout names

* refactor(config): freeze retired tuning defaults

* feat(config): add fast mode default symmetry

* refactor(config): key agent entries by id

* docs(config): update final layout reference

* test(config): cover final layout migrations

* chore(config): refresh final layout baselines

* fix(config): align final layout runtime readers

* fix(config): align remaining readers

* fix(config): stabilize final layout migrations

* fix(config): finalize config projection proof

* fix(config): address final layout review

* docs(release): preserve historical config names

* fix(config): complete keyed agent migration

* fix(config): close final migration gaps

* fix(config): finish full-branch review

* fix(config): complete runtime secret detection

* fix(config): close final review findings

* fix(config): finish canonical docs and heartbeat migration

* fix(config): integrate latest main after rebase

* refactor(env): isolate test-only controls

* refactor(env): isolate build and development controls

* refactor(env): collapse process identity indirection

* refactor(env): remove duplicate config and temp aliases

* docs(env): define the operator-facing allowlist

* ci(env): ratchet production variable count

* fix(env): remove stale provider helper import

* fix(env): make ratchet sorting explicit

* test(env): keep test seam in dead-code audit

* test(env): cover ratchet growth and boundary; document surface budgets

* docs(config): document tier-eval consolidations

* docs(config): clarify speech preference ownership

* test(memory): align retired tuning fixtures

* refactor(memory): freeze engine heuristics

* refactor(config): apply tier-eval tranche

* refactor(tts): move persona shaping to providers

* refactor(compaction): move prompt policy to providers

* test(config): align hookified prompt fixtures

* chore(deadcode): classify test-only exports

* chore(github): remove unused spawn helper

* chore(deadcode): classify queue diagnostics

* chore(deadcode): remove unused lane snapshot export

* chore(plugin-sdk): ratchet consolidated surface

* fix(config): integrate latest main after rebase
2026-07-21 20:28:43 -07:00
Peter Steinberger f695be341c feat(cua-computer): add experimental Windows/Linux computer-use fulfiller (#112267)
* feat(cua-computer): add experimental Windows/Linux computer-use fulfiller

Bundled plugin that fulfills the capability-based computer.act + screen.snapshot
node contract on Windows and Linux by supervising a pinned cua-driver 0.10.x
daemon over MCP stdio. macOS keeps the Peekaboo fulfiller; this plugin is
disabled by default and never available on darwin.

Grounded in cua-driver 0.10.0 source (tool schemas, refusal codes, coordinate
spaces, session/daemon lifecycle). Notable safety and correctness properties:
- Deny-by-default env allowlist so OpenClaw secrets (provider/channel tokens,
  CUA_API_KEY) never reach the separately installed daemon; telemetry and
  update checks forced off.
- Version-gated handshake (exact-minor pin + capability/schema version),
  time-bounded so a corrected driver recovers without a node restart.
- Robust daemon supervision: full readiness-budget polling, startup-race
  tolerance, signal-death and spawn-error recovery, shared-daemon lifecycle
  (never killed on dispose).
- Frame authorization preserved within upstream limits (generation + full live
  geometry; capture refused when screen and screenshot geometry diverge).
- Action mapping refuses inputs cua-driver cannot faithfully deliver:
  layout-shifted keys, modifier-held drag/scroll, Linux modifier clicks,
  hold_key/mouse down-up, non-positive scroll; drag duration clamped.

* fix(cua-computer): satisfy lint, test-types, dead-code, and docs-map gates
2026-07-21 05:58:04 -07:00
Peter Steinberger 40f28f8e81 feat(nodes): make computer.act eligibility capability-based for desktop nodes (#112107) 2026-07-21 00:04:04 -07:00
joshavant 5b20013399 docs(security): clarify computer act authorization 2026-07-10 13:06:53 -05:00
Peter Steinberger 3bec587d74 fix(computer): stop typing after cancellation (#103528) 2026-07-10 08:37:37 +01:00
Peter Steinberger 461772868d fix(computer): prevent stale, replayed, and post-cancel desktop actions (#103422)
* fix(ai): preserve streamed tool-call identity

* fix(computer): bind actions to current tool authority

* fix(macos): serialize computer control lifecycle

* docs(computer): document hardened control contract

* chore: follow release-owned changelog policy

* test(agents): cover node list cancellation
2026-07-10 06:47:56 +01:00
Peter Steinberger f8dbd34ab8 feat(nodes): add computer use (computer.act) on macOS nodes via Peekaboo (#102776)
* feat(nodes): add computer use (computer.act) on macOS nodes via Peekaboo

* fix(nodes): release held computer.act input on node disconnect/stop/disable

* fix(nodes): release held computer.act input armed after a lifecycle release

* fix(nodes): scope computer.act lifecycle catch-up release to the arming action

* chore(nodes): register computer tool display metadata, sync native i18n, doc-comment constants

* fix(nodes): preserve caught-error cause in computer tool resolver; regen docs map
2026-07-09 15:28:09 +01:00