Commit Graph

8 Commits

Author SHA1 Message Date
Peter Steinberger 243f51d314 refactor(macos): name computer-use executors for their scope (#124586)
The computer.act v1 wire contract is gone, but the naming that survived it
still described a version split instead of the real one: screen-coordinate
execution versus window/element-scoped execution. Both are live rungs of the
same ladder.

- Extract the screen-coordinate half of the 1334-line ComputerActionService
  into ComputerScreenActionExecutor (dispatch, typing, scroll, coordinate
  mapping, button-hold watchdog, raw CoreGraphics primitives). Moved code is
  unchanged apart from threading the queue authority check as a parameter
  instead of reaching back into the queue.
- ComputerActionService keeps its name and becomes the coordinator that owns
  the execution queue, the permission probe, and the shared error vocabulary.
- Rename ComputerActionServiceV2 to ComputerWindowActionExecutor, isV2Request
  to isWindowScopedRequest, isComputerActV2Only to isWindowScopedOnly, and
  ComputerActionError.invalidV2Request to .invalidRequest. The emitted
  COMPUTER_INVALID_REQUEST: prefix is unchanged.
- cua-computer: v2-actions.ts becomes window-actions.ts, handleV2Act becomes
  handleWindowAct, and the stale v1Params local in handleDesktopAct becomes
  desktopParams.
- Note at the computer.act idempotency key that its v1 prefix versions the key
  composition, not the wire contract.

Behavior-neutral: no logic edits, no new branches, no changed error strings.
2026-08-16 06:17:31 -07:00
Peter Steinberger 62deb87ee4 fix(computer-use): converge provider ref lifecycles on the contract (#124374)
* fix(computer-use): converge provider ref lifecycles

* test(computer-use): expect stale lifecycle code

* fix(computer-use): key native window refs on stable window identity

Peekaboo matched an existing window ref on the whole WindowMutationIdentity,
which embeds capturedBounds and isMinimized. Any move, resize, or minimize
therefore minted a new ref for the same live window, never evicted the old
one, and let the old ref keep resolving to a stale ServiceWindowInfo whose
identity and bounds were then used as the per-action expectations, so the
operator saw a downstream mismatch instead of COMPUTER_STALE_OBSERVATION.

Window refs now key on stable identity only (WindowServer id plus the owner
process generation that guards pid reuse) and refresh the stored target in
place on every discovery, so one live window keeps one ref for the whole
lifecycle generation and later checks compare against current data. The
one-instantiation generic reference store is folded back into the service as
plain state plus small concrete helpers.

The shared case table gains window_moved, which both providers must satisfy,
and the Swift side now drives the real ComputerActionServiceV2 with real
Peekaboo values instead of a toy store with an equality matcher. Its
in-flight generation case runs a real perform() whose lifecycle is revoked
mid-action rather than throwing the error it asserted.

* test(cua-computer): drive the real in-flight generation-rotation path
2026-08-15 21:08:41 -07:00
Peter Steinberger a4f61d135a feat(macos): Computer Use provider readiness checklist (#124093)
* feat(macos): show computer provider readiness

* refactor(macos): drop the status string superseded by the readiness checklist

* chore(macos): refresh i18n baseline after removing the superseded status string
2026-08-15 01:08:54 -07:00
Peter Steinberger 4a6edc0d27 feat(macos): fulfill computer.act v2 via Peekaboo (#123801) 2026-08-14 14:12:25 -07:00
Peter Steinberger 5c6bccff4a feat(macos): enable Computer Control by default and diagnose stale Accessibility grants (#115280)
* feat(macos): enable and diagnose computer control

* docs(macos): explain computer control permissions

* chore: refresh macOS i18n and docs indexes

* test: update macOS computer control i18n fixture
2026-07-28 13:00:32 -04:00
Peter Steinberger 3bec587d74 fix(computer): stop typing after cancellation (#103528) 2026-07-10 08:37:37 +01:00
Peter Steinberger 461772868d fix(computer): prevent stale, replayed, and post-cancel desktop actions (#103422)
* fix(ai): preserve streamed tool-call identity

* fix(computer): bind actions to current tool authority

* fix(macos): serialize computer control lifecycle

* docs(computer): document hardened control contract

* chore: follow release-owned changelog policy

* test(agents): cover node list cancellation
2026-07-10 06:47:56 +01:00
Peter Steinberger f8dbd34ab8 feat(nodes): add computer use (computer.act) on macOS nodes via Peekaboo (#102776)
* feat(nodes): add computer use (computer.act) on macOS nodes via Peekaboo

* fix(nodes): release held computer.act input on node disconnect/stop/disable

* fix(nodes): release held computer.act input armed after a lifecycle release

* fix(nodes): scope computer.act lifecycle catch-up release to the arming action

* chore(nodes): register computer tool display metadata, sync native i18n, doc-comment constants

* fix(nodes): preserve caught-error cause in computer tool resolver; regen docs map
2026-07-09 15:28:09 +01:00