* feat(agents): mobile_ui agent tool for Android UI control (PR 3/3)
Add a dedicated model-facing tool that drives another Android app through the
PR2 mobile.ui.observe/act node commands, completing the vertical slice
(agent -> tool -> node.invoke -> AccessibilityService).
- Mirrors the desktop computer tool's safety model: owner-only + HTTP-denied
(dangerous-tools), raw node.invoke of mobile.ui.* redirected to this tool so
the generic nodes tool cannot bypass it, run/tool-call idempotency, and the
phone-arm workflow (mobile.ui.* must be explicitly armed).
- One call is observe or one act; every act automatically re-observes for
postcondition verification and preserves the landed outcome if re-observe fails.
- Fail-closed confirmation: all state-changing acts (activate, set_text, tap,
swipe) require confirmed=true after the model reviews the proposed effect;
observe, scroll, wait, and navigation global_actions do not. The keyword list
only enriches the confirmation message and is never the sole gate (a11y labels
are localized/iconographic/coordinate-blind).
- Node selection resolves an explicit id against the full device set first
(case-insensitively) and rejects an ineligible or ambiguous match, so an
explicit selection can never be silently redirected to the wrong phone.
- All observed UI text is treated as untrusted; the tool description forbids
following instructions found in app UI.
Additive; no protocol bump. mobile-ui-tool + registration + policy tests pass;
core tsgo and lint verified on Testbox. On-device drive is exercised via the
PR1/PR2 emulator proof; this PR is the agent-side tool + tests.
Follow-up: computer-tool.ts has the same latent node-resolution ordering bug
(explicit id searched only among eligible nodes); tracked separately.
* fix(agents,phone-control): migrate mobile_ui arming hints/tests to gateway.nodes.commands.allow
Match main's node-command arming rename (gateway.nodes.allowCommands ->
gateway.nodes.commands.allow / commands.deny): update the mobile_ui tool arm-hint
matchers to the gateway's current rejection strings and fix the phone-control /
tool tests to the current config shape. Production write path was already correct.
* fix(agents): register mobile_ui in owner-only denylist; trim phone-control under max-lines
- tool-resolution.exclude.test.ts: mobile_ui joins the owner-only core
tools, so add it to the expected plugin/inherited denylists.
- phone-control: derive ArmGroup from a single ARM_GROUPS const and
collapse parseGroup's ||-chain, dropping index.ts back under 700 lines.
* fix(agents): centralize mobile_ui global-action names; regenerate tool-display snapshot
- Extract GLOBAL_ACTION_NAMES const so the schema, action type, and
validator share one source. This also removes the bare `name: "back"`
type-annotation literal that scripts/tool-display.ts's name-regex was
misreading as a phantom runtime tool.
- Regenerate apps/.../tool-display.json to add the mobile_ui display entry
(was missing from the Swift snapshot).
* feat(sessions): first-class archived-session handling
Archived sessions are now exempt from every automatic maintenance path
(age prune, entry cap, model-run prune, disk budget, cleanup repairs);
only explicit sessions.delete removes them. sessions.list gains an
additive tri-state archived filter (true | false | "all"). The sidebar
gets a persisted Active/Archived/All status filter with inline dimmed
archived rows, replacing the nav-away View-archived button; the Sessions
page toggle becomes the same tri-state with dimmed+badged rows in All
mode; the chat composer's archived notice becomes a full-width banner
with an Unarchive action.
* chore(protocol): regenerate Swift gateway models for tri-state archived filter
* feat(android,gateway): expose mobile.ui.observe/act as node commands (PR 2/3)
Wire the PR1 AccessibilityService executor over the existing node.invoke
transport. Additive: no gateway protocol version bump.
- New commands mobile.ui.observe / mobile.ui.act (capability mobileUI),
generated into the protocol constants.
- thirdParty MobileUiHandler owns one mutex-serialized AccessibilityActionExecutor
and bridges JSON <-> the PR1 snapshot/action model; play flavor ships a
permanently-unavailable no-op stub (Play APK stays accessibility-free).
- Commands advertised only when accessibilityControlEnabled and the service is
connected; NodeRuntime refreshes the advertised surface on connect/disconnect.
- Classified dangerous in node-command-policy (declarable-but-armed, mirroring
computer.act): the transport cannot invoke them until gateway.nodes.allowCommands
explicitly arms them. Screen reads are treated as dangerous too, so observe is
gated as well as act.
Both flavors assemble; thirdParty/play unit tests, ktlint, android lint, and the
gateway node-command-policy tests pass; protocol generator is idempotent; Play
APK verified to contain zero accessibility classes. Emulator: no regression to
PR1 dev-screen observe.
* fix(gateway): migrate mobile.ui arming test/comment to gateway.nodes.commands.allow
Main renamed the node-command arming config from gateway.nodes.allowCommands to
gateway.nodes.commands.allow; update the mobile.ui policy test config shape and
the comment to the current contract. Production resolution already reads the new
path.
* chore(android): update native i18n baseline for PR2 line-number shifts
* feat(apple): add rare working claw stances
Add zen, drummer, and peekaboo animations to the shared iOS and macOS working indicator, with deterministic stance and pose coverage.
* chore(i18n): refresh native source anchors
* feat(android): AccessibilityService UI executor (thirdParty, PR 1/3)
Add a thirdParty-flavor-only AccessibilityService that observes the active
app's UI as a bounded semantic snapshot and performs typed actions, exercised
via a local developer screen. No gateway/agent wiring yet (PR 2 adds
node.invoke commands, PR 3 the agent tool + policy gates).
- Play APK stays accessibility-free: service, config, executor, and UI live
entirely under src/thirdParty; SensitiveFeatureConfig.accessibilityControlEnabled
gates it. Verified: 0 accessibility refs in every merged Play manifest and
0 accessibility classes in the Play dex.
- Semantic-first executor: observe() returns a bounded snapshot (node/depth/text
caps, deterministic order, password + sensitive-field redaction, stable action
vocabulary, generation-scoped refs); act() performs one typed action with a
closed ActionOutcomeCode result set.
- Safety model: coordinate gestures gated by package-match (fail-closed) + a UI
epoch advanced on window/content/scroll/text mutation events; node actions
gated by package-match + per-node refresh(); global actions ungated. Capture
runs off the main thread. Dev UI honestly disables cross-app node controls
(only reachable while the target is foreground; validated via the remote path).
Built and emulator-tested on API 36: live connection status, immediate observe,
a 103-node cross-app Settings capture, and global Home. Both flavors assemble;
thirdParty unit tests, ktlint, and android lint pass.
* feat(android): gate accessibility control behind an off-by-default opt-in
Make landing the accessibility feature a no-op for existing thirdParty users:
the service is invisible and inert until the user explicitly opts in.
- The AccessibilityService and its dev activity are declared
android:enabled="false", so a fresh install exposes NO new accessibility
service in system settings and nothing can bind it (verified on device:
absent from the installed-services list and unbindable; shell cannot enable
it either — only the app can).
- New thirdParty-only "Control other apps" toggle (persisted in the existing
openclaw.node prefs, default OFF). Turning it on enables both components via
PackageManager.setComponentEnabledSetting and deep-links to Accessibility
settings so the user can grant it; turning it off disables them again
(DONT_KILL_APP). A disclosure describes what enabling does.
- Play flavor is a no-op (FlavorPhoneCapabilitiesSettings = Unit); no
accessibility component/controller/toggle references reach the Play APK.
- No new permissions or dependencies. The compile-time flavor gate and the
gateway dangerous-command arming remain as additional layers.
* chore(android): update native i18n baseline for accessibility control strings
* refactor(gateway): declare spawn lineage explicitly at sessions.create
Spawn depth is now a declared fact, never inferred from parentSessionKey.
sessions.create accepts an optional spawnDepth (requires parentSessionKey) that
spawn-owned creations pass; every other fresh session persists spawnDepth 0,
making operator chats and UI forks spawn-capable roots. The visible spawn tool
declares callerDepth + 1, and subagent depth recovery no longer walks
parentSessionKey, which is UI threading only.
Accepted tradeoff (documented inline): pre-upgrade visible children whose only
lineage was parentSessionKey resolve as roots; the transient population may
spawn one extra generation, still capped by maxChildrenPerAgent.
* chore(protocol): regenerate Swift gateway models for sessions.create spawnDepth
* feat(gateway,ui): ask-the-observer card input over sessions.observer.ask
* refactor(ui): single home for observer run-identity helper after restack
* test(ui): drop duplicated observer hud test after restack
* test(ui): give the observer ask flow its own colocated suite
* refactor(gateway): leaf observer contract and ask module split for ci gates
* refactor(gateway): drop observer contract re-export shims
* fix(ui): allow direct sessions in non-Git folders
* test: align New Session fixtures with place picker
* fix(ui): preserve worktree intent on Git probe failures
* fix(types): keep worktree status alias internal
* feat(ui): session observer HUD, subtitle integration, and settings
* test(ui): observer demo fixtures for the mock control-ui harness
* fix(ui): satisfy lint and deadcode gates for observer surfaces
* test(ui): adopt renamed pull-request summary api after rebase
* fix(ui): clean rebase artifacts in observer test files
Branch menu (list/switch) for macOS+iOS shared chat via sessions.branches.*;
rewind/fork/switch gated on run activity and pending outbox work through a
durable session-mutation lease; outbox rows carry branch-epoch ownership with
local-only flush checks, atomic confirm/park transitions, attempt-versioned
delivery callbacks, fresh retry identity for possibly-accepted rows, and
epoch-guarded branch evidence; 21-locale native translations for new strings.
* fix(anthropic): accept "cli" entrypoint in Claude session catalog discovery
Claude Code v2.x writes entrypoint: "cli" in its JSONL session files.
Previously only "sdk-cli" was recognized, causing all v2.x sessions to
be silently skipped.
Extract CLI_ENTRYPOINTS set and isCliEntrypoint() helper, then apply
the same fix to both TypeScript catalog and macOS paired-node native
catalog. Add matching regression coverage on both platforms.
Closes#105164
* fix(anthropic): refresh Claude CLI session discovery
Apply the reviewed cli/sdk-cli allowlist, mirrored negative coverage, and fallback-scope documentation to current main.
Co-authored-by: 黄攀0668000858 <huang.pan@xydigit.com>
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* fix(macos): resolve OPENCLAW_STATE_DIR in openclaw-mac CLI config resolution
Add shared resolveOpenClawConfigURL() to GatewayConfig.swift with
precedence: OPENCLAW_CONFIG_PATH > OPENCLAW_STATE_DIR > default home.
Switch ConfigureRemoteCommand and loadGatewayConfig to use it.
Add focused tests for all three precedence paths.
* fix(macos): honor state directory in openclaw-mac
Use one normalized config resolver for connect, wizard, and configure-remote, with explicit config path precedence over the selected state directory.
Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* feat(chat): add native working progress state
* feat(chat): port working claw indicator to Apple
* fix(apple): isolate generated locale catalog
* style(apple): use doc comments on ChatWorkingPhrase API declaration
* feat(gateway): session observer digests over the utility model
* fix(gateway): split session-observer modules and satisfy ci gates
* fix(gateway): observer reads session entries without materializing agent state
* feat(mobile): session dashboard screens on iOS and Android via authenticated Control UI webview
* fix(android): keep configured Control UI base path in session dashboard URL
* docs(android): note system-trust boundary of the shared Control UI webview
* fix(android): origin-only document-start rule for Control UI auth script
* chore(i18n): refresh native inventory on rebased head
* fix(ios): swiftlint closure form in session dashboard toolbar
* fix(i18n): tolerate workflow-owned pending native rows in PR alignment checks
* fix(android): KTX toUri per lint and refresh native inventory
* fix(android): ktlint import order incl. main-inherited fleet test, refresh inventory
Peter Steinberger