Files
openclaw/apps/android
Peter Steinberger df707a9670 feat: view this machine in the Desktop panel (#122545)
* feat(gateway): add gateway-host desktop source behind desktop.host lab

Introduce the host as a first-class desktop source so operators can view
the machine OpenClaw runs on, not just cloud-worker environments:

- protocol: desktop.observe / desktop.launch with a discriminated
  DesktopSource union (host | environment) plus an additive auth hint;
  EnvironmentSummary gains a top-level desktop flag
- config: desktop.host { enabled, port?, passwordFile? }, Labs-gated
- rfb-probe: pure RFB version/security-type parser used to detect an
  already-running loopback VNC server and classify its auth
- host-source: attaches to 127.0.0.1:<port>, refuses unauthenticated
  (None) and unsupported (VeNCrypt) servers, and refuses ARD with the
  supported alternative until the macOS milestone
- host-guidance: per-OS enablement text so no path dead-ends
- doctor + status report host desktop availability and auth type only

worker.desktop.observe/launch stay as delegating aliases with identical
behavior. Also drops the now-unused WorkerDesktopTunnels type export.

Live-verified against macOS Screen Sharing: probe reads RFB 003.889,
returns security types [30,33,36,35], classifies ard-account.

* test(gateway): probe RFB handshakes through the socket boundary

The probe's banner and security-offer parsers were exported solely so
unit tests could call them, which the dead-export gate rejects and which
tests internals rather than behavior. Keep them module-local and drive
the probe through a scripted loopback server instead.

The boundary tests also cover what pure-function vectors could not:
handshakes split across packets, legacy RFB 3.3 single-word security,
server-rejected handshakes, early hangups, and connect timeouts.

* feat(ui): let the Desktop panel view this machine, not just cloud workers

The Desktop panel was gated on a cloud-worker session placement, so an
operator running OpenClaw locally had no way to see the machine hosting
their main session even with a VNC server running on it.

Availability now follows the advertised desktop.observe method plus
operator.admin instead of session placement, and the picker lists every
environment whose summary reports a desktop, with the gateway row shown
as "This machine". Sources are passed to the generic desktop.observe /
desktop.launch RPCs; the app launcher stays worker-only. When a host
attach needs a password the gateway did not supply, the panel prompts and
keeps the value in memory for that connection only.

Adds the hostDesktop Labs toggle for desktop.host.enabled.

* fix(scripts): keep the env-var ratchet usable in shallow checkouts

The env-var budget check resolved its base ref, then hard-failed when
`git merge-base` found no shared ancestor. Shallow clones and grafted
agent checkouts resolve origin/main but truncate the history behind it,
so an advisory growth ratchet took down the whole check:changed gate
with "Could not resolve env-var count merge base for: origin/main".

Only the growth comparison needs a baseline, and the script already has
a no-baseline path. Treat git's exit 1 with empty output (no shared
ancestor) as that case and say so on stderr; a genuine failure still
exits 128 and still throws, and the absolute count-vs-budget check runs
either way.

* test(ui): measure the inline-code chip against its line box

The inline-code spacing test compared the chip's height to a prose text
rect, so it silently measured the monospace font's default line spacing.
That is ~17px on macOS and several px shorter on Linux, so the assertion
passed on CI and failed locally at 4.5 against a 3.75 bound -- after the
bound had already been widened once to chase browser font metrics.

Compare the chip to the paragraph's CSS line box instead, which is what
"the chip must not disrupt the line" actually means and is platform
independent. The horizontal gap stays as-is: it is em-derived padding
plus border, and it is the assertion that catches detached punctuation.

Verified both directions on macOS: the file is fully green, and
restoring the pre-fix 0.15em/0.35em padding still fails the gap
assertion at 5.41.

* feat(gateway): view macOS Screen Sharing from the Desktop panel

Modern macOS only offers ARD account authentication for Screen Sharing,
so the host desktop source refused every Mac. The Gateway now performs
the ARD handshake itself against the loopback server and hands the
browser a plain RFB 003.008 no-auth handshake, so the operator's macOS
account password authenticates the desktop without ever reaching the
browser, the observe result, a URL, or a log.

- rfb-preauth: ARD (type 30) Diffie-Hellman with MD5-derived AES-128-ECB
  credentials, and VncAuth (type 2) bit-reversed DES, both under a single
  10s negotiation deadline; Apple's RFB 003.889 maps to 3.8
- observe-bridge: runs pre-auth before splicing and starts the view-only
  filter at clientInit, since the browser handshake is consumed here;
  worker tokens keep the original version start phase
- host-source: attaches ARD, requiring per-observation credentials that
  live only in the one-shot observer token and are dropped after use
- doctor: offers an explicitly confirmed sudo launchctl repair when
  Screen Sharing is off, and prints the System Settings path otherwise

Live-verified against this Mac's Screen Sharing: the DH exchange and
credential framing are accepted and the server returns SecurityResult.
The VncAuth DES vector is confirmed against OpenSSL independently.
2026-08-12 06:58:30 -07:00
..
2026-07-01 18:09:57 -05:00

OpenClaw Android App

OpenClaw Android is the officially released Google Play app. It connects to an OpenClaw Gateway as a companion node for chat, voice, approvals, screen, and device-aware automation.

Current App Surface

  • New 4-step onboarding flow
  • Connect tab with Setup Code + Manual modes
  • Encrypted persistence for gateway setup/auth state
  • Chat UI restyled
  • Settings UI restyled and de-duplicated (gateway controls moved to Connect)
  • QR code scanning in onboarding
  • Performance improvements
  • Streaming support in chat UI
  • Dedicated per-device Android chat session created/adopted on connect without resetting history
  • Request camera/location and other permissions in onboarding/settings flow
  • Push notifications for gateway/chat status updates
  • Security hardening (biometric lock, token handling, safer defaults)
  • Authenticated background presence beacons
  • Voice tab full functionality
  • Foreground on-device Voice Wake with Gateway-synced wake words
  • Screen tab full functionality
  • Skill Workshop settings can filter proposals, inspect proposal content, and apply/reject/quarantine drafts through Gateway RPCs
  • Skills settings can search installed skills, enable or disable them, and install Gateway-verified ClawHub releases
  • Per-app language selection for translated resources follows Android system settings and persistence
  • Cron job settings support details, run history, run now, edits, enable/disable, and deletion with admin-scoped Gateway access
  • Wear OS companion proxies sessions, transcripts, replies, aborts, and realtime Talk through the paired phone without storing Gateway credentials on the watch

Open in Android Studio

  • Run pnpm install from the repository root so native Canvas resources can be generated.
  • Open the folder apps/android.

Wear OS companion

The wear app is a paired-phone companion with the same application ID and signing identity as the phone app. The watch discovers the phone through Wear OS Data Layer, then uses the phone's existing authenticated operator session. It never receives or stores Gateway tokens, passwords, TLS pins, or device-signing identity.

The watch supports agent and session selection, bounded text-only transcript history, streaming reply state, text and voice replies, abort, realtime Talk within the selected session, paired-phone Gateway controls, local reply notifications, theme and automatic-speech settings, and a launch Tile. Realtime Talk streams watch microphone and playback audio over a temporary Wear OS Data Layer channel; it still uses the phone's authenticated Gateway session and closes when the selected phone or Gateway connection changes. A missing Data Layer event sequence or changed phone-process epoch triggers a fresh history request instead of applying uncertain deltas. Agent and Gateway controls are capability-negotiated so an older paired phone remains usable during staggered updates.

cd apps/android
./gradlew :wear:testDebugUnitTest :wear:assembleDebug :wear:lintDebug :wear:ktlintCheck

Build / Run

cd apps/android
./gradlew :app:assemblePlayDebug
./gradlew :app:installPlayDebug
./gradlew :app:testPlayDebugUnitTest
cd ../..
pnpm android:release:archive

Third-party debug flavor:

cd apps/android
./gradlew :app:assembleThirdPartyDebug
./gradlew :app:installThirdPartyDebug
./gradlew :app:testThirdPartyDebugUnitTest

Repository-backed debug Gradle invocations, including pnpm android:run and pnpm android:screenshots, stamp the full checkout commit and capture one UTC build timestamp shared by every debug variant in that invocation. Release tasks still require explicit openclawBuildCommit and openclawBuildTimestamp properties so signed artifacts remain reproducible.

Android release archives use the pinned version in apps/android/version.json. Update it with:

pnpm android:version
pnpm android:version:check
pnpm android:version:pin -- --from-gateway
pnpm android:version:pin -- --version 2026.6.5 --version-code 2026060501

Release-owner signing sync:

pnpm android:release:signing:plan
MATCH_PASSWORD=<signing repo password> pnpm android:release:signing:sync:pull
MATCH_PASSWORD=<signing repo password> pnpm android:release:signing:check

The signing sync pulls encrypted Android upload-key assets from the shared apps-signing repo and materializes decrypted files under apps/android/build/release-signing/. Standalone release APK verification also requires that key's public certificate SHA-256 fingerprint to match Config/ReleaseSigning.json.

Generate phone and Wear OS Google Play screenshots:

pnpm android:screenshots

The screenshot script captures both form factors with retained OpenClaw_Screenshots_API36 (Pixel 2) and OpenClaw_Wear_Screenshots_API34 (Wear OS Large Round) AVDs. It creates a missing AVD, boots it headlessly, waits for Android to finish booting, disables animations, captures the screenshots, then shuts down the emulator it started. Install the API 36 Google APIs and API 34 Wear OS system images in the local Android SDK. Use --form-factor phone|wear with --avd or --device to explicitly capture one form factor from another emulator.

pnpm android:release:archive builds signed release artifacts into apps/android/build/release-artifacts/ and writes .sha256 checksum files:

  • Play build: openclaw-<version>-play-release.aab
  • Wear build: openclaw-<version>-wear-release.aab
  • Third-party build: openclaw-<version>-third-party-release.apk

pnpm android:bundle:release is an alias for the same Fastlane archive lane.

Regular final and correction OpenClaw releases publish the signed third-party APK as OpenClaw-Android.apk with a checksum manifest and GitHub Actions provenance. .github/workflows/android-release.yml is the only automated GitHub Release upload path; OpenClaw Release Publish dispatches it while the canonical release is still a draft and blocks publication until the uploaded asset contract verifies.

The protected android-release environment supplies MATCH_PASSWORD; the repository's read-only GitHub App token checks out encrypted material from openclaw/apps-signing. The workflow builds the exact release tag, refuses to replace different existing bytes, and re-downloads the APK for checksum, certificate, and provenance verification.

pnpm android:release:archive is for local archive validation only. It is not a fallback upload path after pnpm android:release:upload fails.

Agent-driven Google Play uploads must use pnpm android:release:upload as the only release path. If that command fails, stop and fix the failing screenshot, metadata, signing, validation, archive, or upload step before trying again. Do not upload archived artifacts through direct Fastlane lanes, Gradle artifacts, Google Play API commands, or Play Console mutation commands.

The release lane uploads the phone and Wear bundles in one atomic Google Play edit. It publishes the phone bundle to GOOGLE_PLAY_TRACK and maps the Wear bundle to the corresponding form-factor track (wear:<track>), so the default internal channel publishes to internal and wear:internal.

See apps/android/VERSIONING.md and apps/android/fastlane/SETUP.md for the release workflow.

Prefer pnpm android:release:archive, which stamps and validates the full Git commit and one UTC build timestamp before signing. Flavor-specific direct Gradle release tasks must pass the same metadata explicitly:

cd apps/android
commit="$(git -C ../.. rev-parse HEAD)"
built_at="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
./gradlew -PopenclawBuildCommit="$commit" -PopenclawBuildTimestamp="$built_at" :app:bundlePlayRelease
./gradlew -PopenclawBuildCommit="$commit" -PopenclawBuildTimestamp="$built_at" :wear:bundleRelease
./gradlew -PopenclawBuildCommit="$commit" -PopenclawBuildTimestamp="$built_at" :app:bundleThirdPartyRelease

Kotlin Lint + Format

pnpm android:lint
pnpm android:format

Android framework/resource lint (separate pass):

pnpm android:lint:android

Direct Gradle tasks:

cd apps/android
./gradlew :app:ktlintCheck :benchmark:ktlintCheck :wear:ktlintCheck :wear-shared:ktlintCheck
./gradlew :app:ktlintFormat :benchmark:ktlintFormat :wear:ktlintFormat :wear-shared:ktlintFormat
./gradlew :app:lintPlayDebug :app:lintThirdPartyDebug :wear:lintDebug :wear-shared:lintDebug

gradlew auto-detects the Android SDK at ~/Library/Android/sdk (macOS default) if ANDROID_SDK_ROOT / ANDROID_HOME are unset.

Macrobenchmark (Startup + Frame Timing)

cd apps/android
./gradlew :benchmark:connectedDebugAndroidTest

Reports are written under:

  • apps/android/benchmark/build/reports/androidTests/connected/

Perf CLI (low-noise)

Deterministic startup measurement + hotspot extraction with compact CLI output:

cd apps/android
./scripts/perf-startup-benchmark.sh
./scripts/perf-startup-hotspots.sh

Benchmark script behavior:

  • Runs only StartupMacrobenchmark#coldStartup (10 iterations).
  • Prints median/min/max/COV in one line.
  • Writes timestamped snapshot JSON to apps/android/benchmark/results/.
  • Auto-compares with previous local snapshot (or pass explicit baseline: --baseline <old-benchmarkData.json>).

Hotspot script behavior:

  • Ensures debug app installed, captures startup simpleperf data for .MainActivity.
  • Prints top DSOs, top symbols, and key app-path clues (Compose/MainActivity/WebView).
  • Writes raw perf.data path for deeper follow-up if needed.

Run on a Real Android Phone (USB)

  1. On phone, enable Developer options + USB debugging.
  2. Connect by USB and accept the debugging trust prompt on phone.
  3. Verify ADB can see the device:
adb devices -l
  1. Install + launch debug build:
pnpm android:install
pnpm android:run

If adb devices -l shows unauthorized, re-plug and accept the trust prompt again.

USB-only gateway testing (no LAN dependency)

Use adb reverse so Android localhost:18789 tunnels to your laptop localhost:18789.

Terminal A (gateway):

pnpm openclaw gateway --port 18789 --verbose

Terminal B (USB tunnel):

adb reverse tcp:18789 tcp:18789

Then in app Connect → Manual:

  • Host: 127.0.0.1
  • Port: 18789
  • TLS: off

Hot Reload / Fast Iteration

This app is native Kotlin + Jetpack Compose.

  • For Compose UI edits: use Android Studio Live Edit on a debug build (works on physical devices; project minSdk=31 already meets API requirement).
  • For many non-structural code/resource changes: use Android Studio Apply Changes.
  • For structural/native/manifest/Gradle changes: do full reinstall (pnpm android:run).
  • Canvas web content already supports live reload when loaded from Gateway __openclaw__/canvas/ (see docs/platforms/android.md).

Connect / Pair

  1. Start the gateway (on your main machine):
pnpm openclaw gateway --port 18789 --verbose
  1. In the Android app:
  • Open the Connect tab.
  • Use Setup Code or Manual mode to connect.
  1. Approve pairing (on the gateway machine):
openclaw devices list
openclaw devices approve <requestId>

More details: docs/platforms/android.md.

Permissions

  • Discovery:
    • Android 13+ (API 33+): NEARBY_WIFI_DEVICES
    • Android 12 and below: ACCESS_FINE_LOCATION (required for NSD scanning)
  • Location:
    • Both flavors: ACCESS_FINE_LOCATION / ACCESS_COARSE_LOCATION for foreground checks.
    • Third-party flavor only: ACCESS_BACKGROUND_LOCATION plus FOREGROUND_SERVICE_LOCATION for user-enabled Always checks.
  • Foreground service notification (Android 13+): POST_NOTIFICATIONS
  • Camera:
    • CAMERA for camera.snap and camera.clip
    • RECORD_AUDIO for camera.clip when includeAudio=true

Google Play Restricted Permissions

As of March 19, 2026, these manifest permissions are the main Google Play policy risk for this app:

  • READ_SMS
  • SEND_SMS
  • READ_CALL_LOG

Why these matter:

  • Google Play treats SMS and Call Log access as highly restricted. In most cases, Play only allows them for the default SMS app, default Phone app, default Assistant, or a narrow policy exception.
  • Review usually involves a Permissions Declaration Form, policy justification, and demo video evidence in Play Console.
  • The Play build removes these behind the play flavor.
  • Photo library access is also removed from the Play build. Use third-party builds for photos.latest.

Current OpenClaw Android implication:

  • APK / sideload build can keep SMS, Call Log, and recent-photo features.
  • Google Play build excludes SMS send/search, Call Log search, and recent-photo access unless the product is intentionally positioned and approved under the relevant policy exception.
  • The repo now ships this split as Android product flavors:
    • play: removes READ_SMS, SEND_SMS, READ_CALL_LOG, READ_MEDIA_IMAGES, READ_MEDIA_VISUAL_USER_SELECTED, READ_EXTERNAL_STORAGE, and background location; hides SMS, Call Log, Photos, and Always location surfaces.
    • Installed-app listing is user controlled. device.apps is advertised only after the user enables Settings > Phone Capabilities > Installed Apps. The command defaults to launcher-visible apps and does not require QUERY_ALL_PACKAGES.
    • thirdParty: keeps the full permission set and the existing SMS / Call Log / Photos functionality, and offers explicit Always location opt-in through Android settings.

Policy links:

Other Play-restricted surfaces to watch if added later:

  • ACCESS_BACKGROUND_LOCATION
  • MANAGE_EXTERNAL_STORAGE
  • QUERY_ALL_PACKAGES
  • REQUEST_INSTALL_PACKAGES
  • AccessibilityService

Reference links:

Integration Capability Test (Preconditioned)

This suite assumes setup is already done manually. It does not install/run/pair automatically.

Pre-req checklist:

  1. Gateway is running and reachable from the Android app.
  2. Android app is connected to that gateway and openclaw nodes status shows it as paired + connected.
  3. App stays unlocked and in foreground for the whole run.
  4. Open the app Screen tab and keep it active during the run (canvas/A2UI commands require the canvas WebView attached there).
  5. Grant runtime permissions for capabilities you expect to pass (camera/mic/location/notification listener/location, etc.).
  6. No interactive system dialogs should be pending before test start.
  7. Canvas host is enabled and reachable from the device for remote Canvas checks (do not run gateway with OPENCLAW_SKIP_CANVAS_HOST=1; startup logs should include canvas host mounted at .../__openclaw__/).
  8. Local operator test client pairing is approved. If first run fails with pairing required, preview the latest pending request, approve the printed request ID, then rerun:
  9. For A2UI checks, keep the app on Screen tab; the node uses its bundled app-owned A2UI page for message application.
openclaw devices list
openclaw devices approve --latest   # preview only; copy the requestId from output
openclaw devices approve <requestId>

Run:

pnpm android:test:integration

Optional overrides:

  • OPENCLAW_ANDROID_GATEWAY_URL=ws://... (default: from your local OpenClaw config)
  • OPENCLAW_ANDROID_GATEWAY_TOKEN=...
  • OPENCLAW_ANDROID_GATEWAY_PASSWORD=...
  • OPENCLAW_ANDROID_NODE_ID=... or OPENCLAW_ANDROID_NODE_NAME=...

What it does:

  • Reads node.describe command list from the selected Android node.
  • Invokes advertised non-interactive commands.
  • Skips screen.record and talk.ptt.* in this suite because they require interactive capture. Use apps/android/scripts/voice-e2e.sh for microphone and voice-path proof.
  • Asserts command contracts (success or expected deterministic error for safe-invalid calls like sms.send and notifications.actions).

Common failure quick-fixes:

  • pairing required before tests start:
    • list pending requests (openclaw devices list), then approve with the exact ID (openclaw devices approve <requestId>) and rerun.
  • A2UI host not reachable / A2UI_HOST_UNAVAILABLE:
    • keep the app foregrounded on the Screen tab and rerun. A2UI commands use the bundled app-owned A2UI page; the Gateway Canvas host is still needed for remote Canvas checks, but not for A2UI message application.
  • NODE_BACKGROUND_UNAVAILABLE: canvas unavailable:
    • app is not effectively ready for canvas commands; keep app foregrounded and Screen tab active.

Contributions

Maintainer: @obviyus. For issues/questions/contributions, please open an issue or reach out on Discord.