* feat(gateway): add gateway-host desktop source behind desktop.host lab
Introduce the host as a first-class desktop source so operators can view
the machine OpenClaw runs on, not just cloud-worker environments:
- protocol: desktop.observe / desktop.launch with a discriminated
DesktopSource union (host | environment) plus an additive auth hint;
EnvironmentSummary gains a top-level desktop flag
- config: desktop.host { enabled, port?, passwordFile? }, Labs-gated
- rfb-probe: pure RFB version/security-type parser used to detect an
already-running loopback VNC server and classify its auth
- host-source: attaches to 127.0.0.1:<port>, refuses unauthenticated
(None) and unsupported (VeNCrypt) servers, and refuses ARD with the
supported alternative until the macOS milestone
- host-guidance: per-OS enablement text so no path dead-ends
- doctor + status report host desktop availability and auth type only
worker.desktop.observe/launch stay as delegating aliases with identical
behavior. Also drops the now-unused WorkerDesktopTunnels type export.
Live-verified against macOS Screen Sharing: probe reads RFB 003.889,
returns security types [30,33,36,35], classifies ard-account.
* test(gateway): probe RFB handshakes through the socket boundary
The probe's banner and security-offer parsers were exported solely so
unit tests could call them, which the dead-export gate rejects and which
tests internals rather than behavior. Keep them module-local and drive
the probe through a scripted loopback server instead.
The boundary tests also cover what pure-function vectors could not:
handshakes split across packets, legacy RFB 3.3 single-word security,
server-rejected handshakes, early hangups, and connect timeouts.
* feat(ui): let the Desktop panel view this machine, not just cloud workers
The Desktop panel was gated on a cloud-worker session placement, so an
operator running OpenClaw locally had no way to see the machine hosting
their main session even with a VNC server running on it.
Availability now follows the advertised desktop.observe method plus
operator.admin instead of session placement, and the picker lists every
environment whose summary reports a desktop, with the gateway row shown
as "This machine". Sources are passed to the generic desktop.observe /
desktop.launch RPCs; the app launcher stays worker-only. When a host
attach needs a password the gateway did not supply, the panel prompts and
keeps the value in memory for that connection only.
Adds the hostDesktop Labs toggle for desktop.host.enabled.
* fix(scripts): keep the env-var ratchet usable in shallow checkouts
The env-var budget check resolved its base ref, then hard-failed when
`git merge-base` found no shared ancestor. Shallow clones and grafted
agent checkouts resolve origin/main but truncate the history behind it,
so an advisory growth ratchet took down the whole check:changed gate
with "Could not resolve env-var count merge base for: origin/main".
Only the growth comparison needs a baseline, and the script already has
a no-baseline path. Treat git's exit 1 with empty output (no shared
ancestor) as that case and say so on stderr; a genuine failure still
exits 128 and still throws, and the absolute count-vs-budget check runs
either way.
* test(ui): measure the inline-code chip against its line box
The inline-code spacing test compared the chip's height to a prose text
rect, so it silently measured the monospace font's default line spacing.
That is ~17px on macOS and several px shorter on Linux, so the assertion
passed on CI and failed locally at 4.5 against a 3.75 bound -- after the
bound had already been widened once to chase browser font metrics.
Compare the chip to the paragraph's CSS line box instead, which is what
"the chip must not disrupt the line" actually means and is platform
independent. The horizontal gap stays as-is: it is em-derived padding
plus border, and it is the assertion that catches detached punctuation.
Verified both directions on macOS: the file is fully green, and
restoring the pre-fix 0.15em/0.35em padding still fails the gap
assertion at 5.41.
* feat(gateway): view macOS Screen Sharing from the Desktop panel
Modern macOS only offers ARD account authentication for Screen Sharing,
so the host desktop source refused every Mac. The Gateway now performs
the ARD handshake itself against the loopback server and hands the
browser a plain RFB 003.008 no-auth handshake, so the operator's macOS
account password authenticates the desktop without ever reaching the
browser, the observe result, a URL, or a log.
- rfb-preauth: ARD (type 30) Diffie-Hellman with MD5-derived AES-128-ECB
credentials, and VncAuth (type 2) bit-reversed DES, both under a single
10s negotiation deadline; Apple's RFB 003.889 maps to 3.8
- observe-bridge: runs pre-auth before splicing and starts the view-only
filter at clientInit, since the browser handshake is consumed here;
worker tokens keep the original version start phase
- host-source: attaches ARD, requiring per-observation credentials that
live only in the one-shot observer token and are dropped after use
- doctor: offers an explicitly confirmed sudo launchctl repair when
Screen Sharing is off, and prints the System Settings path otherwise
Live-verified against this Mac's Screen Sharing: the DH exchange and
credential framing are accepted and the server returns SecurityResult.
The VncAuth DES vector is confirmed against OpenSSL independently.
OpenClaw Android App
OpenClaw Android is the officially released Google Play app. It connects to an OpenClaw Gateway as a companion node for chat, voice, approvals, screen, and device-aware automation.
Current App Surface
- New 4-step onboarding flow
- Connect tab with
Setup Code+Manualmodes - Encrypted persistence for gateway setup/auth state
- Chat UI restyled
- Settings UI restyled and de-duplicated (gateway controls moved to Connect)
- QR code scanning in onboarding
- Performance improvements
- Streaming support in chat UI
- Dedicated per-device Android chat session created/adopted on connect without resetting history
- Request camera/location and other permissions in onboarding/settings flow
- Push notifications for gateway/chat status updates
- Security hardening (biometric lock, token handling, safer defaults)
- Authenticated background presence beacons
- Voice tab full functionality
- Foreground on-device Voice Wake with Gateway-synced wake words
- Screen tab full functionality
- Skill Workshop settings can filter proposals, inspect proposal content, and apply/reject/quarantine drafts through Gateway RPCs
- Skills settings can search installed skills, enable or disable them, and install Gateway-verified ClawHub releases
- Per-app language selection for translated resources follows Android system settings and persistence
- Cron job settings support details, run history, run now, edits, enable/disable, and deletion with admin-scoped Gateway access
- Wear OS companion proxies sessions, transcripts, replies, aborts, and realtime Talk through the paired phone without storing Gateway credentials on the watch
Open in Android Studio
- Run
pnpm installfrom the repository root so native Canvas resources can be generated. - Open the folder
apps/android.
Wear OS companion
The wear app is a paired-phone companion with the same application ID and signing identity as the phone app. The watch discovers the phone through Wear OS Data Layer, then uses the phone's existing authenticated operator session. It never receives or stores Gateway tokens, passwords, TLS pins, or device-signing identity.
The watch supports agent and session selection, bounded text-only transcript history, streaming reply state, text and voice replies, abort, realtime Talk within the selected session, paired-phone Gateway controls, local reply notifications, theme and automatic-speech settings, and a launch Tile. Realtime Talk streams watch microphone and playback audio over a temporary Wear OS Data Layer channel; it still uses the phone's authenticated Gateway session and closes when the selected phone or Gateway connection changes. A missing Data Layer event sequence or changed phone-process epoch triggers a fresh history request instead of applying uncertain deltas. Agent and Gateway controls are capability-negotiated so an older paired phone remains usable during staggered updates.
cd apps/android
./gradlew :wear:testDebugUnitTest :wear:assembleDebug :wear:lintDebug :wear:ktlintCheck
Build / Run
cd apps/android
./gradlew :app:assemblePlayDebug
./gradlew :app:installPlayDebug
./gradlew :app:testPlayDebugUnitTest
cd ../..
pnpm android:release:archive
Third-party debug flavor:
cd apps/android
./gradlew :app:assembleThirdPartyDebug
./gradlew :app:installThirdPartyDebug
./gradlew :app:testThirdPartyDebugUnitTest
Repository-backed debug Gradle invocations, including pnpm android:run and
pnpm android:screenshots, stamp the full checkout commit and capture one UTC
build timestamp shared by every debug variant in that invocation. Release
tasks still require explicit openclawBuildCommit and
openclawBuildTimestamp properties so signed artifacts remain reproducible.
Android release archives use the pinned version in apps/android/version.json. Update it with:
pnpm android:version
pnpm android:version:check
pnpm android:version:pin -- --from-gateway
pnpm android:version:pin -- --version 2026.6.5 --version-code 2026060501
Release-owner signing sync:
pnpm android:release:signing:plan
MATCH_PASSWORD=<signing repo password> pnpm android:release:signing:sync:pull
MATCH_PASSWORD=<signing repo password> pnpm android:release:signing:check
The signing sync pulls encrypted Android upload-key assets from the shared apps-signing repo and materializes decrypted files under apps/android/build/release-signing/.
Standalone release APK verification also requires that key's public certificate SHA-256 fingerprint to match Config/ReleaseSigning.json.
Generate phone and Wear OS Google Play screenshots:
pnpm android:screenshots
The screenshot script captures both form factors with retained
OpenClaw_Screenshots_API36 (Pixel 2) and
OpenClaw_Wear_Screenshots_API34 (Wear OS Large Round) AVDs. It creates a
missing AVD, boots it headlessly, waits for Android to finish booting, disables
animations, captures the screenshots, then shuts down the emulator it started.
Install the API 36 Google APIs and API 34 Wear OS system images in the local
Android SDK. Use --form-factor phone|wear with --avd or --device to
explicitly capture one form factor from another emulator.
pnpm android:release:archive builds signed release artifacts into apps/android/build/release-artifacts/ and writes .sha256 checksum files:
- Play build:
openclaw-<version>-play-release.aab - Wear build:
openclaw-<version>-wear-release.aab - Third-party build:
openclaw-<version>-third-party-release.apk
pnpm android:bundle:release is an alias for the same Fastlane archive lane.
Regular final and correction OpenClaw releases publish the signed third-party APK as OpenClaw-Android.apk with a checksum manifest and GitHub Actions provenance. .github/workflows/android-release.yml is the only automated GitHub Release upload path; OpenClaw Release Publish dispatches it while the canonical release is still a draft and blocks publication until the uploaded asset contract verifies.
The protected android-release environment supplies MATCH_PASSWORD; the repository's read-only GitHub App token checks out encrypted material from openclaw/apps-signing. The workflow builds the exact release tag, refuses to replace different existing bytes, and re-downloads the APK for checksum, certificate, and provenance verification.
pnpm android:release:archive is for local archive validation only. It is not a
fallback upload path after pnpm android:release:upload fails.
Agent-driven Google Play uploads must use pnpm android:release:upload as the
only release path. If that command fails, stop and fix the failing screenshot,
metadata, signing, validation, archive, or upload step before trying again. Do
not upload archived artifacts through direct Fastlane lanes, Gradle artifacts,
Google Play API commands, or Play Console mutation commands.
The release lane uploads the phone and Wear bundles in one atomic Google Play
edit. It publishes the phone bundle to GOOGLE_PLAY_TRACK and maps the Wear
bundle to the corresponding form-factor track (wear:<track>), so the default
internal channel publishes to internal and wear:internal.
See apps/android/VERSIONING.md and apps/android/fastlane/SETUP.md for the release workflow.
Prefer pnpm android:release:archive, which stamps and validates the full Git commit and one UTC build timestamp before signing. Flavor-specific direct Gradle release tasks must pass the same metadata explicitly:
cd apps/android
commit="$(git -C ../.. rev-parse HEAD)"
built_at="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
./gradlew -PopenclawBuildCommit="$commit" -PopenclawBuildTimestamp="$built_at" :app:bundlePlayRelease
./gradlew -PopenclawBuildCommit="$commit" -PopenclawBuildTimestamp="$built_at" :wear:bundleRelease
./gradlew -PopenclawBuildCommit="$commit" -PopenclawBuildTimestamp="$built_at" :app:bundleThirdPartyRelease
Kotlin Lint + Format
pnpm android:lint
pnpm android:format
Android framework/resource lint (separate pass):
pnpm android:lint:android
Direct Gradle tasks:
cd apps/android
./gradlew :app:ktlintCheck :benchmark:ktlintCheck :wear:ktlintCheck :wear-shared:ktlintCheck
./gradlew :app:ktlintFormat :benchmark:ktlintFormat :wear:ktlintFormat :wear-shared:ktlintFormat
./gradlew :app:lintPlayDebug :app:lintThirdPartyDebug :wear:lintDebug :wear-shared:lintDebug
gradlew auto-detects the Android SDK at ~/Library/Android/sdk (macOS default) if ANDROID_SDK_ROOT / ANDROID_HOME are unset.
Macrobenchmark (Startup + Frame Timing)
cd apps/android
./gradlew :benchmark:connectedDebugAndroidTest
Reports are written under:
apps/android/benchmark/build/reports/androidTests/connected/
Perf CLI (low-noise)
Deterministic startup measurement + hotspot extraction with compact CLI output:
cd apps/android
./scripts/perf-startup-benchmark.sh
./scripts/perf-startup-hotspots.sh
Benchmark script behavior:
- Runs only
StartupMacrobenchmark#coldStartup(10 iterations). - Prints median/min/max/COV in one line.
- Writes timestamped snapshot JSON to
apps/android/benchmark/results/. - Auto-compares with previous local snapshot (or pass explicit baseline:
--baseline <old-benchmarkData.json>).
Hotspot script behavior:
- Ensures debug app installed, captures startup
simpleperfdata for.MainActivity. - Prints top DSOs, top symbols, and key app-path clues (Compose/MainActivity/WebView).
- Writes raw
perf.datapath for deeper follow-up if needed.
Run on a Real Android Phone (USB)
- On phone, enable Developer options + USB debugging.
- Connect by USB and accept the debugging trust prompt on phone.
- Verify ADB can see the device:
adb devices -l
- Install + launch debug build:
pnpm android:install
pnpm android:run
If adb devices -l shows unauthorized, re-plug and accept the trust prompt again.
USB-only gateway testing (no LAN dependency)
Use adb reverse so Android localhost:18789 tunnels to your laptop localhost:18789.
Terminal A (gateway):
pnpm openclaw gateway --port 18789 --verbose
Terminal B (USB tunnel):
adb reverse tcp:18789 tcp:18789
Then in app Connect → Manual:
- Host:
127.0.0.1 - Port:
18789 - TLS: off
Hot Reload / Fast Iteration
This app is native Kotlin + Jetpack Compose.
- For Compose UI edits: use Android Studio Live Edit on a debug build (works on physical devices; project
minSdk=31already meets API requirement). - For many non-structural code/resource changes: use Android Studio Apply Changes.
- For structural/native/manifest/Gradle changes: do full reinstall (
pnpm android:run). - Canvas web content already supports live reload when loaded from Gateway
__openclaw__/canvas/(seedocs/platforms/android.md).
Connect / Pair
- Start the gateway (on your main machine):
pnpm openclaw gateway --port 18789 --verbose
- In the Android app:
- Open the Connect tab.
- Use Setup Code or Manual mode to connect.
- Approve pairing (on the gateway machine):
openclaw devices list
openclaw devices approve <requestId>
More details: docs/platforms/android.md.
Permissions
- Discovery:
- Android 13+ (
API 33+):NEARBY_WIFI_DEVICES - Android 12 and below:
ACCESS_FINE_LOCATION(required for NSD scanning)
- Android 13+ (
- Location:
- Both flavors:
ACCESS_FINE_LOCATION/ACCESS_COARSE_LOCATIONfor foreground checks. - Third-party flavor only:
ACCESS_BACKGROUND_LOCATIONplusFOREGROUND_SERVICE_LOCATIONfor user-enabledAlwayschecks.
- Both flavors:
- Foreground service notification (Android 13+):
POST_NOTIFICATIONS - Camera:
CAMERAforcamera.snapandcamera.clipRECORD_AUDIOforcamera.clipwhenincludeAudio=true
Google Play Restricted Permissions
As of March 19, 2026, these manifest permissions are the main Google Play policy risk for this app:
READ_SMSSEND_SMSREAD_CALL_LOG
Why these matter:
- Google Play treats SMS and Call Log access as highly restricted. In most cases, Play only allows them for the default SMS app, default Phone app, default Assistant, or a narrow policy exception.
- Review usually involves a
Permissions Declaration Form, policy justification, and demo video evidence in Play Console. - The Play build removes these behind the
playflavor. - Photo library access is also removed from the Play build. Use third-party builds for
photos.latest.
Current OpenClaw Android implication:
- APK / sideload build can keep SMS, Call Log, and recent-photo features.
- Google Play build excludes SMS send/search, Call Log search, and recent-photo access unless the product is intentionally positioned and approved under the relevant policy exception.
- The repo now ships this split as Android product flavors:
play: removesREAD_SMS,SEND_SMS,READ_CALL_LOG,READ_MEDIA_IMAGES,READ_MEDIA_VISUAL_USER_SELECTED,READ_EXTERNAL_STORAGE, and background location; hides SMS, Call Log, Photos, andAlwayslocation surfaces.- Installed-app listing is user controlled.
device.appsis advertised only after the user enables Settings > Phone Capabilities > Installed Apps. The command defaults to launcher-visible apps and does not requireQUERY_ALL_PACKAGES. thirdParty: keeps the full permission set and the existing SMS / Call Log / Photos functionality, and offers explicitAlwayslocation opt-in through Android settings.
Policy links:
- Google Play SMS and Call Log policy
- Google Play sensitive permissions policy hub
- Android default handlers guide
Other Play-restricted surfaces to watch if added later:
ACCESS_BACKGROUND_LOCATIONMANAGE_EXTERNAL_STORAGEQUERY_ALL_PACKAGESREQUEST_INSTALL_PACKAGESAccessibilityService
Reference links:
Integration Capability Test (Preconditioned)
This suite assumes setup is already done manually. It does not install/run/pair automatically.
Pre-req checklist:
- Gateway is running and reachable from the Android app.
- Android app is connected to that gateway and
openclaw nodes statusshows it as paired + connected. - App stays unlocked and in foreground for the whole run.
- Open the app Screen tab and keep it active during the run (canvas/A2UI commands require the canvas WebView attached there).
- Grant runtime permissions for capabilities you expect to pass (camera/mic/location/notification listener/location, etc.).
- No interactive system dialogs should be pending before test start.
- Canvas host is enabled and reachable from the device for remote Canvas checks (do not run gateway with
OPENCLAW_SKIP_CANVAS_HOST=1; startup logs should includecanvas host mounted at .../__openclaw__/). - Local operator test client pairing is approved. If first run fails with
pairing required, preview the latest pending request, approve the printed request ID, then rerun: - For A2UI checks, keep the app on Screen tab; the node uses its bundled app-owned A2UI page for message application.
openclaw devices list
openclaw devices approve --latest # preview only; copy the requestId from output
openclaw devices approve <requestId>
Run:
pnpm android:test:integration
Optional overrides:
OPENCLAW_ANDROID_GATEWAY_URL=ws://...(default: from your local OpenClaw config)OPENCLAW_ANDROID_GATEWAY_TOKEN=...OPENCLAW_ANDROID_GATEWAY_PASSWORD=...OPENCLAW_ANDROID_NODE_ID=...orOPENCLAW_ANDROID_NODE_NAME=...
What it does:
- Reads
node.describecommand list from the selected Android node. - Invokes advertised non-interactive commands.
- Skips
screen.recordandtalk.ptt.*in this suite because they require interactive capture. Useapps/android/scripts/voice-e2e.shfor microphone and voice-path proof. - Asserts command contracts (success or expected deterministic error for safe-invalid calls like
sms.sendandnotifications.actions).
Common failure quick-fixes:
pairing requiredbefore tests start:- list pending requests (
openclaw devices list), then approve with the exact ID (openclaw devices approve <requestId>) and rerun.
- list pending requests (
A2UI host not reachable/A2UI_HOST_UNAVAILABLE:- keep the app foregrounded on the Screen tab and rerun. A2UI commands use the bundled app-owned A2UI page; the Gateway Canvas host is still needed for remote Canvas checks, but not for A2UI message application.
NODE_BACKGROUND_UNAVAILABLE: canvas unavailable:- app is not effectively ready for canvas commands; keep app foregrounded and Screen tab active.
Contributions
Maintainer: @obviyus. For issues/questions/contributions, please open an issue or reach out on Discord.