diff --git a/.agents/skills/auto-qa/SKILL.md b/.agents/skills/auto-qa/SKILL.md index f8035b56959d..80e0fbfda1a3 100644 --- a/.agents/skills/auto-qa/SKILL.md +++ b/.agents/skills/auto-qa/SKILL.md @@ -5,7 +5,15 @@ description: "Continuously audit, live-test, and stress-test the current OpenCla # OpenClaw Auto QA -Run a continuous, current-`main` OpenClaw product campaign. Treat a reviewer finding as a hypothesis, a passing test as evidence only for its actual head, and a merge as complete only when the canonical repository confirms it. Repair the actual root cause in its canonical owner; a smaller patch is not better if it leaves sibling paths, lifecycle invariants, or the defective abstraction intact. +Run a continuous, current-`main` OpenClaw product campaign. Treat a reviewer finding as a hypothesis, a passing test as evidence only for its actual head, and a merge as complete only when the canonical repository confirms it. Always prefer a clean, appropriately scoped root-cause refactor over a quick fix or smaller diff. Repair the actual root cause in its canonical owner; a patch is not acceptable when it leaves sibling paths, lifecycle invariants, or the defective abstraction intact. + +## Prefer clean refactors over quick fixes + +- Identify the broken ownership boundary, abstraction, state transition, or dependency contract before choosing an implementation. Compare the canonical owner, callers, callees, and sibling paths; prefer the design that makes their shared invariant obvious and reliable. +- Consolidate decisions and authoritative state in their actual owner. Propagate prepared facts through existing lifecycles, repair all affected siblings, and delete obsolete branches, duplicate policy, dead helpers, and stale abstractions when they are no longer needed. +- Reject symptom-masking guards, one-off exceptions, observed-example literals, parallel code paths, extra caches, fallback stacks, compatibility shims, and tests that merely make a narrow reproduction pass. A smaller change is not safer when it preserves the cause or makes the architecture harder to understand. +- Preserve shipped public contracts and ownership boundaries. If the clean refactor would affect security, persistent state, public configuration, plugin SDK compatibility, a protocol, or a product decision, mark it for maintainer review instead of substituting a tactical patch. +- During independent review, explicitly ask whether the change is the cleanest appropriately bounded root-cause solution. Green tests, a minimal diff, and a plausible local fix are insufficient without that architectural judgment. ## Start with the moving source @@ -48,10 +56,10 @@ Read [references/live-proof-routing.md](references/live-proof-routing.md) before 1. Deduplicate against the current ledger, `origin/main`, current open and merged GitHub work, and sibling root causes. Count one broken invariant once, even when it produces multiple model, platform, route, lifecycle, or UI symptoms. 2. Independently reproduce the actual current-main user path. Map the entry point, canonical owner, callers, callees, sibling implementations, state lifecycle, existing regressions, shipped contracts, and relevant direct upstream source. Identify why the current design fails before proposing a repair. -3. Refactor the canonical owner in an isolated worktree. Repair all affected sibling paths in the same coherent change, simplify or remove the defective abstraction, and carry authoritative facts through the existing lifecycle. Prefer the appropriately sized root-cause solution over a minimal guard, special case, extra cache, fallback, compatibility shim, or narrowly passing test. +3. Refactor the canonical owner in an isolated worktree. Repair all affected sibling paths in the same coherent change, simplify or remove the defective abstraction, and carry authoritative facts through the existing lifecycle. Prefer the cleanest appropriately sized root-cause solution over a minimal diff; reject a guard, special case, extra cache, fallback, compatibility shim, or narrowly passing test that leaves the architectural defect behind. 4. Preserve public configuration, plugin ownership, gateway protocol, migrations, provider contracts, persistent state, and external dependencies. When a correct root-cause repair would change a sensitive contract or requires a product decision, prepare it for operator review; do not disguise that risk as a small autonomous fix. 5. Add authentic regression coverage for the original reproduction, affected siblings, lifecycle cleanup, and unchanged legitimate behavior. Run appropriately scoped proof on the exact candidate head. Route Docker, real providers, packaging, full checks, typechecking, broad suites, and browser work through the existing remote workflow; inspect actual exit status, nonzero scenario counts, and artifacts. -6. Run a fresh `$autoreview` on the complete final refactor. Resolve actionable findings; rerun review after any production, test, or head change. Personally read the latest ClawSweeper review, satisfy each applicable rank-up move with real evidence, and update the existing PR body before landing. +6. Run a fresh `$autoreview` on the complete final refactor. Require the reviewer to compare owner boundaries and sibling implementations, confirm this is the best clean root-cause solution, and reject quick-fix residue even when tests pass. Resolve actionable findings; rerun review after any production, test, or head change. Personally read the latest ClawSweeper review, satisfy each applicable rank-up move with real evidence, and update the existing PR body before landing. 7. Check existing open PRs, current author counts, and the actual repository automation before publishing. Read both the current labeler and response policy; verify the authenticated author association, repository permission, account type, automation branch prefix, and actual override label. Apply only exemptions proved by that current policy, including eligible owners, maintainers, collaborators, bots or apps, approved automation branches, and explicit overrides. Never infer capacity from a truncated list or assume that one privileged role represents every exemption. Reuse and repair an existing candidate PR for the same cause. When a real cap applies, hold reviewed worktrees and finish or land existing verified work first. 8. Create a focused PR with the repository's actual template, canonical cause, user impact, frozen head, completed proof, and risk. Use only the current repo-native `scripts/pr` review, artifact, prepare, and merge workflow for authorized main landing. 9. Autonomously merge only when the user authorized it **and** the canonical root-cause refactor is individually reproduced, low-risk, independently reviewed, current-main-compatible, and has green required exact-head proof. Evaluate risk by ownership and behavioral impact, not by whether the diff is the smallest possible. Verify the resulting canonical merge SHA before incrementing the ledger. diff --git a/.agents/skills/openclaw-autonomous-issue-sweep/SKILL.md b/.agents/skills/openclaw-autonomous-issue-sweep/SKILL.md new file mode 100644 index 000000000000..9ce683ef9957 --- /dev/null +++ b/.agents/skills/openclaw-autonomous-issue-sweep/SKILL.md @@ -0,0 +1,253 @@ +--- +name: openclaw-autonomous-issue-sweep +description: "Orchestrate 64 autonomous OpenClaw issue workers newest-to-oldest; find existing PRs, deeply investigate bugs, simplify or refactor, live-test, independently review, land verified fixes, close already-fixed issues, and add only meaningful new evidence." +--- + +# OpenClaw Autonomous Issue Sweep + +Run an end-to-end maintainer campaign, not a candidate shortlist. The parent +conversation is the orchestrator: delegate discovery, investigation, coding, +testing, review, GitHub mutations, PR preparation, landing, and cleanup to +subagents. Keep parent-thread updates to concise progress and clickable URLs. + +## Authority and campaign shape + +- Spawn exactly **64 first-class subagents** unless the user requests another + count or available capacity makes that impossible; disclose the actual count. +- Use full-history forks so every subagent inherits the orchestrator's model + and **xhigh reasoning effort**. Never print, record, or disclose model + identifiers; redact subprocess banners and diagnostics before reporting. +- Treat a request to run this workflow as authority to review, fix, refactor, + commit, push, create/update PRs, land eligible changes, comment, and close + issues individually. Do not ask for routine confirmation again. +- Never treat sweep authority as permission to publish releases, bump protocol + or SQLite schema versions, weaken security, break shipped compatibility, + change another owner's protected product surface, or execute untrusted code + with local credentials. +- Have subagents read the complete root `AGENTS.md`, relevant scoped guides, + `VISION.md`, and companion skills before acting. Use `$gitcrawl`, Octopool, + `$openclaw-pr-maintainer`, `$openclaw-testing`, `$crabbox`, and `$autoreview` + where each owns the workflow. +- Keep the parent out of operational work. It may spawn, assign, receive + results, serialize shared resources, monitor host/pool health, prewarm and + allocate needed remote leases, issue follow-up tasks, and report; it must + not inspect issues, edit code, run tests, mutate GitHub, or land PRs. + +## Coordinate 64 workers safely + +1. Assign one subagent to maintain the live open-issue queue in descending + `createdAt` order, one to coordinate landing/proof capacity, and the rest to + issue investigations. Coordinator agents also investigate when idle. +2. Claim issues from the newest unclaimed end only; replenish workers as they + finish. Parallel completions may arrive out of order, but never knowingly + start an older unclaimed issue ahead of a newer available issue. +3. Deduplicate by canonical root cause, not merely by issue number. Let one + owner fix a shared defect and link related issues/PRs to that outcome. +4. Freeze the reviewed source SHA for each wave. Designate a single fetch owner; + pause shared-ref refreshes while repo-native PR prepare/merge runs. +5. Never switch a shared checkout branch or edit it while sibling agents use it. + Use an existing agent-owned checkout, a repo-native isolated PR worktree, or + an explicitly user-authorized new worktree. Otherwise serialize write + access; parallel read-only investigations may continue. +6. Sample checkout/temp-volume free disk, CPU/load, memory pressure, process + count, operator-gateway health, actual worker count, and Octopool capacity + before each wave and periodically thereafter. Throttle expensive work for + sustained pressure or low disk; never kill unrelated operator processes. +7. Serialize merge operations and each Testbox lease. A lease has one owner and + one active command; never reclaim, sync, or change its head during a run. +8. Respect GitHub rate limits, active assignees, repository ownership, and + existing contributor work. Do not auto-assign broad-discovery candidates. +9. Replace finished workers while the queue remains. Record actual active, + completed, failed, fixed, landed, closed, commented, and skipped counts; + never report launched or finished workers as still running. + +## Conserve GitHub capacity and host resources + +- Prefer local `$gitcrawl` archives and source history for queue discovery, + issue/PR search, duplicate clusters, comments, and previously merged work. + Check archive freshness; do not broadly sync, enrich, or re-embed merely to + start a sweep. +- Prefer `octopool gh ...` or narrowly bounded `octopool request` for + necessary live GitHub reads and mutations. Check `octopool health` and + `octopool stats` periodically; let repo-native PR wrappers retain their + required GitHub transport and authenticated identity. +- Use plain `gh` only when Octopool cannot support the operation or the + canonical maintainer wrapper requires it. Request minimal fields, reuse + results across workers, batch compatible reads, avoid unbounded pagination, + and never use `gh run watch` or frequent unchanged CI polls. +- Require a fresh live state check only before consequential mutations, final + merge decisions, or a stale/contradictory cached result. Rate-limit and + deduplicate worker requests instead of having 64 agents independently fetch + the same issue, PR, author profile, or CI rollup. +- Keep disk, load, memory pressure, active lease IDs, provider trust class, + checkout ownership, and pool capacity in the orchestration ledger. Slow new + assignments, serialize builds/tests, clean only campaign-owned artifacts, + and offload heavy proof before resource pressure threatens the host. +- The parent may prewarm a trusted Crabbox/Testbox lease when a concrete heavy + proof is imminent, then hand its verified lease ID and checkout ownership to + one subagent at a time. Avoid speculative fleets, respect path-scoped lease + ownership, and stop campaign-owned leases before handoff or closeout. +- Keep untrusted contributor proof on a separate sanitized direct-AWS lease; + never transfer a credential-hydrated trusted lease to untrusted work. + +## Search for existing work on every credible issue + +Always investigate existing PRs before implementing a fix: + +1. Read the live issue body, all material comments, labels, assignments, + timeline/cross-references, repro details, affected versions, and ClawSweeper + findings. +2. Search `$gitcrawl` for the issue number, title, error text, affected + subsystem, relevant symbols, duplicate symptoms, open PRs, merged PRs, and + recently closed work. +3. Verify candidates against Octopool-backed live GitHub search, directly + linked PRs, current PR heads, `origin/main`, and commit history. Search + exact issue references and symptom/root-cause terms; do not stop at the + first plausible PR. +4. Read competing implementations deeply enough to decide whether an existing + PR already fixes the real defect, merely masks one symptom, has gone stale, + or reveals a cleaner owner-boundary refactor. +5. Preserve contributor commits, attribution, issue reporter credit, and useful + ideas whenever repairing or replacing existing work. + +Choose outcomes in this order: + +1. **Fixed on main:** prove the original failure is resolved; close with the + exact merged PR, commit, current source/test, or release proof. +2. **Existing PR is the best fix:** improve it as needed, verify the exact + final head, and land it through the repo-native maintainer workflow. +3. **Existing PR is useful but incomplete:** finish it or create a cleaner + replacement that preserves human attribution and links the original. +4. **No suitable PR:** implement the best high-confidence root-cause repair or + a justified simplifying refactor; create, verify, and land a focused PR. +5. **Bug cannot be fixed, but simplification is real:** independently land a + proven behavior-neutral refactor when it meaningfully removes complexity + without pretending the original issue was fixed. +6. **Cannot fix or close:** comment only if investigation uncovered concrete, + material evidence missing from the issue and ClawSweeper's existing review. + +## Prove the bug and choose the best design + +- Trace the actual user path from entry point through caller, canonical owner, + callee, sibling implementations, transport/lifecycle boundaries, tests, + current `main`, shipped contracts, and direct dependency source or docs. +- Personally inspect sibling `../codex` source before any Codex integration + verdict or change, as required by the root guide; another agent's report is + not sufficient for the agent making that decision. +- Require a failing regression, reproducible command, real logs, live product + behavior, dependency contract, or exact source-level proof. Never repair an + issue on title, speculation, ClawSweeper output, or a plausible diff alone. +- Prefer the correct owner-boundary refactor over a narrow guard, workaround, + new fallback, duplicate policy, extra configuration, or compatibility shim. + A larger refactor is appropriate when it fixes the whole bug class more + clearly and its behavior/ownership risk remains understood and bounded. +- While reading, look for dead branches, unused helpers, duplicate paths, + stale abstractions, obsolete tests, and complexity that can be deleted as + part of the same coherent change. +- Measure `git diff --numstat`; aim to reduce **production LOC**, excluding + tests. Production growth is acceptable only when clearly justified by fewer + concepts, better ownership, essential product behavior, or stronger safety. +- Allow small missing product affordances, such as an obviously expected CLI + command, when adjacent behavior and docs establish the contract. Reject + substantial new features, speculative redesign, new paid services, + unsupported integrations, or unrelated drive-by changes. +- Do not edit `CHANGELOG.md`; capture user impact, issue/PR references, and + human credit in the PR body or commit message. + +## Verify behavior and obtain two independent reviews + +For every non-trivial production change: + +1. Add focused regression coverage for the original bug and affected sibling + paths. Delete tests protecting removed obsolete implementation details. +2. Choose proof with `$openclaw-testing`. Live-test the real user/provider/ + channel/CLI/package/UI path whenever feasible. Route heavy, packaging, + Docker, E2E, or broad checks through `$crabbox`; report an unavailable live + prerequisite accurately instead of calling a mock live proof. +3. Classify source trust before executing anything. Never run contributor/fork + scripts, hooks, config, tests, installs, or wrappers locally or on a + credential-hydrated host; follow the sanitized untrusted-source workflow. +4. Run `$autoreview` on the complete final change until no accepted actionable + findings remain. Re-run it after any production, test, or reviewed-head + change. Treat review findings as hypotheses and verify each against source. + Prose-only skill files and other non-production internal notes do not need + autoreview; validate their structure and formatting instead. +5. Separately self-invoke an independent Codex reviewer. First verify the + installed interface with `codex exec --help`, then run a bounded read-only, + ephemeral review from a trusted checkout, for example: + + ```bash + codex exec --json --sandbox read-only --ephemeral \ + -C "$trusted_checkout" --output-last-message "$review_result" \ + "Independently inspect the frozen candidate diff and its owner, callers, + siblings, tests, current main, user behavior, and dependency contracts. + Report only concrete correctness, architecture, simplification, or + verification gaps. Do not modify files or expose secrets." \ + >/dev/null 2>/dev/null + ``` + + Point the reviewer at the exact immutable diff/head. Do not substitute the + `$autoreview` Codex engine for this separate pass. Never run that reviewer + from an untrusted project-controlled checkout. Read only the final review + result; do not emit raw model banners. Verify actionable findings, make + justified fixes, rerun proof, and refresh both independent reviews. + +6. Read the latest ClawSweeper comment and address each applicable `Rank-up +moves:` item with real evidence or an explicit reason for skipping it. + +## Publish, land, and clean up + +- Prefer an existing writable contributor PR. If its head is unsuitable or + cannot be updated safely, open a focused replacement, explain the + relationship, and preserve attribution. +- Before opening replacement PRs, verify author association, active-PR counts, + repository permission, branch policy, current auto-response exemptions, and + override labels; never assume a privileged-role exemption. Reuse or land + existing reviewed work before creating a burst of competing PRs. +- Use the actual PR template and state the user impact, canonical root cause, + rejected alternatives, production LOC delta, exact head SHA, focused/live + proof, autoreview result, independent Codex result, CI state, and credit. +- Read `$agent-transcript` for agent-created PRs, but do not include logs + without the user's explicit transcript approval. During a fully autonomous + sweep, omit transcripts rather than interrupting the user for consent. +- Open new PRs as drafts, wait for a non-null mergeability result, mark them + ready, and verify CI attached to the exact pushed head before landing. +- Autonomously land only a reproduced, high-confidence, bounded-risk repair + or behavior-neutral simplification with clean independent reviews and green + exact-head required proof. Change size alone is not the risk criterion. +- For main-targeted PRs use only the repo-native `scripts/pr` flow: initialize + review, create/validate review artifacts, run + `OPENCLAW_TESTBOX=1 scripts/pr prepare-run `, then + `scripts/pr merge-run `. Verify the canonical merge SHA afterward. +- Keep owner/security/auth/config/public-SDK/protocol/persistent-state/product + decisions outside autonomous landing when the relevant guide requires owner + judgment. Continue with the next issue instead of blocking the whole sweep. +- Close a fixed issue only after live rechecking its open state and matching + the original symptoms to current-main proof. Cite the merged PR/commit and + ask the reporter to reopen if it still reproduces on the current version. +- Never close merely because a repro is difficult, the report is inconvenient, + the behavior might be intentional, or the PR is stale. Product-decision and + won't-implement closures require maintainer judgment. +- If no fix is possible, comment only when supplying new reproducible steps, + an exact failing owner/line, verified dependency behavior, previously + unidentified duplicate/fixing PR, a concrete workaround, or another + meaningful fact absent from prior discussion and ClawSweeper. +- Recheck live state immediately before every mutation; avoid redundant, + speculative, noisy, or duplicate comments. Handle closures individually and + follow repository limits on bulk operations. + +## Parent-thread reporting + +Send concise progress plus URLs only. Prefer updates such as: + +```text +64 agents active · 41 investigated · 3 landed · 5 already-fixed issues closed +Landed: https://github.com/openclaw/openclaw/pull/123 +Closed: https://github.com/openclaw/openclaw/issues/456 +``` + +Do not narrate routine reads, pending hypotheses, unchanged CI, or candidate +URLs that are not actually ready. Count only verified merged PRs, confirmed +closures, and comments that were really posted. Continue until the user stops +the sweep, the requested boundary is reached, or the live issue queue is +genuinely exhausted. diff --git a/.agents/skills/openclaw-autonomous-issue-sweep/agents/openai.yaml b/.agents/skills/openclaw-autonomous-issue-sweep/agents/openai.yaml new file mode 100644 index 000000000000..ac7c3f2276c8 --- /dev/null +++ b/.agents/skills/openclaw-autonomous-issue-sweep/agents/openai.yaml @@ -0,0 +1,4 @@ +interface: + display_name: "OpenClaw Autonomous Issue Sweep" + short_description: "Autonomously fix, refactor, land, and close issues" + default_prompt: "Use $openclaw-autonomous-issue-sweep to orchestrate 64 subagents through OpenClaw issues newest to oldest; reuse existing PRs, prove and land high-confidence fixes or refactors, close resolved issues, and report concise progress plus URLs." diff --git a/docs/cli/openclaw.md b/docs/cli/openclaw.md index a7581f970ba9..7b7ec72b4efd 100644 --- a/docs/cli/openclaw.md +++ b/docs/cli/openclaw.md @@ -265,8 +265,10 @@ a single OpenClaw authority tool plus the inert native planning utility. In all three cases, setup writes remain confined to OpenClaw's audited approval contract. -Gemini CLI remains available for normal agents, but it cannot enforce the -tool-free probe required by the inference gate, so it cannot host OpenClaw. +Gemini CLI remains available as an explicitly configured runtime for normal +agents, but Gemini CLI and Antigravity are not inference-gate setup routes. +Use AI Studio API-key or Vertex AI for the inference gate. The optional Gemini +CLI runtime specifically requires an AI Studio API-key profile. ## Switching to an agent diff --git a/docs/cli/setup.md b/docs/cli/setup.md index 67ef90d724a2..4a2961593918 100644 --- a/docs/cli/setup.md +++ b/docs/cli/setup.md @@ -41,8 +41,9 @@ automatic pass. Detected local runtimes are auto-tested after CLI and API-key candidates; when several local models are available, OpenClaw prefers the strongest tool-calling instruct family. The selected candidate must answer a real completion before its provider and model configuration is saved. -Installed Gemini, Antigravity, Pi, and OpenCode CLIs are also reported when -they cannot serve as the reusable inference route for guided setup. +Pi and OpenCode CLIs may also be reported for context when they cannot serve as +the reusable inference route for guided setup. Gemini CLI and Antigravity are +not offered as detected setup routes. `setup` accepts the same onboarding flags as `openclaw onboard`, including auth (`--auth-choice`, `--token`, provider key flags), Gateway diff --git a/docs/concepts/model-providers.md b/docs/concepts/model-providers.md index 8e799ae96c33..23d297866780 100644 --- a/docs/concepts/model-providers.md +++ b/docs/concepts/model-providers.md @@ -234,49 +234,18 @@ Claude CLI reuse (`claude -p`) is a sanctioned OpenClaw integration path. Anthro - Thinking: `/think adaptive` uses Google dynamic thinking. Gemini 3/3.1 omit a fixed `thinkingLevel`; Gemini 2.5 sends `thinkingBudget: -1`. - Direct Gemini runs also accept `agents.defaults.models["google/"].params.cachedContent` (or legacy `cached_content`) to forward a provider-native `cachedContents/...` handle; Gemini cache hits surface as OpenClaw `cacheRead` -### Google Vertex and Gemini CLI +### Google Vertex and Gemini CLI runtime -- Providers: `google-vertex`, `google-gemini-cli` -- Auth: Vertex uses gcloud ADC; Gemini CLI uses its OAuth flow +- `google-vertex`: managed Google Cloud access through gcloud Application + Default Credentials. +- `google-gemini-cli`: optional local runtime for an explicitly configured + canonical `google/*` model. - -Gemini CLI OAuth in OpenClaw is an unofficial integration. Some users have reported Google account restrictions after using third-party clients. Review Google terms and use a non-critical account if you choose to proceed. - - -Gemini CLI OAuth is shipped as part of the bundled `google` plugin. - - - - - - ```bash - brew install gemini-cli - ``` - - - ```bash - npm install -g @google/gemini-cli - ``` - - - - - ```bash - openclaw plugins enable google - ``` - - - ```bash - openclaw models auth login --provider google-gemini-cli --set-default - ``` - - Default model: `google-gemini-cli/gemini-3-flash-preview`. You do **not** paste a client id or secret into `openclaw.json`. The CLI login flow stores tokens in auth profiles on the gateway host. - - - - If requests fail after login, set `GOOGLE_CLOUD_PROJECT` or `GOOGLE_CLOUD_PROJECT_ID` on the gateway host. - - +OpenClaw does not create Gemini CLI OAuth or Antigravity OAuth profiles. Connect +Google through an AI Studio API key or Vertex AI. If you explicitly choose the +Gemini CLI runtime, it can use the selected Google API-key profile. Existing +valid Gemini CLI OAuth profiles remain runtime-compatible, but they are not a +setup or recovery route. Gemini CLI uses `stream-json` by default. OpenClaw reads assistant stream messages and normalizes `stats.cached` into `cacheRead`; legacy diff --git a/docs/concepts/usage-tracking.md b/docs/concepts/usage-tracking.md index 6b7d1b898913..3e22f1eb51b9 100644 --- a/docs/concepts/usage-tracking.md +++ b/docs/concepts/usage-tracking.md @@ -315,7 +315,7 @@ provider-neutral for CLI, app, and Control UI consumers. - **DeepSeek**: API key via env/config/auth store (`DEEPSEEK_API_KEY`). Shows each provider-reported currency balance. - **GitHub Copilot**: OAuth tokens in auth profiles. -- **Gemini CLI**: OAuth tokens in auth profiles. +- **Gemini CLI**: existing OAuth profiles or supported Google API-key profiles. - **MiniMax**: API key or MiniMax OAuth auth profile. OpenClaw treats `minimax`, `minimax-cn`, and `minimax-portal` as the same MiniMax quota surface, prefers stored MiniMax OAuth when present, and otherwise falls back diff --git a/docs/docs_map.md b/docs/docs_map.md index bfcbc9512f0f..b6749b39c869 100644 --- a/docs/docs_map.md +++ b/docs/docs_map.md @@ -2800,7 +2800,7 @@ Do not edit it by hand; run `pnpm docs:map:gen`. - H3: Other subscription-style hosted options - H3: OpenCode - H3: Google Gemini (API key) - - H3: Google Vertex and Gemini CLI + - H3: Google Vertex and Gemini CLI runtime - H3: Z.AI (GLM) - H3: Vercel AI Gateway - H3: Other bundled provider plugins diff --git a/docs/gateway/cli-backends.md b/docs/gateway/cli-backends.md index f76e7795841f..2d5ace905d96 100644 --- a/docs/gateway/cli-backends.md +++ b/docs/gateway/cli-backends.md @@ -231,7 +231,11 @@ The bundled Google plugin registers for `google-gemini-cli`: | `sessionMode` | `existing` | | `sessionIdFields` | `["session_id", "sessionId"]` | -Prerequisite: the local Gemini CLI must be installed and on `PATH` as `gemini` (`brew install gemini-cli` or `npm install -g @google/gemini-cli`). +Prerequisites: the local Gemini CLI must be installed and on `PATH` as `gemini` +(`brew install gemini-cli` or `npm install -g @google/gemini-cli`), and the +selected model must have a supported Google AI Studio API-key profile. Existing +valid legacy Gemini CLI OAuth profiles remain runtime-compatible, but OpenClaw +does not create or repair them. Gemini CLI output notes: diff --git a/docs/help/faq-first-run.md b/docs/help/faq-first-run.md index 732456b4b73e..e50ee1947649 100644 --- a/docs/help/faq-first-run.md +++ b/docs/help/faq-first-run.md @@ -606,18 +606,16 @@ and troubleshooting see the main [FAQ](/help/faq). - - Gemini CLI uses a **plugin auth flow**, not a client id or secret in `openclaw.json`. + + OpenClaw does not offer new Gemini CLI OAuth or Antigravity OAuth setup. + Connect Google with an AI Studio API key or Vertex AI instead. - 1. Install Gemini CLI locally so `gemini` is on `PATH`: - - Homebrew: `brew install gemini-cli` - - npm: `npm install -g @google/gemini-cli` - 2. Enable the plugin: `openclaw plugins enable google` - 3. Login: `openclaw models auth login --provider google-gemini-cli --set-default` - 4. Default model after login: `google/gemini-3.1-pro-preview` (runtime `google-gemini-cli`) - 5. Requests failing after login? Set `GOOGLE_CLOUD_PROJECT` or `GOOGLE_CLOUD_PROJECT_ID` on the gateway host and retry. + The optional `google-gemini-cli` runtime remains available for advanced + setups using a supported Google API-key profile. Existing valid legacy + Gemini CLI OAuth profiles remain executable for compatibility, but OpenClaw + cannot create or repair them. - OAuth tokens are stored in auth profiles on the gateway host. Details: [Google](/providers/google), [Model providers](/concepts/model-providers). + Details: [Google](/providers/google), [Model providers](/concepts/model-providers). diff --git a/docs/nodes/audio.md b/docs/nodes/audio.md index 21a012a379fd..563f88c85775 100644 --- a/docs/nodes/audio.md +++ b/docs/nodes/audio.md @@ -37,7 +37,8 @@ If you have not configured models and `tools.media.audio.enabled` is not `false` Install/link provenance is capability evidence, not execution evidence. It never moves a candidate ahead of CPU sherpa by itself. OpenClaw does not load a model during setup or status checks just to probe a backend. Auto-detected whisper.cpp keeps its normal model-run logs enabled so OpenClaw can record the upstream `using … backend` line. Explicit CLI entries keep their configured output flags. -Gemini CLI auto-detect for media understanding was replaced by a sandboxed Antigravity CLI (`agy`) fallback for image/video; audio does not use a CLI fallback beyond the local binaries above. +Gemini CLI and Antigravity are not auto-detected for media understanding. Audio +does not use a CLI fallback beyond the local binaries above. To disable auto-detection, set `tools.media.audio.enabled: false`. To customize, add capability-tagged entries to `tools.media.models`. diff --git a/docs/nodes/media-understanding.md b/docs/nodes/media-understanding.md index 475b013d52ae..9909497c1a76 100644 --- a/docs/nodes/media-understanding.md +++ b/docs/nodes/media-understanding.md @@ -174,9 +174,6 @@ When `tools.media..enabled` is not `false` and no models are configu - Video: Google → Qwen → Moonshot - - First installed `agy` or `antigravity` binary (override with `OPENCLAW_ANTIGRAVITY_CLI`), sandboxed against the media's directory. - To disable auto-detection for a capability: diff --git a/docs/plugins/plugin-inventory.md b/docs/plugins/plugin-inventory.md index a66bc377a0ad..d79d99948bae 100644 --- a/docs/plugins/plugin-inventory.md +++ b/docs/plugins/plugin-inventory.md @@ -51,7 +51,7 @@ Each entry lists the package, distribution route, and description. ## Core npm package -67 plugins +66 plugins - **[admin-http-rpc](/plugins/reference/admin-http-rpc)** (`@openclaw/admin-http-rpc`) - included in OpenClaw. OpenClaw admin HTTP RPC endpoint. @@ -159,8 +159,6 @@ Each entry lists the package, distribution route, and description. - **[sglang](/plugins/reference/sglang)** (`@openclaw/sglang-provider`) - included in OpenClaw. Adds SGLang model provider support to OpenClaw. -- **[synthetic](/plugins/reference/synthetic)** (`@openclaw/synthetic-provider`) - included in OpenClaw. Adds Synthetic model provider support to OpenClaw. - - **[telegram](/plugins/reference/telegram)** (`@openclaw/telegram`) - included in OpenClaw. Adds the Telegram channel surface for sending and receiving OpenClaw messages. - **[together](/plugins/reference/together)** (`@openclaw/together-provider`) - included in OpenClaw. Adds Together model provider support to OpenClaw. @@ -189,7 +187,7 @@ Each entry lists the package, distribution route, and description. ## Official external packages -78 plugins +79 plugins - **[acpx](/plugins/reference/acpx)** (`@openclaw/acpx`) - npm; ClawHub. OpenClaw ACP runtime backend with plugin-owned session and transport management. @@ -319,6 +317,8 @@ Each entry lists the package, distribution route, and description. - **[synology-chat](/plugins/reference/synology-chat)** (`@openclaw/synology-chat`) - npm; ClawHub. Synology Chat channel plugin for OpenClaw channels and direct messages. +- **[synthetic](/plugins/reference/synthetic)** (`@openclaw/synthetic-provider`) - npm; ClawHub: `clawhub:@openclaw/synthetic-provider`. Adds Synthetic model provider support to OpenClaw. + - **[tavily](/plugins/reference/tavily)** (`@openclaw/tavily-plugin`) - npm; ClawHub: `clawhub:@openclaw/tavily-plugin`. Adds agent-callable tools. Adds web search provider support. - **[teams-meetings](/plugins/reference/teams-meetings)** (`@openclaw/teams-meetings`) - npm; ClawHub: `clawhub:@openclaw/teams-meetings`. Join Microsoft Teams meetings as a Chrome browser guest. diff --git a/docs/plugins/reference/synthetic.md b/docs/plugins/reference/synthetic.md index 8eacd887529b..ca38a2076a2f 100644 --- a/docs/plugins/reference/synthetic.md +++ b/docs/plugins/reference/synthetic.md @@ -12,7 +12,7 @@ Adds Synthetic model provider support to OpenClaw. ## Distribution - Package: `@openclaw/synthetic-provider` -- Install route: included in OpenClaw +- Install route: npm; ClawHub: `clawhub:@openclaw/synthetic-provider` ## Surface diff --git a/docs/providers/google.md b/docs/providers/google.md index 51cbcc905219..f2cfcd80aeed 100644 --- a/docs/providers/google.md +++ b/docs/providers/google.md @@ -1,9 +1,9 @@ --- -summary: "Google Gemini setup (API key + OAuth, image generation, media understanding, TTS, web search)" +summary: "Google Gemini setup (AI Studio API key, Vertex AI, optional CLI runtime, and multimodal tools)" title: "Google (Gemini)" read_when: - You want to use Google Gemini models with OpenClaw - - You need the API key or OAuth auth flow + - You need Google AI Studio, Vertex AI, or Gemini CLI runtime guidance --- The Google plugin provides access to Gemini models through Google AI Studio, plus image generation, media understanding (image/audio/video), text-to-speech, and web search via Gemini Grounding. @@ -11,15 +11,17 @@ The Google plugin provides access to Gemini models through Google AI Studio, plu - Provider: `google` - Auth: `GEMINI_API_KEY` or `GOOGLE_API_KEY` - API: Google Gemini API -- Runtime option: `agentRuntime.id: "google-gemini-cli"` reuses Gemini CLI OAuth while keeping model refs canonical as `google/*`. +- Managed-cloud provider: `google-vertex` with Google Cloud Application Default Credentials +- Optional runtime: `agentRuntime.id: "google-gemini-cli"` runs an explicitly configured model through the local Gemini CLI ## Getting started -Choose your preferred auth method and follow the setup steps. +For most installations, use a Google AI Studio API key. Use `google-vertex` when +the Gateway already runs inside a managed Google Cloud environment. - - **Best for:** standard Gemini API access through Google AI Studio. + + **Recommended for:** standard Gemini API access. @@ -70,16 +72,23 @@ Choose your preferred auth method and follow the setup steps. - - **Best for:** signing in with your Google account through Gemini CLI OAuth instead of using a separate API key. + + **Advanced use only:** run a canonical `google/*` model through an installed + Gemini CLI while keeping authentication on the supported AI Studio API-key + path. - - The `google-gemini-cli` provider is an unofficial integration. Some users - report account restrictions when using OAuth this way. Use at your own risk. - + OpenClaw does not offer new Gemini CLI OAuth or Antigravity OAuth setup. + [Google ended consumer Gemini CLI Login with Google access on June 18, 2026](https://developers.google.com/gemini-code-assist/docs/deprecations/code-assist-individuals), + and the [Antigravity terms](https://antigravity.google/terms) prohibit + third-party tools from accessing the service through Antigravity OAuth. Use + an AI Studio API key or Vertex AI instead. - + + Complete the API-key setup in the first tab. OpenClaw must have a usable + `google` API-key profile before the CLI runtime can be selected. + + The local `gemini` command must be available on `PATH`. ```bash @@ -93,46 +102,37 @@ Choose your preferred auth method and follow the setup steps. OpenClaw supports both Homebrew installs and global npm installs, including common Windows/npm layouts. - - ```bash - openclaw models auth login --provider google-gemini-cli --set-default - ``` - - - ```bash - openclaw models list --provider google + + Keep the canonical Google model ref and opt that model into the CLI + runtime: + + ```json5 + { + agents: { + defaults: { + model: { primary: "google/gemini-3.1-pro-preview" }, + models: { + "google/gemini-3.1-pro-preview": { + agentRuntime: { id: "google-gemini-cli" }, + }, + }, + }, + }, + } ``` - - Default model: `google/gemini-3.1-pro-preview` - Runtime: `google-gemini-cli` - - Alias: `gemini-cli` + - Auth: selected Google AI Studio API-key profile + - Model refs: canonical `google/*` - Gemini 3.1 Pro's Gemini API model id is `gemini-3.1-pro-preview`. OpenClaw accepts the shorter `google/gemini-3.1-pro` as a convenience alias and normalizes it before provider calls. + Existing valid Gemini CLI OAuth profiles remain executable for compatibility, + but OpenClaw cannot create or repair them. If one breaks, replace it with a + Google AI Studio API-key profile. - **Environment variables:** - - - `OPENCLAW_GEMINI_OAUTH_CLIENT_ID` / `GEMINI_CLI_OAUTH_CLIENT_ID` - - `OPENCLAW_GEMINI_OAUTH_CLIENT_SECRET` / `GEMINI_CLI_OAUTH_CLIENT_SECRET` - - - If Gemini CLI OAuth requests fail after login, set `GOOGLE_CLOUD_PROJECT` or - `GOOGLE_CLOUD_PROJECT_ID` on the gateway host and retry. - - - - If login fails before the browser flow starts, make sure the local `gemini` - command is installed and on `PATH`. - - - Onboarding auto-detection lists an existing Gemini CLI login but never - auto-tests it because Gemini CLI has no tool-free probe. Choose Gemini CLI - OAuth or a Gemini API key to continue. - - `google-gemini-cli/*` model refs are legacy compatibility aliases. New - configs should use `google/*` model refs plus the `google-gemini-cli` - runtime when they want local Gemini CLI execution. + `google-gemini-cli/*` refs remain legacy compatibility aliases. New configs + should use `google/*` model refs plus the explicit runtime selection above. @@ -468,10 +468,9 @@ roundtrip; pass `--openai-audio-cycles 3` for a short repeated lifecycle soak. - When using the `google-gemini-cli` OAuth provider, OpenClaw uses Gemini - CLI `stream-json` output by default and normalizes usage from the final - `stats` payload. Legacy `--output-format json` overrides still use the - JSON parser. + The optional `google-gemini-cli` runtime uses Gemini CLI `stream-json` + output by default and normalizes usage from the final `stats` payload. + Legacy `--output-format json` overrides still use the JSON parser. - Streamed reply text comes from assistant `message` events. - For legacy JSON output, reply text comes from the CLI JSON `response` field. diff --git a/docs/providers/index.md b/docs/providers/index.md index 28d76cfe4045..11802f05f328 100644 --- a/docs/providers/index.md +++ b/docs/providers/index.md @@ -87,7 +87,7 @@ Looking for chat channel docs (WhatsApp/Telegram/Discord/Slack/Mattermost (plugi ## Shared overview pages -- [Additional provider variants](/providers/models#additional-provider-variants) - Anthropic Vertex, Copilot Proxy, and Gemini CLI OAuth +- [Additional provider variants](/providers/models#additional-provider-variants) - Anthropic Vertex, Copilot Proxy, and the optional Gemini CLI runtime - [Image Generation](/tools/image-generation) - Shared `image_generate` tool, provider selection, and failover - [Music Generation](/tools/music-generation) - Shared `music_generate` tool, provider selection, and failover - [Video Generation](/tools/video-generation) - Shared `video_generate` tool, provider selection, and failover diff --git a/docs/providers/models.md b/docs/providers/models.md index 14e07184a0e7..29dced9eea9e 100644 --- a/docs/providers/models.md +++ b/docs/providers/models.md @@ -57,7 +57,7 @@ For the full provider catalog and advanced configuration, see - `anthropic-vertex` - install `@openclaw/anthropic-vertex-provider` for implicit Anthropic on Google Vertex support when Vertex credentials are available; no separate onboarding auth choice - `copilot-proxy` - local VS Code Copilot Proxy bridge; use `openclaw onboard --auth-choice copilot-proxy` -- `google-gemini-cli` - unofficial Gemini CLI OAuth flow; requires a local `gemini` install (`brew install gemini-cli` or `npm install -g @google/gemini-cli`); default model `google-gemini-cli/gemini-3-flash-preview`; use `openclaw onboard --auth-choice google-gemini-cli` or `openclaw models auth login --provider google-gemini-cli --set-default` +- `google-gemini-cli` - optional explicit runtime for canonical `google/*` models; requires a local `gemini` install and a supported Google AI Studio API-key profile; new Gemini CLI or Antigravity OAuth setup is not offered ## Related diff --git a/docs/providers/synthetic.md b/docs/providers/synthetic.md index 29b1d7649e6f..073cdc93a2f9 100644 --- a/docs/providers/synthetic.md +++ b/docs/providers/synthetic.md @@ -7,8 +7,8 @@ title: "Synthetic" --- [Synthetic](https://synthetic.new) exposes Anthropic-compatible endpoints. -OpenClaw bundles it as the `synthetic` provider and uses the Anthropic -Messages API. +OpenClaw provides it through the official `@openclaw/synthetic-provider` +plugin and uses the Anthropic Messages API. | Property | Value | | -------- | ------------------------------------- | @@ -20,6 +20,12 @@ Messages API. ## Getting started + + ```bash + openclaw plugins install @openclaw/synthetic-provider + openclaw gateway restart + ``` + Get a `SYNTHETIC_API_KEY` from your Synthetic account, or let onboarding prompt you for one. diff --git a/docs/start/onboarding-overview.md b/docs/start/onboarding-overview.md index 5c7b39a186fb..55395673d395 100644 --- a/docs/start/onboarding-overview.md +++ b/docs/start/onboarding-overview.md @@ -93,8 +93,10 @@ offering a verified manual API-key step when nothing is found. Sensitive credentials use masked input. Once inference passes, OpenClaw starts and helps configure the rest. -Gemini CLI remains available for normal agents after setup, but it is not -offered for this inference gate because it cannot enforce the tool-free probe. +Gemini CLI remains available as an explicitly configured runtime after setup, +but Gemini CLI and Antigravity are not offered as detected inference routes. +Use Google AI Studio API-key or Vertex AI for guided setup. The optional Gemini +CLI runtime specifically requires an AI Studio API-key profile. Full reference: [Onboarding (macOS App)](/start/onboarding) diff --git a/docs/start/onboarding.md b/docs/start/onboarding.md index baa91b1ab5a9..bc7be10af8ee 100644 --- a/docs/start/onboarding.md +++ b/docs/start/onboarding.md @@ -91,15 +91,16 @@ To use a Claude subscription when the Gateway host has no Claude CLI login, run printed token as **Anthropic setup-token** under **Connect with an API key or token**. -Installed Gemini CLI, Antigravity, Pi, and OpenCode CLIs are shown for context -when they cannot be selected as the reusable guided-setup inference route. -Gemini and Antigravity cannot enforce the tool-free inference probe. Pi and -OpenCode are whole-agent harnesses rather than setup inference routes; their -session integrations require separate runtime and plugin setup. +Pi and OpenCode installs may be shown for context when they cannot be selected +as the reusable guided-setup inference route. They are whole-agent harnesses, +not setup inference routes; their session integrations require separate runtime +and plugin setup. Gemini CLI and Antigravity are not offered as detected setup +routes. You can also sign in through the provider's own OAuth or device-pairing flow. -The built-in choices include OpenAI/ChatGPT, OpenRouter, GitHub Copilot, Google -Gemini CLI, xAI, MiniMax Global and CN, and Chutes. The list comes from the +The built-in choices include OpenAI/ChatGPT, OpenRouter, GitHub Copilot, xAI, +MiniMax Global and CN, and Chutes. Google is available through the supported AI +Studio API-key route. The list comes from the Gateway's active text-inference provider plugins rather than a fixed app list, so another provider can opt in without adding provider-specific macOS code. diff --git a/docs/start/wizard.md b/docs/start/wizard.md index 663c87980ee2..cd7c0de4ef1b 100644 --- a/docs/start/wizard.md +++ b/docs/start/wizard.md @@ -92,10 +92,9 @@ Plain `openclaw onboard` follows this path: 2. Detect configured models, API-key environment variables, supported local AI CLIs, and already installed tool-capable models from reachable Ollama or LM Studio servers on the Gateway host. This read-only pass never downloads a - model. Gemini CLI, Antigravity, Pi, and OpenCode installs are also reported - when they cannot serve as the reusable inference route for guided setup. - Gemini and Antigravity cannot enforce the tool-free probe; Pi and OpenCode - are whole-agent harnesses rather than setup inference routes. + model. Pi and OpenCode installs may also be reported for context when they + cannot serve as the reusable inference route. Gemini CLI and Antigravity are + not offered as detected setup routes. 3. Test the first detected candidate with a real completion. On failure, show the reason and continue to the next usable candidate. 4. If detection is exhausted, choose OpenAI, Anthropic, xAI (Grok), Google, or diff --git a/extensions/google/cli-backend-auth.runtime.ts b/extensions/google/cli-backend-auth.runtime.ts index 9ea86efb0b5d..80e11576d8da 100644 --- a/extensions/google/cli-backend-auth.runtime.ts +++ b/extensions/google/cli-backend-auth.runtime.ts @@ -49,6 +49,8 @@ const GEMINI_CLI_API_KEY_AUTH_ENV = [ ]; const GEMINI_CLI_PROFILE_AUTH_ENV = [...GEMINI_CLI_API_KEY_AUTH_ENV, "GEMINI_API_KEY"]; const GEMINI_CLI_PROFILE_SETTINGS_ENV = ["GEMINI_CLI_SYSTEM_SETTINGS_PATH"]; +const GEMINI_CLI_SUPPORTED_AUTH_GUIDANCE = + "Open Models settings and connect Google with an AI Studio API key, then select that profile for this model."; type GeminiAuthProfileCredential = { type: "api_key" | "oauth" | "token"; @@ -116,14 +118,14 @@ function throwUnstageableSelectedGeminiProfile( } if (!credential) { throw new Error( - "Gemini CLI auth profile was selected but no credential material was found. Re-authenticate with `openclaw models auth login --provider google-gemini-cli --force`.", + `Gemini CLI auth profile was selected but no credential material was found. ${GEMINI_CLI_SUPPORTED_AUTH_GUIDANCE}`, ); } if (credential.provider !== GEMINI_CLI_PROVIDER_ID) { throwUnsupportedGeminiCredential(credential); } throw new Error( - "Gemini CLI execution supports google-gemini-cli OAuth or API-key auth profiles. Re-authenticate with `openclaw models auth login --provider google-gemini-cli --force`.", + `Gemini CLI execution requires a Google AI Studio API-key profile or a previously configured valid Gemini CLI OAuth profile. ${GEMINI_CLI_SUPPORTED_AUTH_GUIDANCE}`, ); } @@ -149,7 +151,7 @@ function requireGeminiOAuthCredential( !Number.isFinite(credential.expires) ) { throw new Error( - "Gemini CLI OAuth profile is missing usable token material. Re-authenticate with `openclaw models auth login --provider google-gemini-cli --force`.", + `Gemini CLI OAuth profile is incomplete and cannot be repaired by OpenClaw. ${GEMINI_CLI_SUPPORTED_AUTH_GUIDANCE}`, ); } diff --git a/extensions/google/cli-backend-auth.test.ts b/extensions/google/cli-backend-auth.test.ts index 8ad549c3bb08..e68c8d5ba740 100644 --- a/extensions/google/cli-backend-auth.test.ts +++ b/extensions/google/cli-backend-auth.test.ts @@ -637,6 +637,30 @@ describe("google gemini cli backend auth bridge", () => { } }); + it("keeps expired but refreshable legacy OAuth profiles on the compatibility path", async () => { + await withTempDir("openclaw-test-workspace-", async (workspaceDir) => { + const context = buildGeminiOAuthPrepareContext(workspaceDir); + if (!context.authCredential) { + throw new Error("expected Gemini OAuth test credentials"); + } + context.authCredential.expires = Date.now() - 60_000; + + const prepared = await buildGoogleGeminiCliBackend().prepareExecution?.(context); + try { + await stageGeminiPreparedExecution(prepared); + const home = prepared?.env?.GEMINI_CLI_HOME; + const raw = await fs.readFile(path.join(home ?? "", ".gemini", "oauth_creds.json"), "utf8"); + expect(JSON.parse(raw)).toMatchObject({ + access_token: "access-token", + refresh_token: "refresh-token", + expiry_date: context.authCredential.expires, + }); + } finally { + await prepared?.cleanup?.(); + } + }); + }); + it("stages Gemini CLI JSON through same-directory atomic renames", async () => { await withTempDir("openclaw-test-workspace-", async (workspaceDir) => { const backend = buildGoogleGeminiCliBackend(); @@ -851,7 +875,7 @@ describe("google gemini cli backend auth bridge", () => { token: "bearer-token", }, } as never), - ).rejects.toThrow(/OAuth or API-key auth profiles/); + ).rejects.toThrow(/Google AI Studio API-key profile/); } finally { await fs.rm(workspaceDir, { recursive: true, force: true }); } @@ -870,7 +894,33 @@ describe("google gemini cli backend auth bridge", () => { modelId: "gemini-3.1-flash-lite", authProfileId: "google-gemini-cli:missing", } as never), - ).rejects.toThrow(/no credential material/); + ).rejects.toThrow(/Open Models settings and connect Google with an AI Studio API key/); + } finally { + await fs.rm(workspaceDir, { recursive: true, force: true }); + } + }); + + it("routes incomplete legacy Gemini OAuth profiles to supported Google setup", async () => { + const backend = buildGoogleGeminiCliBackend(); + const workspaceDir = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-test-workspace-")); + + try { + await expect( + backend.prepareExecution?.({ + workspaceDir, + agentDir: path.join(workspaceDir, "agent"), + provider: "google-gemini-cli", + modelId: "gemini-3.1-flash-lite", + authProfileId: "google-gemini-cli:legacy", + authCredential: { + type: "oauth", + provider: "google-gemini-cli", + access: "expired-access-token", + }, + } as never), + ).rejects.toThrow( + /OAuth profile is incomplete and cannot be repaired by OpenClaw.*AI Studio API key/, + ); } finally { await fs.rm(workspaceDir, { recursive: true, force: true }); } diff --git a/extensions/google/gemini-cli-provider.ts b/extensions/google/gemini-cli-provider.ts index 534557a63644..f21d605e5243 100644 --- a/extensions/google/gemini-cli-provider.ts +++ b/extensions/google/gemini-cli-provider.ts @@ -2,10 +2,8 @@ import { createLazyRuntimeModule } from "openclaw/plugin-sdk/lazy-runtime"; // Google provider module implements model/runtime integration. import type { OpenClawPluginApi, - ProviderAuthContext, ProviderFetchUsageSnapshotContext, } from "openclaw/plugin-sdk/plugin-entry"; -import { buildOauthProviderAuthResult } from "openclaw/plugin-sdk/provider-auth-result"; import type { ProviderPlugin } from "openclaw/plugin-sdk/provider-model-shared"; import { fetchGeminiUsage } from "openclaw/plugin-sdk/provider-usage"; import { GOOGLE_GEMINI_CLI_PROVIDER_ID } from "./gemini-cli-auth-home.js"; @@ -14,14 +12,7 @@ import { GOOGLE_GEMINI_PROVIDER_HOOKS } from "./provider-hooks.js"; import { isModernGoogleModel, resolveGoogleGeminiForwardCompatModel } from "./provider-models.js"; const PROVIDER_ID = GOOGLE_GEMINI_CLI_PROVIDER_ID; -const PROVIDER_LABEL = "Gemini CLI OAuth"; -const DEFAULT_MODEL = "google/gemini-3.1-pro-preview"; -const ENV_VARS = [ - "OPENCLAW_GEMINI_OAUTH_CLIENT_ID", - "OPENCLAW_GEMINI_OAUTH_CLIENT_SECRET", - "GEMINI_CLI_OAUTH_CLIENT_ID", - "GEMINI_CLI_OAUTH_CLIENT_SECRET", -] as const; +const PROVIDER_LABEL = "Gemini CLI runtime"; const loadOauthRuntimeModule = createLazyRuntimeModule(() => import("./oauth.runtime.js")); @@ -35,90 +26,8 @@ export function buildGoogleGeminiCliProvider(): ProviderPlugin { label: PROVIDER_LABEL, docsPath: "/providers/models", aliases: ["gemini-cli"], - envVars: [...ENV_VARS], - auth: [ - { - id: "oauth", - label: "Google OAuth", - hint: "PKCE + localhost callback", - kind: "oauth", - run: async (ctx: ProviderAuthContext) => { - await ctx.prompter.note( - [ - "This is an unofficial integration and is not endorsed by Google.", - "Some users have reported account restrictions or suspensions after using third-party Gemini CLI and Antigravity OAuth clients.", - "Proceed only if you understand and accept this risk.", - ].join("\n"), - "Google Gemini CLI caution", - ); - - const proceed = await ctx.prompter.confirm({ - message: "Continue with Google Gemini CLI OAuth?", - initialValue: false, - }); - if (!proceed) { - await ctx.prompter.note("Skipped Google Gemini CLI OAuth setup.", "Setup skipped"); - return { profiles: [] }; - } - - const spin = ctx.prompter.progress("Starting Gemini CLI OAuth…"); - try { - const { loginGeminiCliOAuth } = await loadOauthRuntimeModule(); - const result = await loginGeminiCliOAuth({ - isRemote: ctx.isRemote, - openUrl: ctx.openUrl, - log: (msg) => ctx.runtime.log(msg), - note: (message, title) => ctx.prompter.note(message, title), - prompt: async (message) => ctx.prompter.text({ message }), - progress: spin, - ...(ctx.signal ? { signal: ctx.signal } : {}), - }); - - spin.stop("Gemini CLI OAuth complete"); - return buildOauthProviderAuthResult({ - providerId: PROVIDER_ID, - defaultModel: DEFAULT_MODEL, - access: result.access, - refresh: result.refresh, - expires: result.expires, - email: result.email, - configPatch: { - agents: { - defaults: { - models: { - [DEFAULT_MODEL]: { agentRuntime: { id: PROVIDER_ID } }, - }, - }, - }, - }, - ...(result.projectId ? { credentialExtra: { projectId: result.projectId } } : {}), - ...(result.projectId - ? { - notes: [ - "If requests fail, set GOOGLE_CLOUD_PROJECT or GOOGLE_CLOUD_PROJECT_ID.", - ], - } - : {}), - }); - } catch (err) { - spin.stop("Gemini CLI OAuth failed"); - await ctx.prompter.note( - "Trouble with OAuth? Ensure your Google account has Gemini CLI access.", - "OAuth help", - ); - throw err; - } - }, - }, - ], - wizard: { - setup: { - choiceId: "google-gemini-cli", - choiceLabel: "Gemini CLI OAuth", - choiceHint: "Sign in with your Google account (opens a browser)", - methodId: "oauth", - }, - }, + envVars: [], + auth: [], resolveDynamicModel: (ctx) => resolveGoogleGeminiForwardCompatModel({ providerId: PROVIDER_ID, diff --git a/extensions/google/index.test.ts b/extensions/google/index.test.ts index 42127b672bf2..88ff7d28b5e9 100644 --- a/extensions/google/index.test.ts +++ b/extensions/google/index.test.ts @@ -129,6 +129,21 @@ describe("google provider plugin hooks", () => { ).toBe("tagged"); }); + it("keeps the Gemini CLI runtime without offering new OAuth setup", async () => { + const { providers } = await registerProviderPlugin({ + plugin: googleProviderPlugin, + id: "google", + name: "Google Provider", + }); + const cliProvider = requireRegisteredProvider(providers, "google-gemini-cli"); + + expect(cliProvider.label).toBe("Gemini CLI runtime"); + expect(cliProvider.auth).toEqual([]); + expect(cliProvider.envVars).toEqual([]); + expect(cliProvider.wizard).toBeUndefined(); + expect(cliProvider.refreshOAuth).toBeTypeOf("function"); + }); + it("keeps google-antigravity hook aliases on tagged reasoning mode", async () => { const { providers } = await registerProviderPlugin({ plugin: googleProviderPlugin, diff --git a/extensions/google/manifest.test.ts b/extensions/google/manifest.test.ts index d19680a8dd91..66aefda656bd 100644 --- a/extensions/google/manifest.test.ts +++ b/extensions/google/manifest.test.ts @@ -3,6 +3,13 @@ import { readFileSync } from "node:fs"; import { describe, expect, it } from "vitest"; type GoogleManifest = { + providerAuthChoices?: Array<{ + provider?: string; + method?: string; + choiceLabel?: string; + choiceHint?: string; + groupHint?: string; + }>; modelIdNormalization?: { providers?: Record< string, @@ -67,6 +74,21 @@ function loadManifest(): GoogleManifest { } describe("google manifest model catalog", () => { + it("offers Google AI Studio API keys without consumer CLI OAuth", () => { + const choices = loadManifest().providerAuthChoices ?? []; + + expect(choices).toEqual([ + expect.objectContaining({ + provider: "google", + method: "api-key", + choiceLabel: "Google AI Studio API key", + choiceHint: "Supported API-key access from aistudio.google.com/apikey", + groupHint: "Supported API-key setup", + }), + ]); + expect(choices.some((choice) => choice.provider === "google-gemini-cli")).toBe(false); + }); + it("suppresses retired Gemini chat model identifiers for all Google chat providers", () => { const manifest = loadManifest(); const suppressionRefs = new Set( diff --git a/extensions/google/openclaw.plugin.json b/extensions/google/openclaw.plugin.json index 2cb4016a34ba..9f89351d32db 100644 --- a/extensions/google/openclaw.plugin.json +++ b/extensions/google/openclaw.plugin.json @@ -692,28 +692,16 @@ "method": "api-key", "choiceId": "gemini-api-key", "appGuidedSecret": true, - "choiceLabel": "Google Gemini API key", - "choiceHint": "Free API key from aistudio.google.com/apikey", + "choiceLabel": "Google AI Studio API key", + "choiceHint": "Supported API-key access from aistudio.google.com/apikey", "groupId": "google", "groupLabel": "Google", - "groupHint": "Gemini API key + OAuth", + "groupHint": "Supported API-key setup", "onboardingFeatured": true, "optionKey": "geminiApiKey", "cliFlag": "--gemini-api-key", "cliOption": "--gemini-api-key ", "cliDescription": "Gemini API key" - }, - { - "provider": "google-gemini-cli", - "method": "oauth", - "choiceId": "google-gemini-cli", - "appGuidedAuth": "oauth", - "choiceLabel": "Gemini CLI OAuth", - "choiceHint": "Sign in with your Google account (opens a browser)", - "groupId": "google", - "groupLabel": "Google", - "groupHint": "Gemini API key + OAuth", - "onboardingFeatured": true } ], "uiHints": { diff --git a/extensions/google/provider-contract-api.test.ts b/extensions/google/provider-contract-api.test.ts new file mode 100644 index 000000000000..f65dfa0b7612 --- /dev/null +++ b/extensions/google/provider-contract-api.test.ts @@ -0,0 +1,29 @@ +import { describe, expect, it } from "vitest"; +import { createGoogleGeminiCliProvider, createGoogleProvider } from "./provider-contract-api.js"; + +describe("google provider contract", () => { + it("exposes Google AI Studio API-key setup", () => { + const provider = createGoogleProvider(); + + expect(provider.auth).toEqual([ + expect.objectContaining({ + id: "api-key", + label: "Google AI Studio API key", + hint: "Supported API-key access from aistudio.google.com/apikey", + wizard: expect.objectContaining({ + choiceLabel: "Google AI Studio API key", + groupHint: "Supported API-key setup", + }), + }), + ]); + }); + + it("keeps Gemini CLI as a runtime-only compatibility provider", () => { + const provider = createGoogleGeminiCliProvider(); + + expect(provider.label).toBe("Gemini CLI runtime"); + expect(provider.auth).toEqual([]); + expect(provider.envVars).toEqual([]); + expect(provider.wizard).toBeUndefined(); + }); +}); diff --git a/extensions/google/provider-contract-api.ts b/extensions/google/provider-contract-api.ts index 9fc246ece0ed..e101e0bf79d2 100644 --- a/extensions/google/provider-contract-api.ts +++ b/extensions/google/provider-contract-api.ts @@ -14,15 +14,15 @@ export function createGoogleProvider(): ProviderPlugin { { id: "api-key", kind: "api_key", - label: "Google Gemini API key", - hint: "Free API key from aistudio.google.com/apikey", + label: "Google AI Studio API key", + hint: "Supported API-key access from aistudio.google.com/apikey", run: noopAuth, wizard: { choiceId: "gemini-api-key", - choiceLabel: "Google Gemini API key", + choiceLabel: "Google AI Studio API key", groupId: "google", groupLabel: "Google", - groupHint: "Gemini API key + OAuth", + groupHint: "Supported API-key setup", }, }, ], @@ -48,31 +48,10 @@ export function createGoogleVertexProvider(): ProviderPlugin { export function createGoogleGeminiCliProvider(): ProviderPlugin { return { id: "google-gemini-cli", - label: "Gemini CLI OAuth", + label: "Gemini CLI runtime", docsPath: "/providers/models", aliases: ["gemini-cli"], - envVars: [ - "OPENCLAW_GEMINI_OAUTH_CLIENT_ID", - "OPENCLAW_GEMINI_OAUTH_CLIENT_SECRET", - "GEMINI_CLI_OAUTH_CLIENT_ID", - "GEMINI_CLI_OAUTH_CLIENT_SECRET", - ], - auth: [ - { - id: "oauth", - kind: "oauth", - label: "Google OAuth", - hint: "PKCE + localhost callback", - run: noopAuth, - }, - ], - wizard: { - setup: { - choiceId: "google-gemini-cli", - choiceLabel: "Gemini CLI OAuth", - choiceHint: "Sign in with your Google account (opens a browser)", - methodId: "oauth", - }, - }, + envVars: [], + auth: [], }; } diff --git a/extensions/google/provider-registration.ts b/extensions/google/provider-registration.ts index cee00bfbf13b..de15ca332aa2 100644 --- a/extensions/google/provider-registration.ts +++ b/extensions/google/provider-registration.ts @@ -48,21 +48,21 @@ export function buildGoogleProvider(): ProviderPlugin { createProviderApiKeyAuthMethod({ providerId: "google", methodId: "api-key", - label: "Google Gemini API key", - hint: "Free API key from aistudio.google.com/apikey", + label: "Google AI Studio API key", + hint: "Supported API-key access from aistudio.google.com/apikey", optionKey: "geminiApiKey", flagName: "--gemini-api-key", envVar: "GEMINI_API_KEY", - promptMessage: "Enter Gemini API key", + promptMessage: "Enter Google AI Studio API key", defaultModel: GOOGLE_GEMINI_DEFAULT_MODEL, expectedProviders: ["google"], applyConfig: (cfg) => applyGoogleGeminiModelDefault(cfg).next, wizard: { choiceId: "gemini-api-key", - choiceLabel: "Google Gemini API key", + choiceLabel: "Google AI Studio API key", groupId: "google", groupLabel: "Google", - groupHint: "Gemini API key + OAuth", + groupHint: "Supported API-key setup", }, }), ], diff --git a/extensions/llama-cpp/src/inference-provider.test.ts b/extensions/llama-cpp/src/inference-provider.test.ts index 8d41060a5829..c18d027fd74d 100644 --- a/extensions/llama-cpp/src/inference-provider.test.ts +++ b/extensions/llama-cpp/src/inference-provider.test.ts @@ -364,6 +364,100 @@ describe("llama.cpp inference provider", () => { expect(mocks.llama.createGrammarForJsonSchema).not.toHaveBeenCalled(); }); + it.each([ + { + format: "Harmony", + text: '<|channel|>commentary to=weather code<|message|>{"city":"Paris"}<|call|>', + }, + { + format: "bracketed", + text: '[weather]\n{"city":"Paris"}\n[END_TOOL_REQUEST]', + }, + ])("promotes $format plaintext tool calls into native tool events", async ({ text }) => { + mocks.generateResponse.mockImplementationOnce(async (_history, options) => { + options.onTextChunk(text.slice(0, 12)); + options.onTextChunk(text.slice(12)); + return { + response: text, + functionCalls: undefined, + metadata: { stopReason: "eogToken" }, + }; + }); + + const stream = await createLlamaCppStreamFn({})(model, { + messages: [{ role: "user", content: "Weather?", timestamp: 1 }], + tools: [ + { + name: "weather", + description: "Get weather", + parameters: { type: "object", properties: { city: { type: "string" } } }, + }, + ], + }); + + const events = await collectEvents(stream); + + expect(events.map((event) => event.type)).toEqual([ + "start", + "toolcall_start", + "toolcall_delta", + "toolcall_end", + "done", + ]); + expect(events.at(-1)).toMatchObject({ + type: "done", + reason: "toolUse", + message: { + stopReason: "toolUse", + content: [ + { + type: "toolCall", + name: "weather", + arguments: { city: "Paris" }, + }, + ], + }, + }); + }); + + it("preserves plaintext calls for tools that are not registered", async () => { + const text = '[tool:calendar] {"city":"Paris"}'; + mocks.generateResponse.mockImplementationOnce(async (_history, options) => { + options.onTextChunk(text); + return { + response: text, + functionCalls: undefined, + metadata: { stopReason: "eogToken" }, + }; + }); + + const stream = await createLlamaCppStreamFn({})(model, { + messages: [{ role: "user", content: "Weather?", timestamp: 1 }], + tools: [ + { + name: "weather", + description: "Get weather", + parameters: { type: "object", properties: { city: { type: "string" } } }, + }, + ], + }); + + const events = await collectEvents(stream); + + expect(events.map((event) => event.type)).toEqual([ + "start", + "text_start", + "text_delta", + "text_end", + "done", + ]); + expect(events.at(-1)).toMatchObject({ + type: "done", + reason: "stop", + message: { content: [{ type: "text", text }] }, + }); + }); + it("lets tools win when responseFormat is also present", async () => { const stream = await createLlamaCppStreamFn({})( model, diff --git a/extensions/llama-cpp/src/inference-provider.ts b/extensions/llama-cpp/src/inference-provider.ts index e163f645c94a..0de0499bf706 100644 --- a/extensions/llama-cpp/src/inference-provider.ts +++ b/extensions/llama-cpp/src/inference-provider.ts @@ -17,6 +17,7 @@ import type { } from "openclaw/plugin-sdk/llm"; import { createAssistantMessageEventStream } from "openclaw/plugin-sdk/llm"; import type { ModelProviderConfig } from "openclaw/plugin-sdk/provider-model-shared"; +import { createPlainTextToolCallCompatWrapper } from "openclaw/plugin-sdk/provider-stream-shared"; import { DEFAULT_LLAMA_CPP_CONTEXT_SIZE, resolveLlamaCppModelCacheDir, @@ -293,7 +294,7 @@ async function clearLlamaCppInferenceCacheForTests(): Promise { } export function createLlamaCppStreamFn(params: { providerConfig?: ModelProviderConfig }): StreamFn { - return (model, context, options) => { + return createPlainTextToolCallCompatWrapper((model, context, options) => { const stream = createAssistantMessageEventStream(); let streamedText = ""; let generationAborted = false; @@ -453,7 +454,7 @@ export function createLlamaCppStreamFn(params: { providerConfig?: ModelProviderC queueMicrotask(() => void serialize(run)); } return stream; - }; + }); } if (process.env.VITEST || process.env.NODE_ENV === "test") { diff --git a/extensions/ollama/index.test.ts b/extensions/ollama/index.test.ts index 46d01c68de80..5e43f2cb00a2 100644 --- a/extensions/ollama/index.test.ts +++ b/extensions/ollama/index.test.ts @@ -1901,7 +1901,7 @@ describe("ollama plugin", () => { expect(rows).toHaveLength(8); }); - it("keeps unknown requested Ollama models unresolved when show has no metadata", async () => { + it("keeps unknown requested Ollama models unresolved when show inspection fails", async () => { const provider = registerProvider(); const previous = process.env.OLLAMA_API_KEY; process.env.OLLAMA_API_KEY = "ollama-local"; @@ -1910,7 +1910,7 @@ describe("ollama plugin", () => { api: "ollama", models: [], }); - queryOllamaModelShowInfoMock.mockResolvedValueOnce({}); + queryOllamaModelShowInfoMock.mockResolvedValueOnce({ showInspectionFailed: true }); try { await provider.prepareDynamicModel?.({ diff --git a/extensions/ollama/provider-discovery.test.ts b/extensions/ollama/provider-discovery.test.ts index 83dd2c74fb9b..c8ba8eb0134c 100644 --- a/extensions/ollama/provider-discovery.test.ts +++ b/extensions/ollama/provider-discovery.test.ts @@ -323,7 +323,7 @@ describe("Ollama provider", () => { const fetchMock = vi.fn(async (input: unknown) => { const url = String(input); if (url.endsWith("/api/tags")) { - return tagsResponse(["qwen3:32b"]); + return tagsResponse(["deepseek-r1:14b"]); } if (url.endsWith("/api/show")) { return jsonResponse({}, 500); @@ -335,8 +335,10 @@ describe("Ollama provider", () => { const provider = await runOllamaCatalog({ env: { OLLAMA_API_KEY: "test-key", VITEST: "", NODE_ENV: "development" }, }); - const model = provider?.models?.find((entry) => entry.id === "qwen3:32b"); + const model = provider?.models?.find((entry) => entry.id === "deepseek-r1:14b"); expect(model?.contextWindow).toBe(128000); + expect(model?.compat?.supportsTools).toBe(false); + expect(model?.reasoning).toBe(true); expectDiscoveryCallCounts(fetchMock, { tags: 1, show: 1 }); }); diff --git a/extensions/ollama/src/cjk-char-estimate.ts b/extensions/ollama/src/cjk-char-estimate.ts new file mode 100644 index 000000000000..db04e2267ad2 --- /dev/null +++ b/extensions/ollama/src/cjk-char-estimate.ts @@ -0,0 +1,47 @@ +/** + * CJK-aware character weighting for Ollama usage fallback estimates. + * + * This stays plugin-private because exposing it through the Plugin SDK would + * create a stable public contract for one provider-specific fallback. Keep + * the weighting aligned with normalization-core's CJK budget heuristic. + */ + +const CHARS_PER_TOKEN_ESTIMATE = 4; + +const NON_ASCII_RE = /[\u0080-\u{10FFFF}]/u; +const COMMON_CJK_RE = /[\u00B7\u3000-\u319F\u4E00-\u9FA5\uAC00-\uD7AF\uFF01-\uFF60]/gu; +const RARE_BMP_CJK_RE = + /[\u1100-\u11FF\u2E80-\u2FFF\u31A0-\u4DFF\u9FA6-\u9FFF\uA000-\uA4FF\uA700-\uA707\uA960-\uA97F\uD7B0-\uD7FF\uF900-\uFAFF]/gu; +const TWO_TOKEN_CJK_RE = + /[\u{02C7}\u{02C9}-\u{02CB}\u{02D9}\u{02EA}-\u{02EB}\uFE10-\uFE4F\uFF61-\uFFDC\uFFE0-\uFFE6]|\u{0305}|\u{0323}/gu; +const THREE_TOKEN_SUPPLEMENTARY_CJK_RE = /[\u{1D360}-\u{1D371}]/gu; +const SUPPLEMENTARY_CJK_RE = + /[\u{16FE0}-\u{16FFF}\u{1AFF0}-\u{1AFFF}\u{1B000}-\u{1B16F}\u{1F200}-\u{1F2FF}\u{20000}-\u{2FA1F}\u{30000}-\u{3347F}]/gu; +const SPECIAL_CJK_RE = + /[\u{02C7}\u{02C9}-\u{02CB}\u{02D9}\u{02EA}-\u{02EB}\u1100-\u11FF\u2E80-\u2FFF\u31A0-\u4DFF\u9FA6-\u9FFF\uA000-\uA4FF\uA700-\uA707\uA960-\uA97F\uD7B0-\uD7FF\uF900-\uFAFF\uFE10-\uFE4F\uFF61-\uFFDC\uFFE0-\uFFE6\u{16FE0}-\u{16FFF}\u{1AFF0}-\u{1AFFF}\u{1B000}-\u{1B16F}\u{1D360}-\u{1D371}\u{1F200}-\u{1F2FF}\u{20000}-\u{2FA1F}\u{30000}-\u{3347F}]|\u{0305}|\u{0323}/u; + +function countMatches(text: string, pattern: RegExp): number { + return (text.match(pattern) ?? []).length; +} + +export function estimateStringChars(text: string): number { + if (!NON_ASCII_RE.test(text)) { + return text.length; + } + const commonCjkCount = countMatches(text, COMMON_CJK_RE); + const commonEstimate = text.length + commonCjkCount * (CHARS_PER_TOKEN_ESTIMATE - 1); + if (!SPECIAL_CJK_RE.test(text)) { + return commonEstimate; + } + const rareBmpCjkCount = countMatches(text, RARE_BMP_CJK_RE); + const twoTokenCjkCount = countMatches(text, TWO_TOKEN_CJK_RE); + const threeTokenSupplementaryCjkCount = countMatches(text, THREE_TOKEN_SUPPLEMENTARY_CJK_RE); + const supplementaryCjkCount = countMatches(text, SUPPLEMENTARY_CJK_RE); + return ( + commonEstimate + + rareBmpCjkCount * (CHARS_PER_TOKEN_ESTIMATE * 3 - 1) + + twoTokenCjkCount * (CHARS_PER_TOKEN_ESTIMATE * 2 - 1) + + threeTokenSupplementaryCjkCount * (CHARS_PER_TOKEN_ESTIMATE * 3 - 2) + + supplementaryCjkCount * (CHARS_PER_TOKEN_ESTIMATE * 4 - 2) + ); +} diff --git a/extensions/ollama/src/provider-models.test.ts b/extensions/ollama/src/provider-models.test.ts index c2ae83a9b861..3be6e9e3d436 100644 --- a/extensions/ollama/src/provider-models.test.ts +++ b/extensions/ollama/src/provider-models.test.ts @@ -471,6 +471,27 @@ describe("ollama provider models", () => { expect(model.compat?.supportsUsageInStreaming).toBe(true); }); + it("keeps failed inspection distinct from omitted and empty capabilities", () => { + const uninspected = buildOllamaModelDefinition("deepseek-r1:14b", 65536); + const authoritativeEmpty = buildOllamaModelDefinition("deepseek-r1:14b", 65536, []); + const inspectionFailed = buildOllamaModelDefinition("deepseek-r1:14b", 65536, undefined, { + showInspectionFailed: true, + }); + + expect(uninspected).toMatchObject({ + reasoning: true, + compat: { supportsTools: true }, + }); + expect(authoritativeEmpty).toMatchObject({ + reasoning: false, + compat: { supportsTools: false }, + }); + expect(inspectionFailed).toMatchObject({ + reasoning: true, + compat: { supportsTools: false }, + }); + }); + it.each([ { parameters: "num_ctx 8192\nnum_ctx 32768", expected: 32768 }, { parameters: "temperature 0.8\nnum_ctx -1\nnum_ctx 0", expected: undefined }, @@ -506,9 +527,9 @@ describe("ollama provider models", () => { vi.fn(async () => showResponse.response), ); - await expect(queryOllamaModelShowInfo("http://127.0.0.1:11434", "llama3:8b")).resolves.toEqual( - {}, - ); + await expect(queryOllamaModelShowInfo("http://127.0.0.1:11434", "llama3:8b")).resolves.toEqual({ + showInspectionFailed: true, + }); expect(showResponse.wasCanceled()).toBe(true); }); @@ -609,7 +630,9 @@ describe("ollama provider models", () => { }); await waitForSocketClose("/api/tags"); - await expect(queryOllamaModelShowInfo(baseUrl, "llama3:8b")).resolves.toEqual({}); + await expect(queryOllamaModelShowInfo(baseUrl, "llama3:8b")).resolves.toEqual({ + showInspectionFailed: true, + }); await waitForSocketClose("/api/show"); mode = "success"; @@ -639,6 +662,50 @@ describe("ollama provider models", () => { } }); + it("keeps tools off after a live /api/show failure", async () => { + const server = createServer((request, response) => { + response.setHeader("Content-Type", "application/json"); + if (request.url === "/api/tags") { + response.end( + JSON.stringify({ + models: [{ name: "deepseek-r1:14b", digest: "sha256:show-failure" }], + }), + ); + return; + } + if (request.url === "/api/show") { + response.statusCode = 500; + response.end(JSON.stringify({ error: "show failed" })); + return; + } + response.statusCode = 404; + response.end(JSON.stringify({ error: "not found" })); + }); + + const listening = once(server, "listening"); + try { + server.listen(0, "127.0.0.1"); + await listening; + const address = server.address(); + if (!address || typeof address === "string") { + throw new Error("Ollama test server did not expose a TCP address"); + } + + const provider = await buildOllamaProvider(`http://127.0.0.1:${address.port}`); + const model = expectDefined(provider.models?.[0], "show-failed Ollama model"); + + expect(model.id).toBe("deepseek-r1:14b"); + expect(model.compat?.supportsTools).toBe(false); + expect(model.reasoning).toBe(true); + } finally { + if (server.listening) { + await new Promise((resolve, reject) => { + server.close((error) => (error ? reject(error) : resolve())); + }); + } + } + }); + it("fails soft and stops reading when discovery streams exceed the JSON byte cap", async () => { // Larger than the shared 16 MiB readProviderJsonResponse cap so the bounded reader cancels // the stream mid-flight; if the cap were removed the reader would buffer the whole payload. @@ -687,7 +754,7 @@ describe("ollama provider models", () => { vi.fn(async () => makeOversizedJsonResponse()), ); const showInfo = await queryOllamaModelShowInfo("http://127.0.0.1:11434", "evil-model:latest"); - expect(showInfo).toEqual({}); + expect(showInfo).toEqual({ showInspectionFailed: true }); expect(canceled).toBe(true); expect(bytesPulled).toBeLessThan(TOTAL_CHUNKS * ONE_MIB); }); diff --git a/extensions/ollama/src/provider-models.ts b/extensions/ollama/src/provider-models.ts index c9cc09871438..b282c56ac214 100644 --- a/extensions/ollama/src/provider-models.ts +++ b/extensions/ollama/src/provider-models.ts @@ -37,6 +37,7 @@ export type OllamaTagsResponse = { export type OllamaModelWithContext = OllamaTagModel & { contextWindow?: number; capabilities?: string[]; + showInspectionFailed?: boolean; }; const OLLAMA_SHOW_CONCURRENCY = 8; @@ -81,8 +82,14 @@ export function resolveOllamaApiBase(configuredBaseUrl?: string): string { export type OllamaModelShowInfo = { contextWindow?: number; capabilities?: string[]; + /** Distinguishes a failed request from a successful response that omitted capabilities. */ + showInspectionFailed?: boolean; }; +const OLLAMA_FAILED_SHOW_INFO: OllamaModelShowInfo = Object.freeze({ + showInspectionFailed: true, +}); + type OllamaModelRequestOptions = { apiKey?: string; timeoutMs?: number; @@ -227,7 +234,7 @@ export async function queryOllamaModelShowInfo( return await readOllamaModelShowInfo(apiBase, modelName, opts); } catch { throwIfOllamaRequestAborted(opts?.signal); - return {}; + return OLLAMA_FAILED_SHOW_INFO; } } @@ -279,10 +286,7 @@ export async function enrichOllamaModelsWithContext( const batchResults = await Promise.all( batch.map(async (model) => { const showInfo = await queryOllamaModelShowInfoCached(apiBase, model, opts); - return Object.assign({}, model, { - contextWindow: showInfo.contextWindow, - capabilities: showInfo.capabilities, - }); + return Object.assign({}, model, showInfo); }), ); enriched.push(...batchResults); @@ -343,6 +347,7 @@ export function buildOllamaModelDefinition( modelId: string, contextWindow?: number, capabilities?: string[], + opts?: { showInspectionFailed?: boolean }, ): ModelDefinitionConfig { const hasVision = capabilities?.includes("vision") ?? false; const input: ("text" | "image")[] = hasVision ? ["text", "image"] : ["text"]; @@ -352,7 +357,8 @@ export function buildOllamaModelDefinition( ? isReasoningModelHeuristic(modelId) : capabilities.includes("thinking")); const compat = { - supportsTools: capabilities?.includes("tools") ?? true, + supportsTools: + opts?.showInspectionFailed === true ? false : (capabilities?.includes("tools") ?? true), supportsUsageInStreaming: true, supportsJsonSchemaResponseFormat: !isOllamaCloudModel(modelId), }; @@ -467,7 +473,9 @@ export async function buildOllamaProvider( baseUrl: apiBase, api: "ollama", models: discovered.map((model) => - buildOllamaModelDefinition(model.name, model.contextWindow, model.capabilities), + buildOllamaModelDefinition(model.name, model.contextWindow, model.capabilities, { + showInspectionFailed: model.showInspectionFailed, + }), ), }; } diff --git a/extensions/ollama/src/setup-body-release.test.ts b/extensions/ollama/src/setup-body-release.test.ts new file mode 100644 index 000000000000..73077c06e27f --- /dev/null +++ b/extensions/ollama/src/setup-body-release.test.ts @@ -0,0 +1,210 @@ +import { once } from "node:events"; +import { createServer } from "node:http"; +import type { Socket } from "node:net"; +import type { WizardPrompter } from "openclaw/plugin-sdk/setup"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { pullOllamaModel } from "./setup-pull.js"; +import { checkOllamaCloudAuth } from "./setup.js"; + +const fetchWithSsrFGuardMock = vi.hoisted(() => vi.fn()); + +vi.mock("openclaw/plugin-sdk/ssrf-runtime", async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + fetchWithSsrFGuard: fetchWithSsrFGuardMock, + }; +}); + +function cancelTrackedResponse( + text: string, + init: ResponseInit, +): { + response: Response; + wasCanceled: () => boolean; +} { + let canceled = false; + const body = new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode(text)); + }, + cancel() { + canceled = true; + }, + }); + return { + response: new Response(body, init), + wasCanceled: () => canceled, + }; +} + +function createPullPrompter(): WizardPrompter { + return { + progress: vi.fn(() => ({ update: vi.fn(), stop: vi.fn() })), + } as unknown as WizardPrompter; +} + +async function waitForSocketClose(closed: Promise | undefined): Promise { + if (!closed) { + throw new Error("Ollama test server did not receive a request"); + } + let timeout: ReturnType | undefined; + try { + await Promise.race([ + closed, + new Promise((_resolve, reject) => { + timeout = setTimeout(() => { + reject(new Error("Ollama response socket was not closed")); + }, 2_000); + }), + ]); + } finally { + if (timeout !== undefined) { + clearTimeout(timeout); + } + } +} + +describe("Ollama setup response cleanup", () => { + afterEach(() => { + fetchWithSsrFGuardMock.mockReset(); + }); + + it.each([200, 503])("cancels the /api/me body for HTTP %s", async (status) => { + const tracked = cancelTrackedResponse('{"status":"unused"}\n', { status }); + const release = vi.fn(async () => {}); + fetchWithSsrFGuardMock.mockResolvedValueOnce({ + response: tracked.response, + finalUrl: "https://ollama.com/api/me", + release, + }); + + await checkOllamaCloudAuth("https://ollama.com"); + + expect(tracked.wasCanceled()).toBe(true); + expect(release).toHaveBeenCalledOnce(); + }); + + it.each([ + { + name: "non-OK /api/pull response", + response: () => cancelTrackedResponse("ollama unavailable", { status: 503 }), + }, + { + name: "streamed /api/pull error", + response: () => cancelTrackedResponse('{"error":"disk full"}\n', { status: 200 }), + }, + ])("cancels a $name body before returning", async ({ response: createResponse }) => { + const tracked = createResponse(); + const release = vi.fn(async () => {}); + fetchWithSsrFGuardMock.mockResolvedValueOnce({ + response: tracked.response, + finalUrl: "http://127.0.0.1:11434/api/pull", + release, + }); + + await expect( + pullOllamaModel("http://127.0.0.1:11434", "gemma4:e2b", createPullPrompter()), + ).resolves.toBe(false); + + expect(tracked.wasCanceled()).toBe(true); + expect(release).toHaveBeenCalledOnce(); + }); + + it.each([ + { + name: "successful auth probe", + path: "/api/me", + status: 200, + body: '{"status":"unused"}\n', + run: async (baseUrl: string) => { + await checkOllamaCloudAuth(baseUrl); + }, + }, + { + name: "failed auth probe", + path: "/api/me", + status: 503, + body: "ollama unavailable", + run: async (baseUrl: string) => { + await checkOllamaCloudAuth(baseUrl); + }, + }, + { + name: "failed pull response", + path: "/api/pull", + status: 503, + body: "ollama unavailable", + run: async (baseUrl: string) => { + await pullOllamaModel(baseUrl, "gemma4:e2b", createPullPrompter()); + }, + }, + { + name: "streamed pull error", + path: "/api/pull", + status: 200, + body: '{"error":"disk full"}\n', + run: async (baseUrl: string) => { + await pullOllamaModel(baseUrl, "gemma4:e2b", createPullPrompter()); + }, + }, + ])("closes the real socket after a $name", async ({ path, status, body, run }) => { + const sockets = new Set(); + let requestSocketClosed: Promise | undefined; + const server = createServer((request, response) => { + if (request.url !== path) { + response.writeHead(404); + response.end(); + return; + } + requestSocketClosed = new Promise((resolve) => { + request.socket.once("close", () => resolve()); + }); + response.writeHead(status, { "content-type": "application/json" }); + response.write(body); + }); + server.on("connection", (socket) => { + sockets.add(socket); + socket.once("close", () => sockets.delete(socket)); + }); + + fetchWithSsrFGuardMock.mockImplementation( + async (params: { url: string; init?: RequestInit; signal?: AbortSignal }) => ({ + response: await globalThis.fetch(params.url, { + ...params.init, + ...(params.signal ? { signal: params.signal } : {}), + }), + finalUrl: params.url, + release: async () => {}, + }), + ); + + const listening = once(server, "listening"); + try { + server.listen(0, "127.0.0.1"); + await listening; + const address = server.address(); + if (!address || typeof address === "string") { + throw new Error("Ollama test server did not expose a TCP address"); + } + + await run(`http://127.0.0.1:${address.port}`); + await waitForSocketClose(requestSocketClosed); + } finally { + for (const socket of sockets) { + socket.destroy(); + } + if (server.listening) { + await new Promise((resolve, reject) => { + server.close((error) => { + if (error) { + reject(error); + return; + } + resolve(); + }); + }); + } + } + }); +}); diff --git a/extensions/ollama/src/setup-model-selection.test.ts b/extensions/ollama/src/setup-model-selection.test.ts index 491c393d5dab..67107cea5c4c 100644 --- a/extensions/ollama/src/setup-model-selection.test.ts +++ b/extensions/ollama/src/setup-model-selection.test.ts @@ -23,11 +23,12 @@ describe("Ollama onboarding model selection", () => { it("keeps failed model inspections distinct from uninspected models", () => { const models = buildOllamaModelsConfig( - ["broken", "uninspected"], - new Map([["broken", { name: "broken", capabilities: [] }]]), + ["deepseek-r1:14b", "uninspected"], + new Map([["deepseek-r1:14b", { name: "deepseek-r1:14b", showInspectionFailed: true }]]), ); expect(models[0]?.compat?.supportsTools).toBe(false); + expect(models[0]?.reasoning).toBe(true); expect(models[1]?.compat?.supportsTools).toBe(true); }); diff --git a/extensions/ollama/src/setup-model-selection.ts b/extensions/ollama/src/setup-model-selection.ts index 2b6a9e3d071e..39fa911980ff 100644 --- a/extensions/ollama/src/setup-model-selection.ts +++ b/extensions/ollama/src/setup-model-selection.ts @@ -78,6 +78,7 @@ export function buildOllamaModelsConfig( name, discovered?.contextWindow ?? defaultModel?.contextWindow, capabilities, + { showInspectionFailed: discovered?.showInspectionFailed }, ); }); } @@ -105,9 +106,11 @@ export async function inspectOllamaModelsForSetup( } catch (error) { signal?.throwIfAborted(); // A failed inspection must not inherit the optimistic tools default - // reserved for models that were never inspected. + // reserved for models that were never inspected. Keep the failure + // distinct from authoritative empty capabilities so name-based + // reasoning detection still applies. inspectionFailures.push(`${model.name}: ${formatErrorMessage(error)}`); - return Object.assign({}, model, { capabilities: [] as string[] }); + return Object.assign({}, model, { showInspectionFailed: true as const }); } }), ); diff --git a/extensions/ollama/src/setup-pull.ts b/extensions/ollama/src/setup-pull.ts index 41b54aba4221..37c0c335ff38 100644 --- a/extensions/ollama/src/setup-pull.ts +++ b/extensions/ollama/src/setup-pull.ts @@ -78,6 +78,7 @@ async function pullOllamaModelCore(params: { clearTimeout(responseTimeout); try { if (!response.ok) { + await response.body?.cancel().catch(() => undefined); return { ok: false, message: `Failed to download ${modelName} (HTTP ${response.status})` }; } if (!response.body) { @@ -135,6 +136,8 @@ async function pullOllamaModelCore(params: { for (const line of lines) { const parsed = parseLine(line); if (!parsed.ok) { + // Ollama can report an error before closing the stream; discard the unread tail. + await reader.cancel().catch(() => undefined); return parsed; } } diff --git a/extensions/ollama/src/setup.ts b/extensions/ollama/src/setup.ts index 9ca354eb7ee1..becfb575fde7 100644 --- a/extensions/ollama/src/setup.ts +++ b/extensions/ollama/src/setup.ts @@ -142,6 +142,7 @@ export async function checkOllamaCloudAuth( } return { signedIn: true }; } finally { + await response.body?.cancel().catch(() => undefined); await release(); } } catch { diff --git a/extensions/ollama/src/stream.test.ts b/extensions/ollama/src/stream.test.ts index 965874831463..e8aa0ec4532b 100644 --- a/extensions/ollama/src/stream.test.ts +++ b/extensions/ollama/src/stream.test.ts @@ -694,4 +694,107 @@ describe("createOllamaStreamFn thinking events", () => { error: { stopReason: "aborted" }, }); }); + + it("uses CJK-aware fallback usage while preserving missing cache provenance", async () => { + const events = await streamOllamaEvents( + [ + { + model: "qwen3.5", + created_at: "2026-01-01T00:00:00Z", + message: { role: "assistant", content: "你好世界测试" }, + done: false, + }, + { + model: "qwen3.5", + created_at: "2026-01-01T00:00:01Z", + message: { role: "assistant", content: "" }, + done: true, + done_reason: "stop", + }, + ], + {}, + { messages: [{ role: "user", content: "这是一个测试用的句子呢" }] } as never, + ); + + const done = events.find((event) => event.type === "done") as { + message?: { + usage?: { + input?: number; + output?: number; + cacheRead?: number; + cacheWrite?: number; + cacheTelemetry?: { state: string }; + }; + }; + }; + expect(done?.message?.usage).toMatchObject({ + input: 12, + output: 6, + cacheRead: 0, + cacheWrite: 0, + cacheTelemetry: { state: "unavailable" }, + }); + }); + + it("keeps provider usage authoritative over the CJK fallback", async () => { + const events = await streamOllamaEvents( + [ + { + model: "qwen3.5", + created_at: "2026-01-01T00:00:00Z", + message: { role: "assistant", content: "你好世界测试" }, + done: false, + }, + { + model: "qwen3.5", + created_at: "2026-01-01T00:00:01Z", + message: { role: "assistant", content: "" }, + done: true, + done_reason: "stop", + prompt_eval_count: 77, + eval_count: 19, + }, + ], + {}, + { messages: [{ role: "user", content: "这是一个测试用的句子呢" }] } as never, + ); + + const done = events.find((event) => event.type === "done") as { + message?: { usage?: { input?: number; output?: number; cacheTelemetry?: { state: string } } }; + }; + expect(done?.message?.usage).toMatchObject({ + input: 77, + output: 19, + cacheTelemetry: { state: "unavailable" }, + }); + }); + + it("keeps the existing fallback estimate for ASCII-only usage", async () => { + const events = await streamOllamaEvents( + [ + { + model: "qwen3.5", + created_at: "2026-01-01T00:00:00Z", + message: { role: "assistant", content: "Hello world" }, + done: false, + }, + { + model: "qwen3.5", + created_at: "2026-01-01T00:00:01Z", + message: { role: "assistant", content: "" }, + done: true, + done_reason: "stop", + }, + ], + {}, + { + messages: [{ role: "user", content: "The quick brown fox jumps over the lazy dog" }], + } as never, + ); + + const done = events.find((event) => event.type === "done") as { + message?: { usage?: { input?: number; output?: number } }; + }; + expect(done?.message?.usage).toMatchObject({ input: 11, output: 3 }); + }); }); diff --git a/extensions/ollama/src/stream.ts b/extensions/ollama/src/stream.ts index 58236b6de570..2cf85b5815d8 100644 --- a/extensions/ollama/src/stream.ts +++ b/extensions/ollama/src/stream.ts @@ -33,6 +33,7 @@ import { createSubsystemLogger } from "openclaw/plugin-sdk/runtime-env"; import { fetchWithSsrFGuard, isLoopbackHost } from "openclaw/plugin-sdk/ssrf-runtime"; import { isRecord, readStringValue } from "openclaw/plugin-sdk/string-coerce-runtime"; import { truncateUtf16Safe } from "openclaw/plugin-sdk/text-utility-runtime"; +import { estimateStringChars } from "./cjk-char-estimate.js"; import { OLLAMA_CLOUD_BASE_URL, OLLAMA_DEFAULT_BASE_URL } from "./defaults.js"; import { shouldWrapOllamaCompatMoonshotThinking } from "./model-behavior.js"; import { normalizeOllamaWireModelId } from "./model-id.js"; @@ -683,7 +684,7 @@ interface OllamaChatResponse { function safeJsonLength(value: unknown): number { try { const serialized = JSON.stringify(value); - return typeof serialized === "string" ? serialized.length : 0; + return typeof serialized === "string" ? estimateStringChars(serialized) : 0; } catch { return 0; } @@ -714,10 +715,10 @@ function estimateOllamaPromptTokens(params: { }): number { let chars = 0; for (const message of params.messages) { - chars += message.content.length; + chars += estimateStringChars(message.content); chars += safeJsonLength(message.images); chars += safeJsonLength(message.tool_calls); - chars += message.tool_name?.length ?? 0; + chars += message.tool_name ? estimateStringChars(message.tool_name) : 0; } chars += safeJsonLength(params.tools); return estimateTokensFromChars(chars); @@ -729,9 +730,9 @@ function estimateOllamaCompletionTokens( ): number { const chars = extraOutputChars + - response.message.content.length + - (response.message.thinking?.length ?? 0) + - (response.message.reasoning?.length ?? 0) + + estimateStringChars(response.message.content) + + (response.message.thinking ? estimateStringChars(response.message.thinking) : 0) + + (response.message.reasoning ? estimateStringChars(response.message.reasoning) : 0) + safeJsonLength(response.message.tool_calls); return estimateTokensFromChars(chars); } @@ -1473,7 +1474,10 @@ function createRawOllamaStreamFn( const usageFallback = { input: estimateOllamaPromptTokens({ messages: ollamaMessages, tools: ollamaTools }), - output: estimateOllamaCompletionTokens(finalResponse, suppressedThinking.length), + output: estimateOllamaCompletionTokens( + finalResponse, + estimateStringChars(suppressedThinking), + ), }; const assistantMessage = buildAssistantMessage(finalResponse, modelInfo, usageFallback, { ...toolCallNameOptions, diff --git a/extensions/qa-lab/src/live-transports/telegram/profiles.test.ts b/extensions/qa-lab/src/live-transports/telegram/profiles.test.ts index 190b3564f971..0f389a5076bd 100644 --- a/extensions/qa-lab/src/live-transports/telegram/profiles.test.ts +++ b/extensions/qa-lab/src/live-transports/telegram/profiles.test.ts @@ -56,6 +56,16 @@ describe("Telegram QA profiles", () => { ).toThrow("execution.kind=flow"); }); + it("selects the native queue-validation regression as an explicit live scenario", () => { + expect( + resolveTelegramQaScenarioIds({ + profile: "release", + providerMode: "live-frontier", + scenarioIds: ["telegram-queue-invalid-mode"], + }), + ).toEqual(["telegram-queue-invalid-mode"]); + }); + it("rejects unknown profiles and channel-ineligible explicit scenarios", () => { expect(() => resolveTelegramQaScenarioIds({ providerMode: "live-frontier", profile: "transport" }), diff --git a/extensions/synthetic/README.md b/extensions/synthetic/README.md new file mode 100644 index 000000000000..782a114cfe2c --- /dev/null +++ b/extensions/synthetic/README.md @@ -0,0 +1,16 @@ +# OpenClaw Synthetic Provider + +Official OpenClaw provider plugin for Synthetic's hosted Anthropic-compatible +API. + +Install from OpenClaw: + +```bash +openclaw plugins install @openclaw/synthetic-provider +openclaw gateway restart +``` + +Configure `SYNTHETIC_API_KEY`, then select a `synthetic/` model. + +See https://docs.openclaw.ai/providers/synthetic for model and configuration +details. diff --git a/extensions/synthetic/index.ts b/extensions/synthetic/index.ts index 32e57a5b7426..a5e24eba3291 100644 --- a/extensions/synthetic/index.ts +++ b/extensions/synthetic/index.ts @@ -9,7 +9,7 @@ const PROVIDER_ID = "synthetic"; export default defineSingleProviderPluginEntry({ id: PROVIDER_ID, name: "Synthetic Provider", - description: "Bundled Synthetic provider plugin", + description: "Synthetic provider plugin", manifest, provider: { label: "Synthetic", diff --git a/extensions/synthetic/package.json b/extensions/synthetic/package.json index 0c61cceb2f3d..64d899c8ba6c 100644 --- a/extensions/synthetic/package.json +++ b/extensions/synthetic/package.json @@ -1,8 +1,11 @@ { "name": "@openclaw/synthetic-provider", "version": "2026.7.2", - "private": true, - "description": "OpenClaw Synthetic provider plugin", + "description": "OpenClaw Synthetic provider plugin.", + "repository": { + "type": "git", + "url": "https://github.com/openclaw/openclaw" + }, "type": "module", "devDependencies": { "@openclaw/plugin-sdk": "workspace:*" @@ -10,6 +13,23 @@ "openclaw": { "extensions": [ "./index.ts" - ] + ], + "install": { + "clawhubSpec": "clawhub:@openclaw/synthetic-provider", + "npmSpec": "@openclaw/synthetic-provider", + "defaultChoice": "npm", + "minHostVersion": ">=2026.7.2" + }, + "compat": { + "pluginApi": ">=2026.7.2" + }, + "build": { + "openclawVersion": "2026.7.2", + "bundledDist": false + }, + "release": { + "publishToClawHub": true, + "publishToNpm": true + } } } diff --git a/extensions/telegram/src/bot-core.ts b/extensions/telegram/src/bot-core.ts index c0cf86f3c5f1..e3cb36cf0b71 100644 --- a/extensions/telegram/src/bot-core.ts +++ b/extensions/telegram/src/bot-core.ts @@ -46,6 +46,7 @@ import { apiThrottler, Bot, sequentialize, type ApiClientOptions } from "./bot.r import type { TelegramBotOptions } from "./bot.types.js"; import { buildTelegramGroupPeerId } from "./bot/helpers.js"; import { setTelegramCallbackQueryAnswerPromise } from "./callback-query-answer-state.js"; +import { TELEGRAM_CHAT_ACTION_INTERVAL_MS } from "./chat-action-timing.js"; import { asTelegramClientFetch, createTelegramClientFetch, @@ -77,8 +78,6 @@ const DEFAULT_TELEGRAM_BOT_RUNTIME: TelegramBotRuntime = { sequentialize, apiThrottler, }; -const TELEGRAM_TYPING_COALESCE_MS = 4_000; - export function createTelegramBotCore( opts: TelegramBotOptions & { telegramDeps: TelegramBotDeps }, ): TelegramBotInstance { @@ -354,7 +353,7 @@ export function createTelegramBotCore( sendChatActionFn: (chatId, action, threadParams) => bot.api.sendChatAction(chatId, action, threadParams), logger: (message) => logVerbose(`telegram: ${message}`), - minIntervalMs: TELEGRAM_TYPING_COALESCE_MS, + minIntervalMs: TELEGRAM_CHAT_ACTION_INTERVAL_MS, }); const processMessage = createTelegramMessageProcessor({ diff --git a/extensions/telegram/src/bot-message-dispatch-turn.ts b/extensions/telegram/src/bot-message-dispatch-turn.ts index dba60b3a4af5..b2bc25178bd1 100644 --- a/extensions/telegram/src/bot-message-dispatch-turn.ts +++ b/extensions/telegram/src/bot-message-dispatch-turn.ts @@ -19,6 +19,7 @@ import type { TelegramProgressController } from "./bot-message-dispatch-progress import type { TelegramReplyDelivery } from "./bot-message-dispatch-reply.js"; import type { TelegramDispatchTurnState } from "./bot-message-dispatch.types.js"; import type { TelegramStreamMode } from "./bot/types.js"; +import { TELEGRAM_CHAT_ACTION_INTERVAL_MS } from "./chat-action-timing.js"; import { beginTelegramInboundEventDeliveryCorrelation } from "./inbound-event-delivery.js"; const TELEGRAM_MAX_CONSECUTIVE_TYPING_FAILURES = 5; @@ -70,6 +71,10 @@ export async function runTelegramDispatchTurn(params: { accountId: context.route.accountId, typing: { start: context.sendTyping, + keepaliveIntervalMs: TELEGRAM_CHAT_ACTION_INTERVAL_MS, + // ReplyOperation owns terminal cleanup; a per-inbound TTL would kill + // feedback while the same long-running task is still active. + maxDurationMs: 0, maxConsecutiveFailures: TELEGRAM_MAX_CONSECUTIVE_TYPING_FAILURES, onStartError: (err) => { logTypingFailure({ diff --git a/extensions/telegram/src/bot-message-dispatch.pipeline-init.test.ts b/extensions/telegram/src/bot-message-dispatch.pipeline-init.test.ts index 1b7d3267d197..24c4159c1b38 100644 --- a/extensions/telegram/src/bot-message-dispatch.pipeline-init.test.ts +++ b/extensions/telegram/src/bot-message-dispatch.pipeline-init.test.ts @@ -11,6 +11,19 @@ import type { TelegramMessageContext } from "./bot-message-dispatch.test-harness import { notifyTelegramInboundEventOutboundSuccess } from "./inbound-event-delivery.js"; describeTelegramDispatch("dispatchTelegramMessage pipeline-init", () => { + it("keeps Telegram typing below its client expiry without a per-message cutoff", async () => { + await dispatchWithContext({ context: createContext() }); + + expect(createChannelMessageReplyPipeline).toHaveBeenCalledWith( + expect.objectContaining({ + typing: expect.objectContaining({ + keepaliveIntervalMs: 4_000, + maxDurationMs: 0, + }), + }), + ); + }); + it("cleans delivery correlation when reply-pipeline initialization fails", async () => { const sessionKey = "agent:main:telegram:direct:pipeline-init-failure"; const statusReactionController = createStatusReactionController(); diff --git a/extensions/telegram/src/bot-native-commands.session-meta.test.ts b/extensions/telegram/src/bot-native-commands.session-meta.test.ts index d240eefa9651..6c6986eb5d7e 100644 --- a/extensions/telegram/src/bot-native-commands.session-meta.test.ts +++ b/extensions/telegram/src/bot-native-commands.session-meta.test.ts @@ -15,6 +15,7 @@ import { type NativeCommandTestParams, } from "./bot-native-commands.fixture-test-support.js"; import type { RegisterTelegramHandlerParams } from "./bot-native-commands.js"; +import { runWithTelegramUpdateProcessingFrame } from "./bot-processing-outcome.js"; // All mocks scoped to this file only — does not affect bot-native-commands.test.ts @@ -731,6 +732,39 @@ describe("registerTelegramNativeCommands — session metadata", () => { expect(turnPlan?.record?.sessionKey).toBe(turnPlan?.ctxPayload.CommandTargetSessionKey); }); + it("records a completed outcome after a native slash command", async () => { + const { handler } = registerAndResolveStatusHandler({ cfg: {} }); + + const { result } = await runWithTelegramUpdateProcessingFrame(async () => { + await handler(createTelegramPrivateCommandContext()); + }); + + expect(result).toEqual({ kind: "completed" }); + }); + + it("preserves every argument on native queue command turns", async () => { + const { handler } = registerAndResolveCommandHandler({ + commandName: "queue", + cfg: {}, + allowFrom: ["*"], + }); + + await handler(createTelegramPrivateCommandContext({ match: "Can you diagnose this?" })); + + expect(dispatchChannelInboundTurnMock).toHaveBeenCalledWith( + expect.objectContaining({ + ctxPayload: expect.objectContaining({ + Body: "/queue Can you diagnose this?", + CommandBody: "/queue Can you diagnose this?", + CommandTurn: expect.objectContaining({ + kind: "native", + body: "/queue Can you diagnose this?", + }), + }), + }), + ); + }); + it("keeps one live config snapshot through native command execution", async () => { const startupCfg: OpenClawConfig = { session: { store: "/tmp/startup-sessions.json" } }; const runtimeCfg: OpenClawConfig = { session: { store: "/tmp/runtime-sessions.json" } }; diff --git a/extensions/telegram/src/bot-native-commands.ts b/extensions/telegram/src/bot-native-commands.ts index 52947497f153..08b5d5d5c71c 100644 --- a/extensions/telegram/src/bot-native-commands.ts +++ b/extensions/telegram/src/bot-native-commands.ts @@ -75,7 +75,10 @@ import { syncTelegramMenuCommands as syncTelegramMenuCommandsRuntime, type TelegramMenuCommand, } from "./bot-native-command-menu.js"; -import type { TelegramMessageProcessingResult } from "./bot-processing-outcome.js"; +import { + recordTelegramMessageProcessingResult, + type TelegramMessageProcessingResult, +} from "./bot-processing-outcome.js"; import type { TelegramUpdateKeyContext } from "./bot-updates.js"; import type { TelegramBotOptions } from "./bot.types.js"; import { @@ -122,6 +125,20 @@ const EMPTY_RESPONSE_FALLBACK = "No response generated. Please try again."; const activeTelegramCodexLoginFlows = new Map(); type TelegramNativeCommandContext = Context & { match?: string }; + +function registerTelegramNativeCommandHandler( + bot: Bot, + command: string, + handler: (ctx: TelegramNativeCommandContext) => Promise, +): void { + bot.command(command, async (ctx: TelegramNativeCommandContext) => { + await handler(ctx); + // Native commands bypass processMessage, so their terminal outcome must be + // recorded here for every built-in, plugin, and direct-delivery branch. + recordTelegramMessageProcessingResult({ kind: "completed" }); + }); +} + type TelegramChunkMode = ReturnType< typeof import("openclaw/plugin-sdk/reply-dispatch-runtime").resolveChunkMode >; @@ -1210,7 +1227,7 @@ export const registerTelegramNativeCommands = ({ if (commandsToRegister.length > 0 || pluginCatalog.commands.length > 0) { for (const command of nativeCommands) { const normalizedCommandName = normalizeTelegramCommandName(command.name); - bot.command(normalizedCommandName, async (ctx: TelegramNativeCommandContext) => { + registerTelegramNativeCommandHandler(bot, normalizedCommandName, async (ctx) => { const msg = ctx.message; if (!msg) { return; @@ -1800,7 +1817,7 @@ export const registerTelegramNativeCommands = ({ } for (const pluginCommand of pluginCatalog.commands) { - bot.command(pluginCommand.command, async (ctx: TelegramNativeCommandContext) => { + registerTelegramNativeCommandHandler(bot, pluginCommand.command, async (ctx) => { const msg = ctx.message; if (!msg) { return; diff --git a/extensions/telegram/src/chat-action-timing.ts b/extensions/telegram/src/chat-action-timing.ts new file mode 100644 index 000000000000..e782c6481bb4 --- /dev/null +++ b/extensions/telegram/src/chat-action-timing.ts @@ -0,0 +1,3 @@ +// Telegram typing expires after five seconds; renew before that without +// fighting the account-scoped sendChatAction coalescing window. +export const TELEGRAM_CHAT_ACTION_INTERVAL_MS = 4_000; diff --git a/package.json b/package.json index f6aa93bd248f..1fd62961b564 100644 --- a/package.json +++ b/package.json @@ -304,6 +304,7 @@ "!dist/extensions/slack/**", "!dist/extensions/sms/**", "!dist/extensions/stepfun/**", + "!dist/extensions/synthetic/**", "!dist/extensions/synology-chat/**", "!dist/extensions/tavily/**", "!dist/extensions/teams-meetings/**", diff --git a/qa/scenarios/channels/telegram-queue-invalid-mode.yaml b/qa/scenarios/channels/telegram-queue-invalid-mode.yaml new file mode 100644 index 000000000000..796f2c114d6a --- /dev/null +++ b/qa/scenarios/channels/telegram-queue-invalid-mode.yaml @@ -0,0 +1,69 @@ +title: Telegram native queue command rejects ordinary prompt text + +scenario: + id: telegram-queue-invalid-mode + surface: channels + category: channels.channel-actions-commands-and-approvals + coverage: + primary: + - telegram.built-in-commands + regressionRefs: + - openclaw/openclaw#116688 + objective: Verify a native Telegram queue command with ordinary trailing text returns its queue-mode validation error without invoking the model or synthesizing a model-failure fallback. + successCriteria: + - Telegram accepts the native queue command with its complete trailing argument text. + - The reply identifies the invalid queue mode and lists supported queue modes. + - The reply never blames the model, and a mock provider receives no request for the command. + codeRefs: + - extensions/telegram/src/bot-native-commands.ts + - src/auto-reply/reply/get-reply-directives.ts + - src/auto-reply/reply/directive-handling.queue-validation.ts + execution: + kind: flow + channel: telegram + summary: Send the reported native queue command and verify its explicit validation reply. + config: + commandText: /queue Can you diagnose this? + invalidModeNeedle: Unrecognized queue mode "Can" + validModesNeedle: "Valid modes: steer, followup, collect, interrupt." + falseFallbackNeedle: temporary model failure + +flow: + steps: + - name: invalid native queue arguments produce a visible command error + actions: + - resetTransport: true + - set: requestCursorBefore + value: + expr: "env.mock ? (await fetchJson(`${env.mock.baseUrl}/debug/request-cursor`)).cursor : 0" + - set: startIndex + value: + expr: "state.getSnapshot().messages.filter((message) => message.direction === 'outbound').length" + - sendInbound: + conversation: { id: telegram-command-room, kind: channel } + senderId: qa-command-operator + senderName: QA Command Operator + text: { ref: config.commandText } + nativeCommand: { name: queue } + - waitForOutbound: + conversation: { id: telegram-command-room, kind: channel } + sinceIndex: { ref: startIndex } + textIncludes: { ref: config.invalidModeNeedle } + timeoutMs: 60000 + saveAs: reply + - assert: + expr: "reply.text.includes(config.validModesNeedle)" + message: + expr: "`queue validation reply omitted the valid modes: ${reply.text}`" + - assert: + expr: "!reply.text.includes(config.falseFallbackNeedle)" + message: + expr: "`queue validation emitted the false model fallback: ${reply.text}`" + - set: scenarioRequests + value: + expr: "env.mock ? await fetchJson(`${env.mock.baseUrl}/debug/requests?after=${requestCursorBefore}`) : []" + - assert: + expr: "!env.mock || scenarioRequests.length === 0" + message: + expr: "`native queue validation unexpectedly invoked the model ${String(scenarioRequests.length)} time(s)`" + detailsExpr: reply.text diff --git a/scripts/lib/official-external-provider-catalog.json b/scripts/lib/official-external-provider-catalog.json index 6590b6abd679..882145a98922 100644 --- a/scripts/lib/official-external-provider-catalog.json +++ b/scripts/lib/official-external-provider-catalog.json @@ -1659,6 +1659,55 @@ } } }, + { + "name": "@openclaw/synthetic-provider", + "description": "OpenClaw Synthetic provider plugin.", + "source": "official", + "kind": "provider", + "openclaw": { + "plugin": { + "id": "synthetic", + "label": "Synthetic" + }, + "providers": [ + { + "id": "synthetic", + "name": "Synthetic", + "docs": "/providers/synthetic", + "categories": [ + "cloud", + "llm" + ], + "envVars": [ + "SYNTHETIC_API_KEY" + ], + "authChoices": [ + { + "method": "api-key", + "choiceId": "synthetic-api-key", + "choiceLabel": "Synthetic API key", + "groupId": "synthetic", + "groupLabel": "Synthetic", + "groupHint": "Anthropic-compatible (multi-model)", + "optionKey": "syntheticApiKey", + "cliFlag": "--synthetic-api-key", + "cliOption": "--synthetic-api-key ", + "cliDescription": "Synthetic API key", + "onboardingScopes": [ + "text-inference" + ] + } + ] + } + ], + "install": { + "clawhubSpec": "clawhub:@openclaw/synthetic-provider", + "npmSpec": "@openclaw/synthetic-provider", + "defaultChoice": "npm", + "minHostVersion": ">=2026.7.2" + } + } + }, { "name": "@openclaw/stepfun-provider", "description": "OpenClaw StepFun provider plugin.", diff --git a/src/agents/main-session-recovery-lifecycle.ts b/src/agents/main-session-recovery-lifecycle.ts index 8066c98f5a66..5fda42d1e9d3 100644 --- a/src/agents/main-session-recovery-lifecycle.ts +++ b/src/agents/main-session-recovery-lifecycle.ts @@ -102,8 +102,15 @@ export function projectMainSessionRecoveryLifecycle(params: { lifecycleGeneration && runs?.some((run) => run.runId === runId && run.lifecycleGeneration === lifecycleGeneration), ); + // The current owner retires stale generations of its own run id. An older + // delayed event consumes only its matching fence and cannot settle its replacement. const remaining = matchesFence - ? runs?.filter((run) => run.runId !== runId || run.lifecycleGeneration !== lifecycleGeneration) + ? runs?.filter( + (run) => + run.runId !== runId || + (lifecycleGeneration !== params.currentLifecycleGeneration && + run.lifecycleGeneration !== lifecycleGeneration), + ) : runs; if (settlesRecovery) { const foregroundClaims = params.entry?.mainRestartRecovery?.foregroundClaims; diff --git a/src/agents/main-session-recovery-run-ownership.test.ts b/src/agents/main-session-recovery-run-ownership.test.ts new file mode 100644 index 000000000000..e41f6d77adf7 --- /dev/null +++ b/src/agents/main-session-recovery-run-ownership.test.ts @@ -0,0 +1,75 @@ +import { describe, expect, it } from "vitest"; +import type { InternalSessionEntry as SessionEntry } from "../config/sessions.js"; +import { projectMainSessionRecoveryLifecycle } from "./main-session-recovery-lifecycle.js"; + +function recoveryEntry(params?: { hasCurrentOwner?: boolean }): SessionEntry { + return { + sessionId: "session-1", + updatedAt: 100, + status: "running", + abortedLastRun: false, + restartRecoveryRuns: [ + { runId: "recovery", lifecycleGeneration: "generation-old" }, + { runId: "recovery", lifecycleGeneration: "generation-current" }, + ], + mainRestartRecovery: { + cycleId: "cycle-1", + revision: 5, + chargedAttempts: 2, + ...(params?.hasCurrentOwner + ? { + foregroundClaims: { + lifecycleGeneration: "generation-current", + tokens: ["current-owner"], + }, + } + : {}), + }, + }; +} + +describe("main-session recovery run ownership", () => { + it("settles a resumed run once when older generations retain the same run id", () => { + expect( + projectMainSessionRecoveryLifecycle({ + currentLifecycleGeneration: "generation-current", + entry: recoveryEntry(), + event: { + runId: "recovery", + lifecycleGeneration: "generation-current", + data: { phase: "end" }, + }, + snapshotPatch: { status: "done", abortedLastRun: false }, + }), + ).toEqual({ + action: "apply", + patch: { + status: "done", + abortedLastRun: false, + restartRecoveryRuns: undefined, + mainRestartRecovery: undefined, + }, + }); + }); + + it("does not let an older same-id terminal settle its replacement generation", () => { + expect( + projectMainSessionRecoveryLifecycle({ + currentLifecycleGeneration: "generation-current", + entry: recoveryEntry({ hasCurrentOwner: true }), + event: { + runId: "recovery", + lifecycleGeneration: "generation-old", + data: { phase: "end" }, + }, + snapshotPatch: { status: "done", abortedLastRun: false }, + }), + ).toEqual({ + action: "apply", + patch: { + restartRecoveryRuns: [{ runId: "recovery", lifecycleGeneration: "generation-current" }], + restartRecoveryTerminalRunIds: ["recovery"], + }, + }); + }); +}); diff --git a/src/agents/main-session-recovery-state.test.ts b/src/agents/main-session-recovery-state.test.ts index 861b93f7d8ac..58a529f057f7 100644 --- a/src/agents/main-session-recovery-state.test.ts +++ b/src/agents/main-session-recovery-state.test.ts @@ -92,7 +92,7 @@ describe("main session recovery state", () => { expect(entry).toEqual(before); }); - it("marks without charging and preserves generation-scoped lifecycle fences", () => { + it("marks without charging and replaces an older lifecycle owner for the same run", () => { const entry = interruptedEntry({ restartRecoveryRuns: [ { runId: "older-run", lifecycleGeneration: "generation-old" }, @@ -123,7 +123,6 @@ describe("main session recovery state", () => { expect(entry.restartRecoveryRuns).toEqual([ { runId: "new-run", lifecycleGeneration: "generation-2" }, { runId: "older-run", lifecycleGeneration: "generation-old" }, - { runId: "shared-run", lifecycleGeneration: "generation-1" }, { runId: "shared-run", lifecycleGeneration: "generation-2" }, ]); }); @@ -422,6 +421,7 @@ describe("main session recovery state", () => { pendingFinalDelivery: { kind: "replayable", text: " captured reply ", createdAt: 1 }, restartRecoveryDeliveryRunId: "recovery-1", restartRecoveryDeliverySourceRunId: "source-1", + restartRecoveryRuns: [{ runId: "recovery-1", lifecycleGeneration: "generation-old" }], mainRestartRecovery: recoveryState({ revision: 2, chargedAttempts: 1, diff --git a/src/agents/main-session-recovery-state.ts b/src/agents/main-session-recovery-state.ts index 35d80dda3d6c..de54fc8b5a5b 100644 --- a/src/agents/main-session-recovery-state.ts +++ b/src/agents/main-session-recovery-state.ts @@ -102,20 +102,28 @@ function validateRecoveryAdmission( return hasCurrentForegroundClaim(state, command.lifecycleGeneration) ? "foreground_active" : null; } -function recordLifecycleFence(entry: SessionEntry, run: RestartRecoveryRun): void { - // Lifecycle fences can overlap and are consumed independently by their matching events. - const runs = new Map(); - for (const existing of entry.restartRecoveryRuns ?? []) { - runs.set(`${existing.runId}\u0000${existing.lifecycleGeneration}`, existing); +/** Keeps distinct concurrent runs while transferring each run id to its newest lifecycle owner. */ +export function normalizeMainSessionRecoveryRunFences( + runs: Iterable, +): RestartRecoveryRun[] { + const ownersByRunId = new Map(); + for (const run of runs) { + ownersByRunId.set(run.runId, run); } - runs.set(`${run.runId}\u0000${run.lifecycleGeneration}`, run); - entry.restartRecoveryRuns = [...runs.values()].toSorted((a, b) => - a.runId === b.runId - ? a.lifecycleGeneration.localeCompare(b.lifecycleGeneration) - : a.runId.localeCompare(b.runId), + return [...ownersByRunId.values()].toSorted((left, right) => + left.runId.localeCompare(right.runId), ); } +function recordLifecycleFence(entry: SessionEntry, run: RestartRecoveryRun): void { + // A resumed run keeps its id across Gateway generations. Leaving its old fence + // behind makes terminal settlement preserve a dead owner and blocks every later turn. + entry.restartRecoveryRuns = normalizeMainSessionRecoveryRunFences([ + ...(entry.restartRecoveryRuns ?? []), + run, + ]); +} + function hasLifecycleFence(entry: SessionEntry, run: RestartRecoveryRun): boolean { return Boolean( entry.restartRecoveryRuns?.some( diff --git a/src/agents/main-session-recovery-store.test.ts b/src/agents/main-session-recovery-store.test.ts index aed1262c28b3..e43c1f058d1a 100644 --- a/src/agents/main-session-recovery-store.test.ts +++ b/src/agents/main-session-recovery-store.test.ts @@ -241,6 +241,35 @@ describe("main session recovery store", () => { expect(readStore()[legacyKey]).toMatchObject({ abortedLastRun: true }); }); + it("transfers a resumed recovery run to one durable lifecycle owner", async () => { + await write( + interruptedEntry({ + restartRecoveryRuns: [{ runId: "recovery-1", lifecycleGeneration: "generation-old" }], + mainRestartRecovery: { + cycleId: "cycle-1", + revision: 2, + chargedAttempts: 1, + reservation: { runId: "recovery-1", attempt: 1, lifecycleGeneration }, + }, + }), + ); + + const admitted = await commitMainSessionRecovery({ + command: { + kind: "admit_recovery", + lifecycleGeneration, + now: 300, + runId: "recovery-1", + sessionId: "session-1", + }, + target: { sessionKey, storePath }, + }); + + expect(admitted.transition).toEqual({ kind: "admitted_recovery" }); + expect(read().restartRecoveryRuns).toEqual([{ runId: "recovery-1", lifecycleGeneration }]); + expect(read().abortedLastRun).toBe(false); + }); + it("rejects an observation after the session is replaced", async () => { await write({ sessionId: "session-2", diff --git a/src/agents/main-session-restart-recovery-marking.ts b/src/agents/main-session-restart-recovery-marking.ts index 4c97e07fca06..9dba9de455a0 100644 --- a/src/agents/main-session-restart-recovery-marking.ts +++ b/src/agents/main-session-restart-recovery-marking.ts @@ -20,7 +20,10 @@ import { listActiveEmbeddedRunSessionIds, listActiveEmbeddedRunSessionKeys, } from "./embedded-agent-runner/run-state.js"; -import { transitionMainSessionRecovery } from "./main-session-recovery-state.js"; +import { + normalizeMainSessionRecoveryRunFences, + transitionMainSessionRecovery, +} from "./main-session-recovery-state.js"; import { hasCurrentProcessOwner, log, @@ -189,34 +192,16 @@ export async function markRestartAbortedMainSessions(params: { continue; } const wasRunning = entry.status === "running"; - const recoveryRuns = new Map(); - for (const run of entry.restartRecoveryRuns ?? []) { - if (run.lifecycleGeneration === currentLifecycleGeneration) { - recoveryRuns.set(`${run.runId}\u0000${run.lifecycleGeneration}`, run); - } - } - const replaceActiveRunMarker = (run: RestartRecoveryRun) => { - for (const [key, existingRun] of recoveryRuns) { - if (existingRun.runId === run.runId) { - recoveryRuns.delete(key); - } - } - recoveryRuns.set(`${run.runId}\u0000${run.lifecycleGeneration}`, run); - }; - for (const run of registeredActiveRuns) { - replaceActiveRunMarker(run); - } - for (const run of matchingActiveRuns) { - replaceActiveRunMarker({ - runId: run.runId, - lifecycleGeneration: run.lifecycleGeneration, - }); - } - entry.restartRecoveryRuns = [...recoveryRuns.values()].toSorted((a, b) => - a.runId === b.runId - ? a.lifecycleGeneration.localeCompare(b.lifecycleGeneration) - : a.runId.localeCompare(b.runId), - ); + entry.restartRecoveryRuns = normalizeMainSessionRecoveryRunFences([ + ...(entry.restartRecoveryRuns ?? []).filter( + (run) => run.lifecycleGeneration === currentLifecycleGeneration, + ), + ...registeredActiveRuns, + ...matchingActiveRuns.map(({ runId, lifecycleGeneration }) => ({ + runId, + lifecycleGeneration, + })), + ]); transitionMainSessionRecovery(entry, { kind: "mark_interrupted", cycleId: randomUUID(), diff --git a/src/agents/openclaw-tools.session-status.test.ts b/src/agents/openclaw-tools.session-status.test.ts index 8fafe6ed5d16..0a3a0ec1c0ad 100644 --- a/src/agents/openclaw-tools.session-status.test.ts +++ b/src/agents/openclaw-tools.session-status.test.ts @@ -578,6 +578,25 @@ describe("session_status tool", () => { getSessionStateVersionMock.mockReturnValue(12); listSessionStateEventsSinceMock.mockReturnValue({ events: [ + { + sequence: 11, + sessionKey: "main", + sessionId: "s1", + agentId: "main", + kind: "run_failed", + actorType: "agent", + actorId: "worker-1", + runId: "run-11", + occurredAt: 90, + summary: "child run timed out", + payload: { + outcome: "timeout", + channel: "codex", + turns: 2, + catalogId: "internal-catalog", + nested: { drop: true }, + }, + }, { sequence: 12, sessionKey: "main", @@ -591,7 +610,7 @@ describe("session_status tool", () => { }, ], truncated: false, - earliestAvailableSequence: 12, + earliestAvailableSequence: 11, historyGap: true, }); @@ -603,9 +622,18 @@ describe("session_status tool", () => { expect(getSessionStateVersionMock).toHaveBeenCalledWith("main", "main"); expect(listSessionStateEventsSinceMock).toHaveBeenCalledWith("main", "main", 3, 200); expect(details.stateVersion).toBe(12); - expect(details.stateChanges).toMatchObject({ - historyGap: true, + const expectedStateChanges = { events: [ + { + sequence: 11, + kind: "run_failed", + actorType: "agent", + occurredAt: 90, + summary: "child run timed out", + actorId: "worker-1", + runId: "run-11", + payload: { outcome: "timeout", channel: "codex", turns: 2 }, + }, { sequence: 12, kind: "upstream_missing", @@ -615,11 +643,35 @@ describe("session_status tool", () => { payload: { channel: "codex" }, }, ], - }); + truncated: false, + earliestAvailableSequence: 11, + historyGap: true, + }; + expect(details.stateChanges).toEqual(expectedStateChanges); expect(Value.Check(tool.outputSchema!, result.details)).toBe(true); - expect(JSON.stringify(details.stateChanges)).not.toContain("internal-catalog"); - expect(text).toContain("Session state changes:"); - expect(text).toContain('"kind": "upstream_missing"'); + expect(details.statusText).toBe(text); + const stateChangesMarker = "Session state changes:\n```json\n"; + const stateChangesStart = text.indexOf(stateChangesMarker); + expect(stateChangesStart).toBeGreaterThanOrEqual(0); + const stateChangesJsonStart = stateChangesStart + stateChangesMarker.length; + const stateChangesJsonEnd = text.indexOf("\n```", stateChangesJsonStart); + expect(stateChangesJsonEnd).toBeGreaterThan(stateChangesJsonStart); + const visibleStateChangesText = text.slice(stateChangesJsonStart, stateChangesJsonEnd); + expect(JSON.parse(visibleStateChangesText)).toEqual({ + stateVersion: 12, + stateChanges: expectedStateChanges, + }); + for (const omittedField of [ + '"sessionKey"', + '"sessionId"', + '"agentId"', + '"catalogId"', + '"nested"', + "internal-catalog", + ]) { + expect(visibleStateChangesText).not.toContain(omittedField); + expect(String(details.statusText)).not.toContain(omittedField); + } }); it("returns watched group changesSince under tree visibility", async () => { diff --git a/src/agents/tools/session-status-tool.ts b/src/agents/tools/session-status-tool.ts index b6357324b4d3..331017840082 100644 --- a/src/agents/tools/session-status-tool.ts +++ b/src/agents/tools/session-status-tool.ts @@ -363,7 +363,7 @@ ${JSON.stringify(details, null, 2)} function formatSessionStateChanges(details: { stateVersion: number; - stateChanges: ReturnType; + stateChanges: ReturnType; }): string { return `Session state changes: \`\`\`json @@ -1085,9 +1085,7 @@ export function createSessionStatusTool(opts?: { : undefined; const extraBlocks = [ routeContextText, - rawStateChanges - ? formatSessionStateChanges({ stateVersion, stateChanges: rawStateChanges }) - : undefined, + stateChanges ? formatSessionStateChanges({ stateVersion, stateChanges }) : undefined, ].filter((block): block is string => Boolean(block)); const visibleStatusText = extraBlocks.length > 0 diff --git a/src/auto-reply/reply/agent-runner-run.ts b/src/auto-reply/reply/agent-runner-run.ts index 3e03df5ef454..39617776aacd 100644 --- a/src/auto-reply/reply/agent-runner-run.ts +++ b/src/auto-reply/reply/agent-runner-run.ts @@ -57,6 +57,7 @@ import { enqueueFollowupRun, type FollowupRun, scheduleFollowupDrain } from "./q import { createReplyMediaContext } from "./reply-media-paths.js"; import { resolveReplyOperationRunState } from "./reply-operation-run-state.js"; import { type ReplyOperation, replyRunRegistry } from "./reply-run-registry.js"; +import { bindReplyOperationTyping, refreshReplyOperationTyping } from "./reply-run-typing.js"; import { createReplyToModeFilterForChannel, resolveReplyToMode } from "./reply-threading.js"; import { admitReplyTurn, resolveReplyTurnKind } from "./reply-turn-admission.js"; import { @@ -331,6 +332,13 @@ export async function runReplyAgent( if (followupRun.currentInboundAudio === true) { activeReplyOperation?.markAcceptedSteeredInboundAudio(); } + if (activeReplyOperation) { + // Steering joins the existing task; its dispatch-local controller is + // disposable, while the task-owned controller must keep its lifetime. + await refreshReplyOperationTyping(activeReplyOperation, { + startIfIdle: typingSignals.shouldStartImmediately, + }); + } await touchActiveSessionEntry(); typing.cleanup(); return undefined; @@ -557,6 +565,7 @@ export async function runReplyAgent( } } } + bindReplyOperationTyping(replyOperation, typing); let runFollowupTurn = queuedRunFollowupTurn; let shouldDrainQueuedFollowupsAfterClear = false; const returnWithQueuedFollowupDrain = (value: T): T => { diff --git a/src/auto-reply/reply/agent-runner.runreplyagent.e2e.test.ts b/src/auto-reply/reply/agent-runner.runreplyagent.e2e.test.ts index cfe5fefaa52e..01a286de97f0 100644 --- a/src/auto-reply/reply/agent-runner.runreplyagent.e2e.test.ts +++ b/src/auto-reply/reply/agent-runner.runreplyagent.e2e.test.ts @@ -42,6 +42,7 @@ import { } from "./reply-operation-run-state.js"; import { createReplyOperation, type ReplyOperation } from "./reply-run-registry.js"; import { testing as replyRunTesting } from "./reply-run-registry.test-support.js"; +import { bindReplyOperationTyping } from "./reply-run-typing.js"; import { consumeReplyUsageState } from "./reply-usage-state.js"; import { buildChannelSourceTurnId, setChannelSourceTurnId } from "./source-turn-id.js"; import { createMockTypingController } from "./test-helpers.js"; @@ -437,6 +438,44 @@ function requireBuiltChannelSourceTurnId( } describe("runReplyAgent active steering", () => { + it("keeps the continuing Telegram task's typing alive after an accepted steer", async () => { + state.queueEmbeddedAgentMessageMock.mockReturnValueOnce(true); + const active = createReplyOperation({ + sessionKey: "main", + sessionId: "session", + resetTriggered: false, + }); + active.setPhase("running"); + const taskTyping = createMockTypingController({ isActive: vi.fn(() => true) }); + bindReplyOperationTyping(active, taskTyping); + const { run, typing } = createMinimalRun({ + isActive: true, + isStreaming: true, + shouldSteer: true, + resolvedQueueMode: "steer", + sessionCtx: { + Provider: "telegram", + OriginatingChannel: "telegram", + OriginatingTo: "123", + NativeChannelId: "123", + MessageSid: "steer-telegram", + }, + runOverrides: { agentId: "main", messageProvider: "telegram" }, + }); + + await expect(run()).resolves.toBeUndefined(); + + expect(state.runEmbeddedAgentMock).not.toHaveBeenCalled(); + expect(taskTyping.startTypingLoop).toHaveBeenCalledOnce(); + expect(taskTyping.refreshTypingTtl).toHaveBeenCalledOnce(); + expect(taskTyping.cleanup).not.toHaveBeenCalled(); + expect(typing.cleanup).toHaveBeenCalledOnce(); + + active.complete(); + + expect(taskTyping.cleanup).toHaveBeenCalledOnce(); + }); + it("dispatches a declined steer once with its source-turn identity", async () => { const runState: ReplyOperationRunState = {}; state.beforeAgentReplyHasHooksMock.mockImplementation( diff --git a/src/auto-reply/reply/get-reply-directives.ts b/src/auto-reply/reply/get-reply-directives.ts index 47edb7f7ab68..e78d10860048 100644 --- a/src/auto-reply/reply/get-reply-directives.ts +++ b/src/auto-reply/reply/get-reply-directives.ts @@ -18,6 +18,7 @@ import { normalizeAgentId } from "../../routing/session-key.js"; import { ModelSelectionLockedError } from "../../sessions/model-overrides.js"; import { createLazyImportLoader } from "../../shared/lazy-promise.js"; import type { SkillCommandSpec } from "../../skills/types.js"; +import { isNativeCommandTurn, resolveCommandTurnContext } from "../command-turn-context.js"; import { shouldHandleTextCommands } from "../commands-text-routing.js"; import { markCommandReplyForDelivery } from "../reply-payload.js"; import type { @@ -36,6 +37,7 @@ import type { GetReplyOptions, ReplyPayload } from "../types.js"; import { resolveBlockStreamingChunking } from "./block-streaming.js"; import { buildCommandContext } from "./commands-context.js"; import { type InlineDirectives, parseInlineDirectives } from "./directive-handling.parse.js"; +import { maybeHandleQueueDirective } from "./directive-handling.queue-validation.js"; import { reserveSkillCommandNames, resolveConfiguredDirectiveAliases, @@ -275,6 +277,25 @@ export async function resolveReplyDirectives(params: { modelAliases: configuredAliases, allowStatusDirective, }); + const commandTurn = resolveCommandTurnContext(ctx); + if ( + command.isAuthorizedSender && + isNativeCommandTurn(commandTurn) && + commandTurn.commandName === "queue" && + parsedDirectives.hasQueueDirective + ) { + // Native command arguments belong to the command, not to an inline prompt; + // validate them before mixed-text cleanup can erase an invalid queue mode. + const queueReply = maybeHandleQueueDirective({ + directives: parsedDirectives, + cfg, + channel: command.channel, + sessionEntry: targetSessionEntry, + }); + if (queueReply) { + return { kind: "reply", reply: markCommandReplyForDelivery(queueReply) }; + } + } const hasInlineStatus = parsedDirectives.hasStatusDirective && parsedDirectives.cleaned.trim().length > 0; if (hasInlineStatus) { diff --git a/src/auto-reply/reply/get-reply-native-slash-fast-path.test.ts b/src/auto-reply/reply/get-reply-native-slash-fast-path.test.ts index 1be062bd7423..9ea0f8f1dfc4 100644 --- a/src/auto-reply/reply/get-reply-native-slash-fast-path.test.ts +++ b/src/auto-reply/reply/get-reply-native-slash-fast-path.test.ts @@ -43,6 +43,59 @@ describe("maybeResolveNativeSlashCommandFastReply", () => { handleCommandsMock.mockReset(); }); + it("returns native queue validation instead of discarding trailing command arguments", async () => { + handleCommandsMock.mockResolvedValue({ shouldContinue: true }); + + const body = "/queue Can you diagnose this?"; + const typing = createTypingController(); + const result = await maybeResolveNativeSlashCommandFastReply({ + ctx: buildTestCtx({ + Body: body, + BodyForAgent: body, + RawBody: body, + CommandBody: body, + CommandSource: "native", + CommandAuthorized: true, + Provider: "telegram", + Surface: "telegram", + SessionKey: "telegram:slash:123", + CommandTargetSessionKey: "agent:main:telegram:123", + CommandTurn: { + kind: "native", + source: "native", + authorized: true, + commandName: "queue", + body, + }, + }), + cfg: markCompleteReplyConfig({ + session: { + store: path.join(tempDirs.make("openclaw-native-queue-"), "sessions.json"), + }, + } as OpenClawConfig), + agentId: "main", + agentDir: "/tmp/agent", + agentCfg: undefined, + commandAuthorized: true, + defaultProvider: "openai", + defaultModel: "gpt-5.5", + aliasIndex: { byKey: new Map(), byAlias: new Map() }, + provider: "openai", + model: "gpt-5.5", + workspaceDir: "/tmp/workspace", + typing, + }); + + expect(result).toEqual({ + handled: true, + reply: expect.objectContaining({ + text: 'Unrecognized queue mode "Can". Valid modes: steer, followup, collect, interrupt.', + }), + }); + expect(handleCommandsMock).toHaveBeenCalledOnce(); + expect(typing.cleanup).toHaveBeenCalledOnce(); + }); + it("marks native /compact terminal replies for delivery under message_tool_only (#90185)", async () => { handleCommandsMock.mockResolvedValueOnce({ shouldContinue: false, diff --git a/src/auto-reply/reply/reply-run-typing.ts b/src/auto-reply/reply/reply-run-typing.ts new file mode 100644 index 000000000000..f2ca73edb83f --- /dev/null +++ b/src/auto-reply/reply/reply-run-typing.ts @@ -0,0 +1,36 @@ +import { runAfterReplyOperationClear, type ReplyOperation } from "./reply-run-registry.js"; +import type { TypingController } from "./typing.js"; + +const typingByReplyOperation = new WeakMap(); + +/** Keep one feedback controller attached to the task that owns a reply run. */ +export function bindReplyOperationTyping( + operation: ReplyOperation, + typing: TypingController, +): void { + if (typingByReplyOperation.has(operation)) { + return; + } + typingByReplyOperation.set(operation, typing); + runAfterReplyOperationClear(operation, () => { + if (typingByReplyOperation.get(operation) !== typing) { + return; + } + typingByReplyOperation.delete(operation); + typing.cleanup(); + }); +} + +/** Refresh the continuing task's feedback after it adopts another inbound turn. */ +export async function refreshReplyOperationTyping( + operation: ReplyOperation, + options: { startIfIdle: boolean }, +): Promise { + const typing = typingByReplyOperation.get(operation); + if (!typing || operation.result || (!options.startIfIdle && !typing.isActive())) { + return false; + } + await typing.startTypingLoop(); + typing.refreshTypingTtl(); + return true; +} diff --git a/src/channels/typing.test.ts b/src/channels/typing.test.ts index 7a0bd9424e93..58797915bfec 100644 --- a/src/channels/typing.test.ts +++ b/src/channels/typing.test.ts @@ -169,6 +169,56 @@ describe("createTypingCallbacks", () => { }); }); + it("preserves the existing keepalive cadence when an active reply starts again", async () => { + await withFakeTimers(async () => { + const { start, callbacks } = createTypingHarness({ keepaliveIntervalMs: 4_000 }); + + await callbacks.onReplyStart(); + await vi.advanceTimersByTimeAsync(3_000); + await callbacks.onReplyStart(); + expect(start).toHaveBeenCalledTimes(2); + + await vi.advanceTimersByTimeAsync(1_000); + + expect(start).toHaveBeenCalledTimes(3); + }); + }); + + it("keeps coalesced typing alive beyond 60 seconds while the same task refreshes it", async () => { + await withFakeTimers(async () => { + vi.setSystemTime(0); + const acceptedStarts: number[] = []; + const { callbacks } = createTypingHarness({ + keepaliveIntervalMs: 4_000, + maxDurationMs: 0, + start: async () => { + const now = Date.now(); + const previous = acceptedStarts.at(-1); + if (previous !== undefined && now - previous < 4_000) { + return; + } + acceptedStarts.push(now); + }, + }); + + await callbacks.onReplyStart(); + for (let elapsedMs = 6_000; elapsedMs <= 132_000; elapsedMs += 6_000) { + await vi.advanceTimersByTimeAsync(6_000); + await callbacks.onReplyStart(); + } + + expect(acceptedStarts.at(-1)).toBeGreaterThan(120_000); + for (let index = 1; index < acceptedStarts.length; index += 1) { + expect(acceptedStarts[index]! - acceptedStarts[index - 1]!).toBeLessThanOrEqual(4_000); + } + + callbacks.onIdle?.(); + const countAtTaskCompletion = acceptedStarts.length; + await vi.advanceTimersByTimeAsync(12_000); + expect(acceptedStarts).toHaveLength(countAtTaskCompletion); + }); + }); + it("stops keepalive after consecutive start failures", async () => { await withFakeTimers(async () => { const { start, onStartError, callbacks } = createTypingHarness({ diff --git a/src/channels/typing.ts b/src/channels/typing.ts index 784a98d9e533..91e0951baa62 100644 --- a/src/channels/typing.ts +++ b/src/channels/typing.ts @@ -97,13 +97,15 @@ export function createTypingCallbacks(params: CreateTypingCallbacksParams): Typi } stopSent = false; startGuard.reset(); - keepaliveLoop.stop(); clearTtlTimer(); const startPromise = fireStart(); void startPromise.then(() => { if (closed || startGuard.isTripped()) { return; } + // Core can refresh an active reply independently of this channel loop. + // Restarting the interval here shifts its deadline and can outlive a + // provider's visible typing window between consecutive renewals. keepaliveLoop.start(); startTtlTimer(); }); diff --git a/src/cli/plugins-location-bridges.test.ts b/src/cli/plugins-location-bridges.test.ts index 1ab8b5a7c718..d76637285bfc 100644 --- a/src/cli/plugins-location-bridges.test.ts +++ b/src/cli/plugins-location-bridges.test.ts @@ -165,6 +165,7 @@ describe("listPersistedBundledPluginLocationBridges", () => { }); it.each([ + ["synthetic", "@openclaw/synthetic-provider"], ["teams-meetings", "@openclaw/teams-meetings"], ["zoom-meetings", "@openclaw/zoom-meetings"], ])( diff --git a/src/gateway/server-http.mcp-app-admission.test.ts b/src/gateway/server-http.mcp-app-admission.test.ts new file mode 100644 index 000000000000..8b670969f883 --- /dev/null +++ b/src/gateway/server-http.mcp-app-admission.test.ts @@ -0,0 +1,166 @@ +// Proves standalone MCP App HTTP work participates in Gateway suspension admission. +import type { IncomingMessage, ServerResponse } from "node:http"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import type { OpenClawConfig } from "../config/types.openclaw.js"; +import { + getActiveGatewayRootWorkCount, + markGatewayRestartDraining, + resetGatewayWorkAdmission, + tryBeginGatewaySuspendAdmission, + waitForActiveGatewayRootWork, +} from "../process/gateway-work-admission.js"; + +const mocks = vi.hoisted(() => ({ + handleMcpAppStandaloneHttpRequest: vi.fn(), +})); + +vi.mock("./mcp-app-standalone.js", () => ({ + handleMcpAppStandaloneHttpRequest: mocks.handleMcpAppStandaloneHttpRequest, +})); + +import { + AUTH_NONE, + createRequest, + createResponse, + dispatchRequest, + withGatewayServer, +} from "./server-http.test-harness.js"; + +const MCP_APP_PATH = "/__openclaw__/mcp-app"; + +function deferred() { + let resolve = () => {}; + const promise = new Promise((done) => { + resolve = done; + }); + return { promise, resolve }; +} + +function mcpAppsConfig(): OpenClawConfig { + return { + gateway: { trustedProxies: [] }, + mcp: { apps: { enabled: true } }, + }; +} + +async function withMcpAppServer( + run: (server: Parameters[0]) => Promise, +): Promise { + await withGatewayServer({ + prefix: "mcp-app-http-admission", + resolvedAuth: AUTH_NONE, + overrides: { getRuntimeConfig: mcpAppsConfig }, + run, + }); +} + +beforeEach(() => { + resetGatewayWorkAdmission(); + mocks.handleMcpAppStandaloneHttpRequest.mockReset(); +}); + +afterEach(() => { + vi.restoreAllMocks(); + resetGatewayWorkAdmission(); +}); + +describe("standalone MCP App HTTP admission", () => { + it("rejects new requests with the canonical 503 after admission closes", async () => { + mocks.handleMcpAppStandaloneHttpRequest.mockImplementation( + (_req: IncomingMessage, res: ServerResponse) => { + res.statusCode = 204; + res.end(); + return true; + }, + ); + const suspension = tryBeginGatewaySuspendAdmission(() => {}); + expect(suspension?.commit()).toBe(true); + + await withMcpAppServer(async (server) => { + const response = createResponse(); + await dispatchRequest(server, createRequest({ path: MCP_APP_PATH }), response.res); + + expect(mocks.handleMcpAppStandaloneHttpRequest).not.toHaveBeenCalled(); + expect(response.res.statusCode).toBe(503); + expect(response.setHeader).toHaveBeenCalledWith("Retry-After", "1"); + expect(JSON.parse(response.getBody())).toMatchObject({ + error: { code: "gateway_unavailable" }, + }); + expect(getActiveGatewayRootWorkCount()).toBe(0); + }); + + expect(suspension?.release()).toBe(true); + }); + + it("keeps deferred handler work visible until it settles", async () => { + const started = deferred(); + const finish = deferred(); + mocks.handleMcpAppStandaloneHttpRequest.mockImplementation( + async (_req: IncomingMessage, res: ServerResponse) => { + started.resolve(); + await finish.promise; + res.statusCode = 200; + res.end("ok"); + return true; + }, + ); + + await withMcpAppServer(async (server) => { + const response = createResponse(); + const pending = dispatchRequest(server, createRequest({ path: MCP_APP_PATH }), response.res); + await started.promise; + + try { + expect(getActiveGatewayRootWorkCount()).toBe(1); + markGatewayRestartDraining(); + await expect(waitForActiveGatewayRootWork(0)).resolves.toEqual({ + drained: false, + active: 1, + }); + } finally { + finish.resolve(); + } + await pending; + expect(response.res.statusCode).toBe(200); + // The response mock resolves from res.end(), immediately before the + // admission wrapper's finally block releases the request root. + await vi.waitFor(() => expect(getActiveGatewayRootWorkCount()).toBe(0)); + await expect(waitForActiveGatewayRootWork(0)).resolves.toEqual({ + drained: true, + active: 0, + }); + }); + }); + + it("releases admission when the standalone handler fails", async () => { + const errorLog = vi.spyOn(console, "error").mockImplementation(() => {}); + mocks.handleMcpAppStandaloneHttpRequest.mockRejectedValue(new Error("standalone failed")); + + await withMcpAppServer(async (server) => { + const response = createResponse(); + await dispatchRequest(server, createRequest({ path: MCP_APP_PATH }), response.res); + + expect(response.res.statusCode).toBe(500); + expect(response.getBody()).toBe("Internal Server Error"); + expect(getActiveGatewayRootWorkCount()).toBe(0); + expect(errorLog).toHaveBeenCalledWith( + "[gateway-http] unhandled error in request handler:", + expect.objectContaining({ message: "standalone failed" }), + ); + }); + }); + + it("releases admission and preserves fallthrough when the handler declines", async () => { + mocks.handleMcpAppStandaloneHttpRequest.mockResolvedValue(false); + + await withMcpAppServer(async (server) => { + const response = createResponse(); + await dispatchRequest(server, createRequest({ path: MCP_APP_PATH }), response.res); + + expect(mocks.handleMcpAppStandaloneHttpRequest).toHaveBeenCalledOnce(); + expect(response.res.statusCode).toBe(404); + expect(response.getBody()).toBe("Not Found"); + expect(getActiveGatewayRootWorkCount()).toBe(0); + }); + }); +}); diff --git a/src/gateway/server-http.ts b/src/gateway/server-http.ts index 9439e68f621c..b1e98833b348 100644 --- a/src/gateway/server-http.ts +++ b/src/gateway/server-http.ts @@ -876,13 +876,14 @@ export function createGatewayHttpServer(opts: { if (configSnapshot.mcp?.apps?.enabled === true && isMcpAppStandalonePath(scopedRequestPath)) { requestStages.push({ name: "mcp-app-standalone", - run: async () => { - const standalone = await getMcpAppStandaloneModule(); - return await standalone.handleMcpAppStandaloneHttpRequest(req, res, { - sandboxPort: configSnapshot.mcp?.apps?.sandboxPort, - sandboxOrigin: configSnapshot.mcp?.apps?.sandboxOrigin, - }); - }, + run: async () => + await runWithGatewayHttpWorkAdmission(res, async () => { + const standalone = await getMcpAppStandaloneModule(); + return await standalone.handleMcpAppStandaloneHttpRequest(req, res, { + sandboxPort: configSnapshot.mcp?.apps?.sandboxPort, + sandboxOrigin: configSnapshot.mcp?.apps?.sandboxOrigin, + }); + }), }); } // Plugin routes run before the general Control UI SPA catch-all so diff --git a/src/gateway/session-lifecycle-state.test.ts b/src/gateway/session-lifecycle-state.test.ts index ee20ed09c265..8c40dfbb691f 100644 --- a/src/gateway/session-lifecycle-state.test.ts +++ b/src/gateway/session-lifecycle-state.test.ts @@ -397,6 +397,43 @@ describe("session lifecycle state", () => { expect(persisted.mainRestartRecovery).toBeUndefined(); }); + it("clears every generation of a resumed run when its current owner completes", async () => { + const lifecycleGeneration = getAgentEventLifecycleGeneration(); + const persisted = await persistLifecycle( + { + sessionId: "session-id", + updatedAt: 1_000, + startedAt: 1_050, + status: "running", + abortedLastRun: false, + restartRecoveryRuns: [ + { runId: "recovery-run", lifecycleGeneration: "pre-restart" }, + { runId: "recovery-run", lifecycleGeneration }, + ], + mainRestartRecovery: { + cycleId: "cycle-1", + revision: 5, + chargedAttempts: 2, + }, + }, + { + ts: 2_000, + sessionId: "session-id", + runId: "recovery-run", + lifecycleGeneration, + data: { phase: "end", endedAt: 1_800 }, + }, + ); + + expect(persisted).toMatchObject({ + status: "done", + endedAt: 1_800, + abortedLastRun: false, + }); + expect(persisted.restartRecoveryRuns).toBeUndefined(); + expect(persisted.mainRestartRecovery).toBeUndefined(); + }); + it("does not settle a foreground owner from a stale lifecycle generation", async () => { const persisted = await persistLifecycle( { diff --git a/src/media-understanding/apply.test.ts b/src/media-understanding/apply.test.ts index f1ee1da6843c..a3e42155acda 100644 --- a/src/media-understanding/apply.test.ts +++ b/src/media-understanding/apply.test.ts @@ -208,7 +208,6 @@ async function withMediaAutoDetectEnv( GROQ_API_KEY: undefined, DEEPGRAM_API_KEY: undefined, GEMINI_API_KEY: undefined, - OPENCLAW_ANTIGRAVITY_CLI: undefined, OPENCLAW_AGENT_DIR: undefined, ...env, }, @@ -1029,7 +1028,7 @@ describe("applyMediaUnderstanding", () => { expect(mockedRunExec).not.toHaveBeenCalled(); }); - it("uses Antigravity CLI as the last auto image fallback", async () => { + it("does not auto-detect Antigravity CLI for images", async () => { clearMediaUnderstandingBinaryCacheForTests(); const binDir = await createTempMediaDir(); await createMockExecutable(binDir, "agy"); @@ -1046,40 +1045,14 @@ describe("applyMediaUnderstanding", () => { source: "none", mode: "api-key", }); - mockedRunExec.mockImplementation(async (_command, args) => { - if (Array.isArray(args) && args.includes("--help")) { - return { stdout: "--print\n--add-dir\n--sandbox\n", stderr: "" }; - } - return { stdout: "antigravity image description\n", stderr: "" }; - }); await withMediaAutoDetectEnv({ PATH: binDir }, async () => { const result = await applyMediaUnderstanding({ ctx, cfg }); - expect(result.appliedImage).toBe(true); + expect(result.appliedImage).toBe(false); }); - expect(ctx.Body).toBe("[Image]\nDescription:\nantigravity image description"); - expect(mockedRunExec).toHaveBeenCalledTimes(2); - const realImagePath = await fs.realpath(imagePath); - const [_probeCommand, _probeArgs, probeOptions] = getRunExecCall(0); - expect(probeOptions).toEqual({ - timeoutMs: 3000, - cwd: expect.stringContaining("openclaw-antigravity-probe-"), - }); - const [command, args, options] = getRunExecCall(1); - expect(command).toBe(path.join(binDir, "agy")); - expect(args).toEqual([ - "--sandbox", - "--add-dir", - path.dirname(realImagePath), - "--print", - expect.stringContaining(realImagePath), - ]); - expect(options).toEqual({ - timeoutMs: 60_000, - maxBuffer: CLI_OUTPUT_MAX_BUFFER, - cwd: path.dirname(realImagePath), - }); + expect(ctx.Body).toBe(""); + expect(mockedRunExec).not.toHaveBeenCalled(); }); it("uses CLI image understanding and preserves caption for commands", async () => { diff --git a/src/media-understanding/runner.ts b/src/media-understanding/runner.ts index d1c9caaa7fa2..df28188cf9b0 100644 --- a/src/media-understanding/runner.ts +++ b/src/media-understanding/runner.ts @@ -1,8 +1,5 @@ // Media-understanding runner resolves providers/models, local roots, auth, and // per-capability execution decisions for message attachments. -import { constants as fsConstants } from "node:fs"; -import fs from "node:fs/promises"; -import os from "node:os"; import path from "node:path"; import { mergeInboundPathRoots } from "@openclaw/media-core/inbound-path-policy"; import { findNormalizedProviderValue } from "@openclaw/model-catalog-core/provider-id"; @@ -11,10 +8,7 @@ import { normalizeNullableString, normalizeOptionalString, } from "@openclaw/normalization-core/string-coerce"; -import { - normalizeStringEntries, - uniqueStrings, -} from "@openclaw/normalization-core/string-normalization"; +import { uniqueStrings } from "@openclaw/normalization-core/string-normalization"; import type { ActiveMediaModel } from "../../packages/media-understanding-common/src/active-model.js"; import { isMediaUnderstandingSkipError } from "../../packages/media-understanding-common/src/errors.js"; import { providerSupportsCapability } from "../../packages/media-understanding-common/src/provider-supports.js"; @@ -36,13 +30,10 @@ import type { MediaUnderstandingModelConfig, } from "../config/types.tools.js"; import { logVerbose, shouldLogVerbose } from "../globals.js"; -import { resolvePreferredOpenClawTmpDir } from "../infra/tmp-openclaw-dir.js"; import { logWarn } from "../logger.js"; import { resolveChannelInboundAttachmentRoots } from "../media/channel-inbound-roots.js"; import { getDefaultMediaLocalRoots } from "../media/local-roots.js"; import { normalizeMediaFacts } from "../media/media-facts.js"; -import { runExec } from "../process/exec.js"; -import { getOrCreatePromise } from "../shared/lazy-promise.js"; import { createLazyRuntimeModule, createLazyRuntimeNamedExport } from "../shared/lazy-runtime.js"; import { MediaAttachmentCache, selectAttachments } from "./attachments.js"; import { matchesMediaEntryCapability } from "./entry-capabilities.js"; @@ -357,12 +348,7 @@ export function resolveMediaAttachmentLocalRoots(params: { ); } -const binaryCache = new Map>(); -const antigravityCliCache = new Map>(); - function clearMediaUnderstandingBinaryCacheForTests(): void { - binaryCache.clear(); - antigravityCliCache.clear(); clearLocalAudioInspectionCacheForTests(); } @@ -372,152 +358,6 @@ if (process.env.VITEST || process.env.NODE_ENV === "test") { ] = { clearMediaUnderstandingBinaryCacheForTests }; } -function expandHomeDir(value: string): string { - if (!value.startsWith("~")) { - return value; - } - const home = os.homedir(); - if (value === "~") { - return home; - } - if (value.startsWith("~/")) { - return path.join(home, value.slice(2)); - } - return value; -} - -function hasPathSeparator(value: string): boolean { - return value.includes("/") || value.includes("\\"); -} - -function candidateBinaryNames(name: string): string[] { - if (process.platform !== "win32") { - return [name]; - } - const ext = path.extname(name); - if (ext) { - return [name]; - } - const pathext = normalizeStringEntries( - (process.env.PATHEXT ?? ".EXE;.CMD;.BAT;.COM").split(";"), - ).map((item) => (item.startsWith(".") ? item : `.${item}`)); - return [name, ...uniqueStrings(pathext).map((item) => `${name}${item}`)]; -} - -async function isExecutable(filePath: string): Promise { - try { - const stat = await fs.stat(filePath); - if (!stat.isFile()) { - return false; - } - if (process.platform === "win32") { - return true; - } - await fs.access(filePath, fsConstants.X_OK); - return true; - } catch { - return false; - } -} - -async function findBinary(name: string): Promise { - return await getOrCreatePromise(binaryCache, name, async () => { - const direct = expandHomeDir(name.trim()); - if (direct && hasPathSeparator(direct)) { - for (const candidate of candidateBinaryNames(direct)) { - if (await isExecutable(candidate)) { - return candidate; - } - } - } - - const searchName = name.trim(); - if (!searchName) { - return null; - } - const pathEntries = (process.env.PATH ?? "").split(path.delimiter); - const candidates = candidateBinaryNames(searchName); - for (const entryRaw of pathEntries) { - const entry = expandHomeDir(entryRaw.trim().replace(/^"(.*)"$/, "$1")); - if (!entry) { - continue; - } - for (const candidate of candidates) { - const fullPath = path.join(entry, candidate); - if (await isExecutable(fullPath)) { - return fullPath; - } - } - } - - return null; - }); -} - -async function probeAntigravityCliCandidate(command: string): Promise { - const resolved = await findBinary(command); - if (!resolved) { - return null; - } - const probeDir = await fs.mkdtemp( - path.join(resolvePreferredOpenClawTmpDir(), "openclaw-antigravity-probe-"), - ); - try { - const { stdout } = await runExec(resolved, ["--help"], { - timeoutMs: 3000, - cwd: probeDir, - }); - return stdout.includes("--print") && - stdout.includes("--add-dir") && - stdout.includes("--sandbox") - ? resolved - : null; - } catch { - return null; - } finally { - await fs.rm(probeDir, { recursive: true, force: true }).catch(() => {}); - } -} - -async function resolveAntigravityCliBinary(): Promise { - return await getOrCreatePromise(antigravityCliCache, "agy", async () => { - const configured = process.env.OPENCLAW_ANTIGRAVITY_CLI?.trim(); - const candidates = [configured, "agy", "antigravity"].filter((value): value is string => - Boolean(value), - ); - for (const candidate of candidates) { - const command = await probeAntigravityCliCandidate(candidate); - if (command) { - return command; - } - } - return null; - }); -} - -async function resolveAntigravityCliEntry( - capability: MediaUnderstandingCapability, -): Promise { - if (capability === "audio") { - return null; - } - const command = await resolveAntigravityCliBinary(); - if (!command) { - return null; - } - return { - type: "cli", - command, - args: [ - "--sandbox", - "--add-dir", - "{{AttachmentDir}}", - "--print", - "{{Prompt}} Inspect {{AttachmentPath}} and reply with only the requested media description.", - ], - }; -} - async function resolveKeyEntry(params: { cfg: OpenClawConfig; agentId?: string; @@ -762,10 +602,6 @@ async function resolveAutoEntries(params: { if (keys) { return [keys]; } - const antigravity = await resolveAntigravityCliEntry(params.capability); - if (antigravity) { - return [antigravity]; - } return []; } diff --git a/src/plugins/official-external-plugin-catalog.test.ts b/src/plugins/official-external-plugin-catalog.test.ts index 637755ebe992..72c2c481ba13 100644 --- a/src/plugins/official-external-plugin-catalog.test.ts +++ b/src/plugins/official-external-plugin-catalog.test.ts @@ -1960,6 +1960,18 @@ describe("official external plugin catalog", () => { }); }); + it("lists Synthetic as an official external provider", () => { + const synthetic = expectCatalogEntry("synthetic"); + + expect(resolveOfficialExternalPluginId(synthetic)).toBe("synthetic"); + expect(resolveOfficialExternalPluginInstall(synthetic)).toEqual({ + clawhubSpec: "clawhub:@openclaw/synthetic-provider", + npmSpec: "@openclaw/synthetic-provider", + defaultChoice: "npm", + minHostVersion: ">=2026.7.2", + }); + }); + it.each([ ["teams-meetings", "@openclaw/teams-meetings", "teams_meetings", "teams"], ["zoom-meetings", "@openclaw/zoom-meetings", "zoom_meetings", "zoom"], diff --git a/src/system-agent/setup-inference-detect.ts b/src/system-agent/setup-inference-detect.ts index 57a39ef59889..7686a758f8c9 100644 --- a/src/system-agent/setup-inference-detect.ts +++ b/src/system-agent/setup-inference-detect.ts @@ -114,20 +114,7 @@ export async function detectSetupInference( const unavailableCandidates: SetupInferenceUnavailableCandidate[] = []; const deferredUnavailableCandidates: SetupInferenceUnavailableCandidate[] = []; const probe = deps.probeLocalCommand ?? probeLocalCommand; - const [antigravity, pi, opencode] = await Promise.all([ - probe("agy"), - probe("pi"), - probe("opencode"), - ]); - if (antigravity.found && !antigravity.timedOut) { - deferredUnavailableCandidates.push({ - id: "antigravity-cli", - label: "Antigravity CLI", - detail: "installed", - reason: - "Can't be auto-tested safely here. Sign in with a provider or use an API key instead.", - }); - } + const [pi, opencode] = await Promise.all([probe("pi"), probe("opencode")]); if (pi.found && !pi.timedOut) { deferredUnavailableCandidates.push({ id: "pi-cli", @@ -176,40 +163,6 @@ export async function detectSetupInference( const manualProviders = listSetupInferenceManualProviders(authChoices); const authOptions = listSetupInferenceAuthOptions(authChoices); const prepareOptions = listSetupInferencePrepareOptions(authChoices); - const manualProviderIds = new Set(manualProviders.map((provider) => provider.id)); - const authOptionIds = new Set(authOptions.map((option) => option.id)); - // Gemini CLI has no hard tool-off mode: wildcard exclusions can be - // overridden by admin policy and do not stop discovery or MCP startup. - // Keep normal agent support, but route setup through provider-owned methods - // that OpenClaw can verify without inspecting Gemini's private auth store. - for (const candidate of detected.filter((entry) => entry.kind === "gemini-cli")) { - const providerId = parseRef(candidate.modelRef).provider; - const ownerChoice = authChoices.find( - (choice) => normalizeProviderId(choice.providerId) === normalizeProviderId(providerId), - ); - const ownerGroup = ownerChoice?.groupId ?? ownerChoice?.providerId ?? providerId; - const relatedChoices = authChoices.filter( - (choice) => (choice.groupId ?? choice.providerId) === ownerGroup, - ); - const authOptionId = relatedChoices.find((choice) => - authOptionIds.has(choice.choiceId), - )?.choiceId; - const manualProviderId = relatedChoices.find((choice) => - manualProviderIds.has(choice.choiceId), - )?.choiceId; - unavailableCandidates.push({ - id: candidate.kind, - brandId: providerId, - label: candidate.label, - detail: candidate.detail, - reason: - "OpenClaw cannot confirm whether this private Gemini CLI login works without starting a session that may expose tools. Sign in through OpenClaw or use a Gemini API key to create a connection it can verify.", - ...(authOptionId ? { authOptionId } : {}), - ...(manualProviderId ? { manualProviderId } : {}), - ...(ownerChoice?.icon ? { icon: ownerChoice.icon } : {}), - ...(ownerChoice?.website ? { website: ownerChoice.website } : {}), - }); - } unavailableCandidates.push(...deferredUnavailableCandidates); const candidates: SetupInferenceCandidate[] = raw.map((candidate) => // Released macOS clients require this field. Keep it false so the wire diff --git a/src/system-agent/setup-inference.test.ts b/src/system-agent/setup-inference.test.ts index 253fa6b82305..a79721917b72 100644 --- a/src/system-agent/setup-inference.test.ts +++ b/src/system-agent/setup-inference.test.ts @@ -482,7 +482,7 @@ describe("detectSetupInference", () => { ); }); - it("discovers provider-owned local inference and reports unsafe CLIs without running them", async () => { + it("discovers provider-owned local inference without surfacing unsupported Google CLIs", async () => { const prepare = vi.fn(); const detect = vi.fn(async () => ({ modelRef: "local/qwen-tool", @@ -521,25 +521,15 @@ describe("detectSetupInference", () => { ], probeLocalCommand: vi.fn(async (command) => ({ command, - found: command === "agy" || command === "pi" || command === "opencode", + found: command === "pi" || command === "opencode", })), resolveManifestProviderAuthChoices: () => [ - { - pluginId: "google", - providerId: "google-gemini-cli", - methodId: "oauth", - choiceId: "google-gemini-cli", - choiceLabel: "Gemini CLI OAuth", - groupId: "google", - groupLabel: "Google", - appGuidedAuth: "oauth", - }, { pluginId: "google", providerId: "google", methodId: "api-key", choiceId: "gemini-api-key", - choiceLabel: "Google Gemini API key", + choiceLabel: "Google AI Studio API key", groupId: "google", groupLabel: "Google", appGuidedSecret: true, @@ -574,13 +564,6 @@ describe("detectSetupInference", () => { }, ]); expect(detection.unavailableCandidates).toEqual([ - expect.objectContaining({ - id: "gemini-cli", - brandId: "google-gemini-cli", - authOptionId: "google-gemini-cli", - manualProviderId: "gemini-api-key", - }), - expect.objectContaining({ id: "antigravity-cli" }), expect.objectContaining({ id: "pi-cli" }), expect.objectContaining({ id: "opencode-cli" }), ]); @@ -975,7 +958,7 @@ describe("detectSetupInference", () => { ]); }); - it("omits Gemini CLI because setup verification cannot hard-disable its tools", async () => { + it("omits Gemini CLI instead of presenting an unverifiable setup route", async () => { vi.mocked(detectInferenceBackends).mockResolvedValueOnce([ { kind: "gemini-cli", @@ -1001,9 +984,7 @@ describe("detectSetupInference", () => { expect(detection.candidates).toEqual([ expect.objectContaining({ kind: "claude-cli", recommended: false }), ]); - expect(detection.unavailableCandidates).toEqual([ - expect.objectContaining({ id: "gemini-cli" }), - ]); + expect(detection.unavailableCandidates).toEqual([]); }); it("reports installed Pi and OpenCode without offering them as setup inference routes", async () => { @@ -1037,6 +1018,7 @@ describe("detectSetupInference", () => { ]); expect(probeLocalCommand).toHaveBeenCalledWith("pi"); expect(probeLocalCommand).toHaveBeenCalledWith("opencode"); + expect(probeLocalCommand).not.toHaveBeenCalledWith("agy"); }); }); diff --git a/test/scripts/bundled-plugin-build-entries.test.ts b/test/scripts/bundled-plugin-build-entries.test.ts index c144c7fb101a..bb18e4c1531c 100644 --- a/test/scripts/bundled-plugin-build-entries.test.ts +++ b/test/scripts/bundled-plugin-build-entries.test.ts @@ -359,6 +359,14 @@ describe("bundled plugin build entries", () => { } }); + it("excludes the externalized Synthetic provider from bundled artifacts", () => { + const entries = listBundledPluginBuildEntries(); + const artifacts = listBundledPluginPackArtifacts(); + + expectNoPrefixMatches(Object.keys(entries), "extensions/synthetic/"); + expectNoPrefixMatches(artifacts, "dist/extensions/synthetic/"); + }); + it("keeps bundled channel secret contracts on packed top-level sidecars", () => { const artifacts = listBundledPluginPackArtifacts(); const excludedPackageDirs = collectRootPackageExcludedExtensionDirs(); diff --git a/ui/src/e2e/browser-talk-start-stop.e2e.test.ts b/ui/src/e2e/browser-talk-start-stop.e2e.test.ts index 3f79d6d1b2ac..19ea6da0fe08 100644 --- a/ui/src/e2e/browser-talk-start-stop.e2e.test.ts +++ b/ui/src/e2e/browser-talk-start-stop.e2e.test.ts @@ -1,7 +1,5 @@ // Control UI E2E tests cover browser Talk start and stop through a real page. -import { mkdir } from "node:fs/promises"; -import path from "node:path"; -import { chromium, type Browser, type Page } from "playwright"; +import { chromium, type Browser } from "playwright"; import { afterAll, beforeAll, describe, expect, it } from "vitest"; import { canRunPlaywrightChromium, @@ -10,6 +8,14 @@ import { startControlUiE2eServer, type ControlUiE2eServer, } from "../test-helpers/control-ui-e2e.ts"; +import { + captureComposerProof, + captureVideoTalkProof, + installBlockedMicrophoneFixture, + installBlockedVideoTalkFixture, + installTalkBrowserFixtures, + videoTalkCatalog, +} from "./browser-talk-start-stop.fixtures.ts"; const chromiumExecutablePath = resolvePlaywrightChromiumExecutablePath(chromium.executablePath()); const chromiumAvailable = canRunPlaywrightChromium(chromiumExecutablePath); @@ -20,152 +26,6 @@ let server: ControlUiE2eServer; // Browser contexts preserve test isolation; keep one process warm for this file. let browser: Browser; -function videoTalkCatalog(activeProvider: "google" | "openai") { - return { - realtime: { - activeProvider, - providers: [{ id: activeProvider, label: activeProvider, supportsVideoFrames: true }], - }, - }; -} - -async function installTalkBrowserFixtures(page: Page) { - await page.addInitScript(() => { - type InputProcessor = { - onaudioprocess: - | ((event: { inputBuffer: { getChannelData: () => Float32Array } }) => void) - | null; - }; - const state = { - audioContextsClosed: 0, - tracksStopped: 0, - constraints: [] as unknown[], - inputProcessor: null as InputProcessor | null, - meterLevel: 0, - }; - const track = { stop: () => (state.tracksStopped += 1) }; - Object.defineProperty(navigator, "mediaDevices", { - configurable: true, - value: { - enumerateDevices: async () => [ - { kind: "audioinput", deviceId: "built-in", label: "Built-in Microphone" }, - { kind: "audioinput", deviceId: "usb", label: "USB Audio Interface" }, - { kind: "videoinput", deviceId: "camera", label: "Camera" }, - ], - getUserMedia: async (constraints: unknown) => { - state.constraints.push(constraints); - return { getTracks: () => [track] }; - }, - }, - }); - - class MockAudioContext { - readonly currentTime = 0; - readonly destination = {}; - readonly sampleRate: number; - - constructor(options?: { sampleRate?: number }) { - this.sampleRate = options?.sampleRate ?? 24_000; - } - - createMediaStreamSource() { - return { connect() {}, disconnect() {} }; - } - - createGain() { - return { connect() {}, disconnect() {}, gain: { value: 1 } }; - } - - createScriptProcessor() { - const processor = { connect() {}, disconnect() {}, onaudioprocess: null }; - state.inputProcessor = processor; - return processor; - } - - createAnalyser() { - return { - fftSize: 0, - smoothingTimeConstant: 0, - disconnect() {}, - getFloatTimeDomainData(samples: Float32Array) { - samples.fill(state.meterLevel); - }, - }; - } - - async close() { - state.audioContextsClosed += 1; - } - } - - Object.defineProperty(window, "AudioContext", { - configurable: true, - value: MockAudioContext, - }); - Object.defineProperty(window, "openclawTalkE2eState", { - configurable: true, - value: state, - }); - }); -} - -async function captureComposerProof(page: Page, fileName: string) { - const artifactDir = path.join(process.cwd(), ".artifacts", "control-ui-e2e", "voice-controls"); - await mkdir(artifactDir, { recursive: true }); - await page - .locator(".agent-chat__composer-shell") - .screenshot({ path: path.join(artifactDir, fileName) }); -} - -async function captureVideoTalkProof(page: Page, fileName: string) { - const artifactDir = path.join(process.cwd(), ".artifacts", "control-ui-e2e", "video-talk"); - await mkdir(artifactDir, { recursive: true }); - await page - .locator(".agent-chat__composer-shell") - .screenshot({ path: path.join(artifactDir, fileName) }); -} - -async function installBlockedMicrophoneFixture(page: Page) { - await page.addInitScript(() => { - Object.defineProperty(navigator, "mediaDevices", { - configurable: true, - value: { - enumerateDevices: async () => [], - getUserMedia: async () => { - throw new DOMException("Permission denied", "NotAllowedError"); - }, - }, - }); - }); -} - -async function installBlockedVideoTalkFixture(page: Page) { - await page.addInitScript(() => { - const getUserMedia = navigator.mediaDevices.getUserMedia.bind(navigator.mediaDevices); - Object.defineProperty(navigator, "mediaDevices", { - configurable: true, - value: { - getUserMedia: async (constraints: MediaStreamConstraints) => { - if (constraints.video) { - throw new DOMException("Permission denied", "NotAllowedError"); - } - return getUserMedia(constraints); - }, - }, - }); - class FakePeerConnection extends EventTarget { - connectionState = "new"; - close() { - this.connectionState = "closed"; - } - } - Object.defineProperty(window, "RTCPeerConnection", { - configurable: true, - value: FakePeerConnection, - }); - }); -} - describeControlUiE2e("Control UI browser Talk", () => { beforeAll(async () => { browser = await chromium.launch({ @@ -1004,6 +864,123 @@ describeControlUiE2e("Control UI browser Talk", () => { } }); + it("closes a stale relay when stop and restart race its create response", async () => { + const context = await browser.newContext({ locale: "en-US", permissions: ["microphone"] }); + const page = await context.newPage(); + const currentRelaySessionId = "relay-current-e2e"; + const staleRelaySessionId = "relay-stale-e2e"; + const gateway = await installMockGateway(page, { + methodResponses: { + "talk.client.create": { + provider: "openai", + transport: "gateway-relay", + relaySessionId: currentRelaySessionId, + audio: { + inputEncoding: "pcm16", + inputSampleRateHz: 16_000, + outputEncoding: "pcm16", + outputSampleRateHz: 24_000, + }, + }, + "talk.session.appendAudio": {}, + "talk.session.close": {}, + }, + }); + await installTalkBrowserFixtures(page); + + try { + await page.goto(`${server.baseUrl}chat`); + await gateway.deferNext("talk.client.create"); + + await page.getByRole("button", { name: "Start voice input" }).click(); + await expect + .poll(() => gateway.getRequests("talk.client.create").then((requests) => requests.length)) + .toBe(1); + await page.getByRole("button", { name: "Stop voice input" }).click(); + await page.getByRole("button", { name: "Start voice input" }).click(); + await expect + .poll(() => gateway.getRequests("talk.client.create").then((requests) => requests.length)) + .toBe(2); + + await expect + .poll(() => + page.evaluate( + () => + ( + window as Window & { + openclawTalkE2eState?: { constraints: unknown[] }; + } + ).openclawTalkE2eState?.constraints.length, + ), + ) + .toBe(1); + await gateway.emitGatewayEvent("talk.event", { + relaySessionId: currentRelaySessionId, + type: "ready", + }); + await expect + .poll(() => page.locator('.agent-chat__voice-activity[data-status="listening"]').count()) + .toBe(1); + + await gateway.resolveDeferred("talk.client.create", { + provider: "openai", + transport: "gateway-relay", + relaySessionId: staleRelaySessionId, + audio: { + inputEncoding: "pcm16", + inputSampleRateHz: 16_000, + outputEncoding: "pcm16", + outputSampleRateHz: 24_000, + }, + }); + await expect + .poll(() => gateway.getRequests("talk.session.close")) + .toEqual([ + expect.objectContaining({ + params: { sessionId: staleRelaySessionId }, + }), + ]); + + await page.evaluate(() => { + const state = ( + window as Window & { + openclawTalkE2eState?: { + inputProcessor?: { + onaudioprocess?: (event: { + inputBuffer: { getChannelData: () => Float32Array }; + }) => void; + }; + }; + } + ).openclawTalkE2eState; + state?.inputProcessor?.onaudioprocess?.({ + inputBuffer: { getChannelData: () => new Float32Array(4096).fill(0.1) }, + }); + }); + await expect + .poll(() => gateway.getRequests("talk.session.appendAudio")) + .toEqual([ + expect.objectContaining({ + params: expect.objectContaining({ sessionId: currentRelaySessionId }), + }), + ]); + await expect + .poll(() => page.getByRole("button", { name: "Stop voice input" }).isVisible()) + .toBe(true); + + await page.getByRole("button", { name: "Stop voice input" }).click(); + await expect + .poll(() => + gateway + .getRequests("talk.session.close") + .then((requests) => requests.map((request) => request.params)), + ) + .toEqual([{ sessionId: staleRelaySessionId }, { sessionId: currentRelaySessionId }]); + } finally { + await context.close(); + } + }); + it("keeps blocked microphone guidance readable in a narrow viewport", async () => { const context = await browser.newContext(); const page = await context.newPage(); diff --git a/ui/src/e2e/browser-talk-start-stop.fixtures.ts b/ui/src/e2e/browser-talk-start-stop.fixtures.ts new file mode 100644 index 000000000000..60a8baa3fbe0 --- /dev/null +++ b/ui/src/e2e/browser-talk-start-stop.fixtures.ts @@ -0,0 +1,149 @@ +import { mkdir } from "node:fs/promises"; +import path from "node:path"; +import type { Page } from "playwright"; + +export function videoTalkCatalog(activeProvider: "google" | "openai") { + return { + realtime: { + activeProvider, + providers: [{ id: activeProvider, label: activeProvider, supportsVideoFrames: true }], + }, + }; +} + +export async function installTalkBrowserFixtures(page: Page) { + await page.addInitScript(() => { + type InputProcessor = { + onaudioprocess: + | ((event: { inputBuffer: { getChannelData: () => Float32Array } }) => void) + | null; + }; + const state = { + audioContextsClosed: 0, + tracksStopped: 0, + constraints: [] as unknown[], + inputProcessor: null as InputProcessor | null, + meterLevel: 0, + }; + const track = { stop: () => (state.tracksStopped += 1) }; + Object.defineProperty(navigator, "mediaDevices", { + configurable: true, + value: { + enumerateDevices: async () => [ + { kind: "audioinput", deviceId: "built-in", label: "Built-in Microphone" }, + { kind: "audioinput", deviceId: "usb", label: "USB Audio Interface" }, + { kind: "videoinput", deviceId: "camera", label: "Camera" }, + ], + getUserMedia: async (constraints: unknown) => { + state.constraints.push(constraints); + return { getTracks: () => [track] }; + }, + }, + }); + + class MockAudioContext { + readonly currentTime = 0; + readonly destination = {}; + readonly sampleRate: number; + + constructor(options?: { sampleRate?: number }) { + this.sampleRate = options?.sampleRate ?? 24_000; + } + + createMediaStreamSource() { + return { connect() {}, disconnect() {} }; + } + + createGain() { + return { connect() {}, disconnect() {}, gain: { value: 1 } }; + } + + createScriptProcessor() { + const processor = { connect() {}, disconnect() {}, onaudioprocess: null }; + state.inputProcessor = processor; + return processor; + } + + createAnalyser() { + return { + fftSize: 0, + smoothingTimeConstant: 0, + disconnect() {}, + getFloatTimeDomainData(samples: Float32Array) { + samples.fill(state.meterLevel); + }, + }; + } + + async close() { + state.audioContextsClosed += 1; + } + } + + Object.defineProperty(window, "AudioContext", { + configurable: true, + value: MockAudioContext, + }); + Object.defineProperty(window, "openclawTalkE2eState", { + configurable: true, + value: state, + }); + }); +} + +export async function captureComposerProof(page: Page, fileName: string) { + const artifactDir = path.join(process.cwd(), ".artifacts", "control-ui-e2e", "voice-controls"); + await mkdir(artifactDir, { recursive: true }); + await page + .locator(".agent-chat__composer-shell") + .screenshot({ path: path.join(artifactDir, fileName) }); +} + +export async function captureVideoTalkProof(page: Page, fileName: string) { + const artifactDir = path.join(process.cwd(), ".artifacts", "control-ui-e2e", "video-talk"); + await mkdir(artifactDir, { recursive: true }); + await page + .locator(".agent-chat__composer-shell") + .screenshot({ path: path.join(artifactDir, fileName) }); +} + +export async function installBlockedMicrophoneFixture(page: Page) { + await page.addInitScript(() => { + Object.defineProperty(navigator, "mediaDevices", { + configurable: true, + value: { + enumerateDevices: async () => [], + getUserMedia: async () => { + throw new DOMException("Permission denied", "NotAllowedError"); + }, + }, + }); + }); +} + +export async function installBlockedVideoTalkFixture(page: Page) { + await page.addInitScript(() => { + const getUserMedia = navigator.mediaDevices.getUserMedia.bind(navigator.mediaDevices); + Object.defineProperty(navigator, "mediaDevices", { + configurable: true, + value: { + getUserMedia: async (constraints: MediaStreamConstraints) => { + if (constraints.video) { + throw new DOMException("Permission denied", "NotAllowedError"); + } + return getUserMedia(constraints); + }, + }, + }); + class FakePeerConnection extends EventTarget { + connectionState = "new"; + close() { + this.connectionState = "closed"; + } + } + Object.defineProperty(window, "RTCPeerConnection", { + configurable: true, + value: FakePeerConnection, + }); + }); +} diff --git a/ui/src/e2e/inference-setup-gate.e2e.test.ts b/ui/src/e2e/inference-setup-gate.e2e.test.ts index dffcb95ee624..0d24a94c5a99 100644 --- a/ui/src/e2e/inference-setup-gate.e2e.test.ts +++ b/ui/src/e2e/inference-setup-gate.e2e.test.ts @@ -24,6 +24,7 @@ async function captureProof(page: import("playwright").Page, fileName: string) { suite.define(() => { it("blocks empty chat home until a model is connected", async () => { const context = await suite.browser.newContext({ + colorScheme: "dark", locale: "en-US", serviceWorkers: "block", viewport: { height: 900, width: 1440 }, @@ -38,10 +39,10 @@ suite.define(() => { await expect.poll(() => page.locator(".agent-chat__composer-shell").count()).toBe(0); await expect.poll(() => page.locator("textarea").count()).toBe(0); await expect - .poll(() => page.getByRole("button", { name: "Configure a provider" }).count()) + .poll(() => page.getByRole("button", { name: "Connect an AI provider" }).count()) .toBe(1); await captureProof(page, "chat-home-desktop.png"); - await page.getByRole("button", { name: "Configure a provider" }).click(); + await page.getByRole("button", { name: "Connect an AI provider" }).click(); await expect.poll(() => new URL(page.url()).pathname).toBe("/settings/model-setup"); } finally { await context.close(); @@ -50,6 +51,7 @@ suite.define(() => { it("blocks the new-session composer until a model is connected", async () => { const context = await suite.browser.newContext({ + colorScheme: "dark", locale: "en-US", serviceWorkers: "block", viewport: { height: 900, width: 1440 }, @@ -64,7 +66,7 @@ suite.define(() => { await expect.poll(() => page.locator(".new-session-page__composer").count()).toBe(0); await expect.poll(() => page.locator("textarea").count()).toBe(0); await captureProof(page, "new-session-desktop.png"); - await page.getByRole("button", { name: "Configure a provider" }).click(); + await page.getByRole("button", { name: "Connect an AI provider" }).click(); await expect.poll(() => new URL(page.url()).pathname).toBe("/settings/model-setup"); } finally { await context.close(); @@ -73,6 +75,7 @@ suite.define(() => { it("shows the setup splash before starting custodian chat", async () => { const context = await suite.browser.newContext({ + colorScheme: "dark", locale: "en-US", serviceWorkers: "block", viewport: { height: 900, width: 1660 }, @@ -98,7 +101,7 @@ suite.define(() => { await page.setViewportSize({ height: 520, width: 900 }); await expect - .poll(() => page.getByRole("button", { name: "Configure a provider" }).isVisible()) + .poll(() => page.getByRole("button", { name: "Connect an AI provider" }).isVisible()) .toBe(true); await expect .poll(() => @@ -110,7 +113,7 @@ suite.define(() => { await captureProof(page, "custodian-short-window.png"); await page.setViewportSize({ height: 900, width: 1660 }); - await page.getByRole("button", { name: "Configure a provider" }).click(); + await page.getByRole("button", { name: "Connect an AI provider" }).click(); await expect.poll(() => new URL(page.url()).pathname).toBe("/settings/model-setup"); const modelsLink = page.locator('.settings-sidebar__item[href="/settings/model-providers"]'); await expect.poll(() => modelsLink.getAttribute("aria-current")).toBe("page"); @@ -122,6 +125,7 @@ suite.define(() => { it("distinguishes a configured provider that fails its live check", async () => { const context = await suite.browser.newContext({ + colorScheme: "dark", locale: "en-US", serviceWorkers: "block", viewport: { height: 900, width: 1660 }, @@ -156,17 +160,15 @@ suite.define(() => { message: "OpenClaw requires working inference: provider authentication failed", }); - await page - .getByRole("heading", { name: "OpenClaw couldn't use your configured AI" }) - .waitFor(); + await page.getByRole("heading", { name: "Configured AI needs attention" }).waitFor(); await expect.poll(() => page.locator(".agent-chat__composer-shell").count()).toBe(0); await expect - .poll(() => page.getByRole("button", { name: "Check provider settings" }).count()) + .poll(() => page.getByRole("button", { name: "Review connection" }).count()) .toBe(1); await expect.poll(() => page.getByRole("button", { name: "Retry" }).count()).toBe(1); await captureProof(page, "custodian-provider-unavailable.png"); - await page.getByRole("button", { name: "Check provider settings" }).click(); + await page.getByRole("button", { name: "Review connection" }).click(); await expect.poll(() => new URL(page.url()).pathname).toBe("/settings/model-setup"); const modelsLink = page.locator('.settings-sidebar__item[href="/settings/model-providers"]'); await expect.poll(() => modelsLink.getAttribute("aria-current")).toBe("page"); diff --git a/ui/src/e2e/model-providers.e2e.test.ts b/ui/src/e2e/model-providers.e2e.test.ts index b4954127ab02..1112fb1380ff 100644 --- a/ui/src/e2e/model-providers.e2e.test.ts +++ b/ui/src/e2e/model-providers.e2e.test.ts @@ -125,7 +125,9 @@ describeControlUiE2e("Control UI Models mocked Gateway E2E", () => { const openaiCard = page.locator('[data-provider-id="openai"]'); const readiness = page.locator('[data-model-readiness="model-required"]'); await readiness.waitFor(); - await expect.poll(async () => readiness.textContent()).toContain("Connect your AI"); + await expect + .poll(async () => readiness.textContent()) + .toContain("Connect a verified AI model"); await expect.poll(async () => readiness.textContent()).toContain("No models available"); await expect.poll(async () => openaiCard.textContent()).toContain("Signed in"); expect(await page.locator(".model-providers__defaults").count()).toBe(0); diff --git a/ui/src/e2e/model-setup.e2e.test.ts b/ui/src/e2e/model-setup.e2e.test.ts index 89fd21be0458..b39a14c51ad0 100644 --- a/ui/src/e2e/model-setup.e2e.test.ts +++ b/ui/src/e2e/model-setup.e2e.test.ts @@ -116,7 +116,7 @@ describeControlUiE2e("Control UI Model Setup mocked Gateway E2E", () => { try { const response = await page.goto(`${server.baseUrl}settings/model-setup?firstRun=1`); expect(response?.status()).toBe(200); - await page.getByRole("heading", { name: "Connect your AI" }).waitFor(); + await page.getByRole("heading", { name: "Connect a verified AI model" }).waitFor(); const candidate = page.locator('[data-candidate-kind="codex-cli"]'); await expect.poll(() => candidate.locator('[data-provider-icon="codex"]').count()).toBe(1); await candidate.getByRole("button", { name: "Test & use" }).click(); @@ -510,7 +510,7 @@ describeControlUiE2e("Control UI Model Setup mocked Gateway E2E", () => { } }); - it("turns an unverifiable Gemini CLI login into direct recovery actions", async () => { + it("offers supported Google setup in an accessible provider picker", async () => { const context = await browser.newContext({ colorScheme: "dark", locale: "en-US", @@ -524,24 +524,12 @@ describeControlUiE2e("Control UI Model Setup mocked Gateway E2E", () => { "chat.startup", "openclaw.setup.detect", "openclaw.setup.activate", - "openclaw.setup.auth.start", "openclaw.setup.prepare.start", ], methodResponses: { "openclaw.setup.detect": { candidates: [], - unavailableCandidates: [ - { - id: "gemini-cli", - brandId: "google-gemini-cli", - label: "Gemini CLI", - detail: "installed; login status unavailable", - reason: - "OpenClaw cannot confirm whether this private Gemini CLI login works without starting a session that may expose tools. Sign in through OpenClaw or use a Gemini API key to create a connection it can verify.", - authOptionId: "google-gemini-cli", - manualProviderId: "gemini-api-key", - }, - ], + unavailableCandidates: [], manualProviders: [ { id: "qwen-cn", @@ -567,20 +555,11 @@ describeControlUiE2e("Control UI Model Setup mocked Gateway E2E", () => { id: "gemini-api-key", brandId: "google", groupLabel: "Google", - label: "Google Gemini API key", - hint: "Use an AI Studio API key.", - }, - ], - authOptions: [ - { - id: "google-gemini-cli", - brandId: "google-gemini-cli", - label: "Gemini CLI OAuth", - groupLabel: "Google", - kind: "oauth", - featured: true, + label: "Google AI Studio API key", + hint: "Supported API-key access from aistudio.google.com/apikey", }, ], + authOptions: [], workspace: "/tmp/openclaw-e2e", setupComplete: false, }, @@ -590,20 +569,15 @@ describeControlUiE2e("Control UI Model Setup mocked Gateway E2E", () => { latencyMs: 412, lines: ["Model ready"], }, - "openclaw.setup.auth.start": { - sessionId: "gemini-oauth-session", - done: false, - status: "running", - }, }, }); try { const response = await page.goto(`${server.baseUrl}settings/model-setup`); expect(response?.status()).toBe(200); - await page.getByRole("heading", { name: "Found, but needs attention" }).waitFor(); - await page.getByRole("button", { name: "Sign in with Google" }).waitFor(); - await page.getByRole("button", { name: "Use API key" }).waitFor(); + await page.getByRole("heading", { name: "Connect a verified AI model" }).waitFor(); + await expect.poll(() => page.getByText("Gemini CLI OAuth").count()).toBe(0); + await expect.poll(() => page.getByText("Found, but needs attention").count()).toBe(0); const providerPicker = page.locator(".model-setup-provider-select"); const providerTrigger = providerPicker.locator(".model-setup-provider-select__trigger"); @@ -727,11 +701,16 @@ describeControlUiE2e("Control UI Model Setup mocked Gateway E2E", () => { .toBe(true); await accessValue.fill("sk-old-provider-secret"); - await page.getByRole("button", { name: "Use API key" }).click(); + await providerTrigger.click(); + await expect.poll(manualProviderMenuReady).toBe(true); + const googleProviderHidden = waitForProviderHide(); + await page.locator('[data-manual-provider="gemini-api-key"]').click(); + await googleProviderHidden; await expect.poll(() => providerTrigger.textContent()).toContain("Google"); + await expect.poll(() => providerTrigger.textContent()).toContain("AI Studio API key"); await expect.poll(() => accessValue.inputValue()).toBe(""); await expect - .poll(() => accessValue.evaluate((element) => element === document.activeElement)) + .poll(() => providerTrigger.evaluate((element) => element === document.activeElement)) .toBe(true); await providerTrigger.click(); @@ -793,21 +772,13 @@ describeControlUiE2e("Control UI Model Setup mocked Gateway E2E", () => { await expect .poll(async () => (await gateway.getRequests("openclaw.setup.detect")).length) .toBe(detectCountBeforeDismiss + 1); - await page.getByRole("button", { name: "Use API key" }).click(); + await providerTrigger.click(); + await expect.poll(manualProviderMenuReady).toBe(true); + const googleProviderHiddenAfterDismiss = waitForProviderHide(); + await page.locator('[data-manual-provider="gemini-api-key"]').click(); + await googleProviderHiddenAfterDismiss; await expect.poll(() => providerTrigger.textContent()).toContain("Google"); - - const detectCount = (await gateway.getRequests("openclaw.setup.detect")).length; - await page - .locator('[data-unavailable-candidate="gemini-cli"]') - .getByRole("button", { name: "Check again" }) - .click(); - await expect - .poll(async () => (await gateway.getRequests("openclaw.setup.detect")).length) - .toBe(detectCount + 1); - - await page.getByRole("button", { name: "Sign in with Google" }).click(); - const start = await gateway.waitForRequest("openclaw.setup.auth.start"); - expect(start.params).toMatchObject({ authChoice: "google-gemini-cli" }); + await expect.poll(() => page.getByText("Gemini CLI OAuth").count()).toBe(0); } finally { await context.close(); } diff --git a/ui/src/i18n/locales/en.ts b/ui/src/i18n/locales/en.ts index c3711e43ea81..544ff72a38ca 100644 --- a/ui/src/i18n/locales/en.ts +++ b/ui/src/i18n/locales/en.ts @@ -1968,18 +1968,18 @@ export const en: TranslationMap = { plugin: "Plugin-provided panel.", }, modelSetup: { - heading: "Connect your AI", + heading: "Connect a verified AI model", intro: - "OpenClaw reuses AI access you already have — a CLI login, an API key, or a provider sign-in.", + "OpenClaw checks the AI access available on this Gateway and verifies the exact model before it enables conversations.", required: { title: "No AI provider configured", - body: "OpenClaw couldn't find a provider and model configured for this agent. Add one before starting a conversation.", - action: "Configure a provider", + body: "We couldn't find a provider and model configured for this agent. Choose a supported connection; OpenClaw will test it before enabling chat.", + action: "Connect an AI provider", }, connectionFailure: { - title: "OpenClaw couldn't use your configured AI", - body: "This agent has a provider and model selected, but the connection failed. Check the provider login or API key, model access, and service status, then try again.", - action: "Check provider settings", + title: "Configured AI needs attention", + body: "OpenClaw found the provider and model selected for this agent, but the live check failed. Your configuration is still intact. Review the credential, model access, or provider status, then verify again.", + action: "Review connection", }, loading: "Checking this Gateway for available AI access…", retry: "Retry", @@ -3930,7 +3930,7 @@ export const en: TranslationMap = { }, readiness: { title: "AI setup", - heading: "Connect your AI", + heading: "Connect a verified AI model", signedInNoModels: "You're signed in, but this account exposes no usable models. Choose another provider or account to continue.", notConfigured: "Choose a provider and verify the model OpenClaw will use.", diff --git a/ui/src/pages/chat/chat-view.test.ts b/ui/src/pages/chat/chat-view.test.ts index cdad38310305..17fd51fc7b3b 100644 --- a/ui/src/pages/chat/chat-view.test.ts +++ b/ui/src/pages/chat/chat-view.test.ts @@ -4871,8 +4871,8 @@ describe("chat welcome", () => { canSend: false, disabledBanner: { kind: "composer-replacement", - text: "OpenClaw couldn't find a provider and model configured for this agent. Add one before starting a conversation.", - actionLabel: "Configure a provider", + text: "We couldn't find a provider and model configured for this agent. Choose a supported connection; OpenClaw will test it before enabling chat.", + actionLabel: "Connect an AI provider", onAction: () => undefined, }, modelSetupRequired: true, diff --git a/ui/src/pages/chat/realtime-talk.test.ts b/ui/src/pages/chat/realtime-talk.test.ts index 29f0b22f0b88..34133d9f5ef5 100644 --- a/ui/src/pages/chat/realtime-talk.test.ts +++ b/ui/src/pages/chat/realtime-talk.test.ts @@ -47,6 +47,14 @@ function transportContext(transport: object | undefined): RealtimeTalkTransportC return (transport as { ctx: RealtimeTalkTransportContext }).ctx; } +function createDeferred() { + let resolve!: (value: T) => void; + const promise = new Promise((resolvePromise) => { + resolve = resolvePromise; + }); + return { promise, resolve }; +} + describe("RealtimeTalkSession", () => { beforeEach(() => { googleStart.mockClear(); @@ -209,6 +217,97 @@ describe("RealtimeTalkSession", () => { expect(webRtcInstances).toHaveLength(0); }); + it("closes a Gateway relay allocated after the session stops", async () => { + const create = createDeferred<{ + provider: string; + transport: "gateway-relay"; + relaySessionId: string; + audio: { + inputEncoding: "pcm16"; + inputSampleRateHz: number; + outputEncoding: "pcm16"; + outputSampleRateHz: number; + }; + }>(); + const request = vi.fn((method: string) => { + if (method === "talk.client.create") { + return create.promise; + } + if (method === "talk.session.close") { + return Promise.resolve({ ok: true }); + } + throw new Error(`Unexpected request: ${method}`); + }); + const session = new RealtimeTalkSession({ request } as never, "main"); + + const starting = session.start(); + await vi.waitFor(() => + expect(request).toHaveBeenCalledWith("talk.client.create", expect.anything()), + ); + session.stop(); + create.resolve({ + provider: "openai", + transport: "gateway-relay", + relaySessionId: "relay-stale", + audio: { + inputEncoding: "pcm16", + inputSampleRateHz: 24_000, + outputEncoding: "pcm16", + outputSampleRateHz: 24_000, + }, + }); + await starting; + + expect(request).toHaveBeenCalledWith("talk.session.close", { sessionId: "relay-stale" }); + expect(relayInstances).toHaveLength(0); + }); + + it("closes a superseded client-owned allocation without replacing the active call", async () => { + const creates: Array>> = []; + const request = vi.fn((method: string) => { + if (method === "talk.client.create") { + const create = createDeferred(); + creates.push(create); + return create.promise; + } + if (method === "talk.client.close") { + return Promise.resolve({ ok: true }); + } + throw new Error(`Unexpected request: ${method}`); + }); + const session = new RealtimeTalkSession({ request } as never, "main"); + + const firstStart = session.start(); + await vi.waitFor(() => expect(creates).toHaveLength(1)); + session.stop(); + const secondStart = session.start(); + await vi.waitFor(() => expect(creates).toHaveLength(2)); + creates[1]!.resolve({ + provider: "openai", + transport: "webrtc", + voiceSessionId: "voice-current", + clientSecret: "secret", + }); + await secondStart; + creates[0]!.resolve({ + provider: "openai", + transport: "webrtc", + voiceSessionId: "voice-stale", + clientSecret: "secret", + }); + await firstStart; + + await vi.waitFor(() => + expect(request).toHaveBeenCalledWith("talk.client.close", { + sessionKey: "main", + voiceSessionId: "voice-stale", + }), + ); + expect(webRtcInstances).toHaveLength(1); + expect(webRtcStart).toHaveBeenCalledTimes(1); + session.stop(); + }); + it("falls back to talk.session.create when gateway-relay is rejected by talk.client.create", async () => { const request = vi .fn() diff --git a/ui/src/pages/chat/realtime-talk.ts b/ui/src/pages/chat/realtime-talk.ts index d4de594fa822..20633051545c 100644 --- a/ui/src/pages/chat/realtime-talk.ts +++ b/ui/src/pages/chat/realtime-talk.ts @@ -61,7 +61,7 @@ type RealtimeTalkLaunchTransport = NonNullable; }; @@ -137,6 +137,7 @@ function compactLaunchParams( export class RealtimeTalkSession { private transport: RealtimeTalkTransport | null = null; private closed = false; + private lifecycleGeneration = 0; private videoEnabled = false; private videoOperation = 0; private voiceSessionId: string | undefined; @@ -155,10 +156,11 @@ export class RealtimeTalkSession { ) {} async start(): Promise { + const lifecycleGeneration = ++this.lifecycleGeneration; this.closed = false; this.callbacks.onStatus?.("connecting"); const providerVideoCapable = await this.resolveVideoCapability(); - if (this.closed) { + if (this.closed || lifecycleGeneration !== this.lifecycleGeneration) { return; } // Declaring voice-transcript arms the server-side spoken-confirmation gate; @@ -182,16 +184,13 @@ export class RealtimeTalkSession { if (!voiceSessionId) { throw new Error("Realtime Talk session did not return a voice session id"); } + if (this.closed || lifecycleGeneration !== this.lifecycleGeneration) { + this.closeUnadoptedVoiceSession(voiceSessionId, transport); + return; + } this.voiceSessionId = voiceSessionId; this.acceptingTranscripts = true; this.serverOwnedVoiceSession = transport === "gateway-relay"; - if (this.closed) { - const detached = this.detachVoiceSession(); - if (detached) { - this.closeLogicalVoiceSession(detached); - } - return; - } this.transportGeneration += 1; const callbacks = transport === "gateway-relay" @@ -300,6 +299,7 @@ export class RealtimeTalkSession { } stop(): void { + this.lifecycleGeneration += 1; this.closed = true; this.videoOperation += 1; this.videoEnabled = false; @@ -313,6 +313,22 @@ export class RealtimeTalkSession { } } + private closeUnadoptedVoiceSession(voiceSessionId: string, transport: string): void { + // A stopped or superseded create still owns the allocation returned to it. + // Close at the provider boundary without installing a stale transport. + if (transport === "gateway-relay") { + void this.client + .request("talk.session.close", { sessionId: voiceSessionId }) + .catch(() => undefined); + return; + } + this.closeLogicalVoiceSession({ + voiceSessionId, + serverOwned: false, + transcriptWrites: Promise.resolve(), + }); + } + private clientOwnedTranscriptCallbacks( owningVoiceSessionId: string, owningGeneration: number, diff --git a/ui/src/pages/model-providers/view.test.ts b/ui/src/pages/model-providers/view.test.ts index 02718444dccb..f4aabefa6d04 100644 --- a/ui/src/pages/model-providers/view.test.ts +++ b/ui/src/pages/model-providers/view.test.ts @@ -230,7 +230,7 @@ describe("renderModelProviders", () => { ); const readiness = container.querySelector('[data-model-readiness="model-required"]'); - expect(text(readiness)).toContain("Connect your AI"); + expect(text(readiness)).toContain("Connect a verified AI model"); expect(text(readiness)).toContain("No models available"); expect(text(readiness)).toContain("Choose another provider"); expect(container.querySelector(".model-providers__defaults")).toBeNull(); @@ -269,7 +269,7 @@ describe("renderModelProviders", () => { const readiness = container.querySelector('[data-model-readiness="model-required"]'); expect(text(readiness)).toContain("Model required"); - expect(button(readiness!, "Connect your AI")).toBeDefined(); + expect(button(readiness!, "Connect a verified AI model")).toBeDefined(); expect(container.querySelector(".model-providers__defaults")).toBeNull(); }); diff --git a/ui/src/pages/model-setup/view.test.ts b/ui/src/pages/model-setup/view.test.ts index 7b8786c77b41..1c3836d383dc 100644 --- a/ui/src/pages/model-setup/view.test.ts +++ b/ui/src/pages/model-setup/view.test.ts @@ -23,13 +23,10 @@ const detected: SystemAgentSetupDetectResult = { ], unavailableCandidates: [ { - id: "gemini-cli", - brandId: "google-gemini-cli", - label: "Gemini CLI", - detail: "installed; login status unavailable", - reason: "OpenClaw could not confirm a usable login.", - authOptionId: "google-gemini-cli", - manualProviderId: "gemini-api-key", + id: "pi-cli", + label: "Pi", + detail: "installed; no setup route available", + reason: "This local runtime must be configured outside OpenClaw.", }, ], manualProviders: [ @@ -37,8 +34,8 @@ const detected: SystemAgentSetupDetectResult = { id: "gemini-api-key", brandId: "google", groupLabel: "Google", - label: "Google Gemini API key", - hint: "Use an AI Studio API key.", + label: "Google AI Studio API key", + hint: "Supported API-key access from aistudio.google.com/apikey", }, { id: "openai", @@ -50,15 +47,6 @@ const detected: SystemAgentSetupDetectResult = { }, ], authOptions: [ - { - id: "google-gemini-cli", - brandId: "google-gemini-cli", - label: "Gemini CLI OAuth", - groupLabel: "Google", - kind: "oauth", - featured: true, - hint: "Continue with Google.", - }, { id: "openai-oauth", brandId: "openai", @@ -199,12 +187,12 @@ describe("renderModelSetup", () => { it("renders candidate, unavailable, sign-in, and manual sections", () => { const container = mount(props()); - expect(text(container)).toContain("Connect your AI"); + expect(text(container)).toContain("Connect a verified AI model"); expect(text(container)).toContain("Found on this Gateway"); expect(text(container)).toContain("Codex CLI"); expect(text(container)).toContain("openai/gpt-5 · Signed in locally"); expect(text(container)).toContain("Found, but needs attention"); - expect(text(container)).toContain("OpenClaw could not confirm a usable login"); + expect(text(container)).toContain("This local runtime must be configured outside OpenClaw"); expect(text(container)).toContain("Sign in with a provider"); expect(text(container)).toContain("Run a model locally"); expect(text(container)).toContain("LM Studio"); @@ -217,9 +205,6 @@ describe("renderModelSetup", () => { ); expect(container.querySelector('input[type="password"]')).not.toBeNull(); expect(container.querySelector("details")?.open).toBe(false); - expect( - container.querySelector('[data-unavailable-candidate="gemini-cli"] [data-provider-icon]'), - ).not.toBeNull(); expect( container.querySelector('[data-candidate-kind="codex-cli"] [data-provider-icon="codex"]'), ).not.toBeNull(); @@ -490,26 +475,16 @@ describe("renderModelSetup", () => { expect(onSuccessClose).toHaveBeenCalledOnce(); }); - it("offers direct recovery actions for an unavailable provider", () => { - const onStartAuth = vi.fn(); - const onUseManualProvider = vi.fn(); + it("only rechecks unavailable runtimes without a supported setup route", () => { const onDetect = vi.fn(); - const container = mount(props({ onStartAuth, onUseManualProvider, onDetect })); + const container = mount(props({ onDetect })); const buttons = container.querySelectorAll( - '[data-unavailable-candidate="gemini-cli"] button', + '[data-unavailable-candidate="pi-cli"] button', ); - expect([...buttons].map((button) => button.textContent?.trim())).toEqual([ - "Sign in with Google", - "Use API key", - "Check again", - ]); + expect([...buttons].map((button) => button.textContent?.trim())).toEqual(["Check again"]); buttons[0]?.click(); - buttons[1]?.click(); - buttons[2]?.click(); - expect(onStartAuth).toHaveBeenCalledWith(expect.objectContaining({ id: "google-gemini-cli" })); - expect(onUseManualProvider).toHaveBeenCalledWith("gemini-api-key"); expect(onDetect).toHaveBeenCalledOnce(); });