* feat(canvas): let widget buttons send prompts into web chat
Inline show_widget documents get a sendPrompt(text) bridge: a private
MessageChannel offered to the Control UI before widget code runs. The chat
adopts only the first offer per frame, requires transient user activation,
a visible focused frame, plain text (no slash commands), and rate limits
10 prompts/minute per widget. Accepted prompts run the normal user send
path of the owning chat pane.
* fix(canvas): appease lint/knip/docs gates for widget prompts
Split widget prompt tests into their own file, drive validation and rate
limits through the real port path instead of exported internals, use
addEventListener on the adopted port, and regenerate docs_map.
* fix(canvas): install widget listeners per window for non-isolated tests
Module-boolean listener flags broke in shared vitest workers where each test
file gets a fresh jsdom window; key installation by window instead and give
port flushing more headroom.
* feat(linux): canvas via CLI-node + Tauri app IPC bridge
* refactor: extract gateway helper modules
* build(linux-canvas): register plugin package in lockfile
* fix(linux-canvas): move canvas advertise test out of core, regen docs/protocol/deadcode
* fix(gateway): break node-catalog/registry import cycle via leaf normalize module; add canvas glossary term
* style: oxfmt invoke.ts and runtime.ts after buildNodeEventParams extraction
* fix(linux): load Canvas WebView via dedicated data_directory context
Wry's Linux/WebKitGTK incognito mode discards Tauri's registered
WebContext (wry webkitgtk/mod.rs), so the Canvas window got a fresh
ephemeral context without the openclaw-canvas:// scheme handler — the
bundled A2UI page never committed (stayed about:blank) and every A2UI
command timed out. Use an isolated cache-backed data_directory instead,
which keeps the protocol handler while still isolating Canvas storage
from the dashboard window.
* fix(linux): keep Canvas WebView ephemeral via incognito + data_directory
Autoreview flagged that a dedicated data_directory alone persists Canvas
browser state (cookies, localStorage, IndexedDB, service workers) across
restarts, so an agent that navigates Canvas to a site could leak an
authenticated session into a later session. iOS uses a non-persistent
store; Linux should match.
Add .incognito(true) alongside .data_directory(): the distinct directory
gives Tauri a fresh WebContext key so it still attaches the
openclaw-canvas:// protocol closure, and incognito makes Wry swap in a
fresh *ephemeral* context carrying those protocols. Live-verified on a
Wayland/WebKitGTK box: the bundled page still loads
(location.href=openclaw-canvas://localhost/index.html, openclawA2UI
present, A2UI renders) and the canvas-webview dir holds no persistent
cookie/storage files.
* fix(extensions): make indexed access explicit across channel plugins
Transport-payload-safe burn-down: malformed Telegram/Discord/QQ/LINE
and sibling channel input keeps existing skip paths; no synthesized
fields, no new throws in delivery loops. Zalo escape sentinels preserve
literal matches instead of undefined replacements.
* fix(extensions): make indexed access explicit across provider and memory plugins
Stream and model iteration, tool-block guards, capture guards, and
sparse accumulators; singleton model reads carry named invariants.
* fix(extensions): make indexed access explicit across tooling plugins, flip the extensions lane
Remaining plugins (oc-path, qa-lab, browser, logbook, and siblings) plus
the tsconfig.extensions.json flag flip. Cleanup: logbook sampleFrames
NaN index at max=1, QA retry clamp at non-positive attempts, dead Canvas
probe and OpenShell no-op slice removed, twitch test setup leak excluded
from the prod lane.
* refactor(plugin-sdk): expose expectDefined via a focused SDK subpath
Extensions imported @openclaw/normalization-core directly, crossing the
external-plugin packaging boundary (it only worked because the runtime
builder bundles undeclared workspace helpers). expect-runtime joins the
canonical entrypoints JSON, generated exports, API baseline, docs, and
subpath contract test; all 78 extension imports now use the SDK seam.
Two scanner-shaped locals renamed for review-bundle hygiene.
* chore(plugin-sdk): raise surface budgets for the expect-runtime subpath
One new entrypoint with one callable export, added intentionally as the
packaging-honest seam for extension invariant helpers.
Adds client-capability-gated tool availability: gateway clients declare
capabilities at connect (new inline-widgets cap), chat.send stamps them into
the run context, and every tool assembly path (embedded runner, queued
followups, Codex app-server harness, plugin-only construction plans) drops
tools whose requiredClientCaps the originating client did not declare. The
Canvas plugin ships the first such tool, show_widget: agents pass SVG or an
HTML fragment plus a title; the plugin hosts it as a bounded, retention-scoped
Canvas document and returns the existing canvas preview handle, which web chat
renders as a sandboxed iframe fitted to the widget's reported content height.
Widget frames never get allow-same-origin (per-preview sandbox ceiling,
including the sidebar path) and the Canvas host serves widget documents with a
CSP sandbox header so direct navigation runs in an opaque origin. Verified
live end-to-end on a Testbox with gpt-5.5 (screenshots on the PR). CLI-backed
model backends do not carry client caps yet and stay fail-closed (#102577).
Closes#101790
* feat(browser): add optional vision understanding to screenshot tool
* fix(browser): wrap vision output as external content, enforce maxBytes, forward auth profiles
* fix(browser): remove no-op scope/attachments config, drop profile pass-through lacking runtime support
* feat(media-understanding): add profile/preferredProfile to DescribeImageFileWithModelParams and forward to describeImage
* style(browser): add curly braces to satisfy eslint curly rule
* fix(browser): correct tools.browser.enabled help text to match actual behavior
* fix(browser): thread agentDir/workspaceDir from plugin tool context into browser vision
* refactor(browser): move vision config from tools.browser to browser.models
The browser plugin's vision configuration now lives on the top-level
`browser` config namespace (browser.models, browser.visionEnabled,
browser.visionPrompt, etc.) instead of `tools.browser`. This aligns
with the plugin's existing config location and avoids confusion between
tool-level and plugin-level settings.
- Remove tools.browser from ToolsSchema and ToolsConfig
- Add models/vision* fields to BrowserConfig and its zod schema
- Update getBrowserVisionConfig to read from cfg.browser
- Update schema help, labels, and quality test
- Update vision.test.ts to use new config shape
* docs(browser): add screenshot vision configuration section
Document the new browser.models config for automatic screenshot
description via vision models, enabling text-only main models to
reason about web page content.
* fix(browser): remove deliverable media markers from vision result, drop unused import
P1: Vision-success path no longer exposes the raw screenshot as
deliverable media (removes MEDIA: line and details.media.mediaUrl).
This prevents channel delivery from auto-sending sensitive page content
when the intended output is a text description.
P2: Remove unused ToolsMediaUnderstandingSchema import that would fail
noUnusedLocals typecheck.
* fix(browser): add command/args fields to browser models schema
The browser vision model schema uses .strict(), so CLI-type entries
with command/args were rejected by TypeScript. Add these fields to
align with MediaUnderstandingModelSchema.
* chore(browser): remove debug console.log statements
* fix(browser): harden screenshot vision result against MEDIA: directive injection and restore image sanitization on failure fallback
ClawSweeper #84247 review round 2:
P1 (security, high): neutralize line-start MEDIA: directives in vision descriptions
before wrapping with wrapExternalContent. The agent media extractor scans every
browser tool-result text block via splitMediaFromOutput which treats line-start
MEDIA: as a trusted local-media delivery directive, and browser is on the
trusted-media allowlist. Without neutralization, page or vision-provider output
containing 'MEDIA:/tmp/secret.png' could synthesize a channel-deliverable media
artifact from untrusted content. wrapExternalContent itself does not strip
line-start directives. Introduce neutralizeMediaDirectives in vision.ts that
prepends '[neutralized] ' to any line whose trimStart() begins with MEDIA:
(case-insensitive), defanging the parser anchor while keeping the original
text human-readable.
P2 (compatibility): pass resolveRuntimeImageSanitization() to imageResultFromFile
in the vision-failure catch fallback. The non-vision screenshot path already
forwards this option (d5cc0d53b7) so configured agents.defaults.imageMaxDimensionPx
takes effect. Without this fix, any provider timeout/error silently bypasses the
sanitization guard and returns a raw full-resolution screenshot.
Regression coverage:
- vision.test.ts: 6 unit cases for neutralizeMediaDirectives (no-op fast path,
mid-line MEDIA: untouched, line-start defanged, leading-whitespace defanged,
case-insensitive, multiple directives per blob).
- browser-tool.test.ts: 2 integration cases that drive the full screenshot
tool execute path:
- 'neutralizes MEDIA: directives in vision text and does not attach media'
asserts no line matches /^\s*MEDIA:/i in returned text, secret path text
is preserved verbatim, details.media is absent, and imageResultFromFile
is not called on the success path.
- 'preserves screenshot image sanitization on vision failure fallback'
mocks describeImageFileWithModel to reject and asserts the fallback
imageResultFromFile call receives imageSanitization: {maxDimensionPx:1600}
plus the 'browser screenshot vision failed' extraText.
* fix(browser): apply clawsweeper fallback media fix from PR #84247
* refactor: reuse media image understanding for browser screenshots
* refactor: use structured media delivery
* test: update music completion media instruction expectation
* fix: trim buffered reply directive padding
* test: refresh codex prompt snapshots for message media aliases
---------
Co-authored-by: scotthuang <scotthuang@tencent.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Peter Steinberger