diff --git a/docs/concepts/streaming.md b/docs/concepts/streaming.md index 189e126b337b..c7dcc5f488c4 100644 --- a/docs/concepts/streaming.md +++ b/docs/concepts/streaming.md @@ -54,16 +54,16 @@ Legend: ### Media delivery with block streaming -`MEDIA:` directives are normal delivery metadata. When block streaming sends a -media block early, OpenClaw remembers that delivery for the turn. If the final -assistant payload repeats the same media URL, the final delivery strips the -duplicate media instead of sending the attachment again. +Streaming media must use structured payload fields such as `mediaUrl` or +`mediaUrls`; streamed text is not parsed as an attachment command. When block +streaming sends media early, OpenClaw remembers that delivery for the turn. If +the final assistant payload repeats the same media URL, the final delivery +strips the duplicate media instead of sending the attachment again. Exact duplicate final payloads are suppressed. If the final payload adds distinct text around media that was already streamed, OpenClaw still sends the new text while keeping the media single-delivery. This prevents duplicate voice -notes or files on channels such as Telegram when an agent emits `MEDIA:` during -streaming and the provider also includes it in the completed reply. +notes or files on channels such as Telegram. ## Chunking algorithm (low/high bounds) diff --git a/docs/help/faq.md b/docs/help/faq.md index 558d797a6c9a..be5d9838ecff 100644 --- a/docs/help/faq.md +++ b/docs/help/faq.md @@ -1737,7 +1737,7 @@ lives on the [Models FAQ](/help/faq-models). - Outbound attachments from the agent must include a `MEDIA:` line (on its own line). See [OpenClaw assistant setup](/start/openclaw) and [Agent send](/tools/agent-send). + Outbound attachments from the agent must use structured media fields such as `media`, `mediaUrl`, `path`, or `filePath`. See [OpenClaw assistant setup](/start/openclaw) and [Agent send](/tools/agent-send). CLI sending: @@ -1750,7 +1750,7 @@ lives on the [Models FAQ](/help/faq-models). - The target channel supports outbound media and isn't blocked by allowlists. - The file is within the provider's size limits (images are resized to max 2048px). - `tools.fs.workspaceOnly=true` keeps local-path sends limited to workspace, temp/media-store, and sandbox-validated files. - - `tools.fs.workspaceOnly=false` lets `MEDIA:` send host-local files the agent can already read, but only for media plus safe document types (images, audio, video, PDF, and Office docs). Plain text and secret-like files are still blocked. + - `tools.fs.workspaceOnly=false` lets structured local media sends use host-local files the agent can already read, but only for media plus safe document types (images, audio, video, PDF, and Office docs). Plain text and secret-like files are still blocked. See [Images](/nodes/images). diff --git a/docs/nodes/camera.md b/docs/nodes/camera.md index 7ac293033cb0..7982ddf2579f 100644 --- a/docs/nodes/camera.md +++ b/docs/nodes/camera.md @@ -59,9 +59,9 @@ All camera access is gated behind **user-controlled settings**. Like `canvas.*`, the iOS node only allows `camera.*` commands in the **foreground**. Background invocations return `NODE_BACKGROUND_UNAVAILABLE`. -### CLI helper (temp files + MEDIA) +### CLI helper -The easiest way to get attachments is via the CLI helper, which writes decoded media to a temp file and prints `MEDIA:`. +The easiest way to get media files is via the CLI helper, which writes decoded media to a temp file and prints the saved path. Examples: @@ -126,12 +126,12 @@ Examples: ```bash openclaw nodes camera list --node # list camera ids -openclaw nodes camera snap --node # prints MEDIA: +openclaw nodes camera snap --node # prints saved path openclaw nodes camera snap --node --max-width 1280 openclaw nodes camera snap --node --delay-ms 2000 openclaw nodes camera snap --node --device-id -openclaw nodes camera clip --node --duration 10s # prints MEDIA: -openclaw nodes camera clip --node --duration-ms 3000 # prints MEDIA: (legacy flag) +openclaw nodes camera clip --node --duration 10s # prints saved path +openclaw nodes camera clip --node --duration-ms 3000 # prints saved path (legacy flag) openclaw nodes camera clip --node --device-id openclaw nodes camera clip --node --no-audio ``` @@ -152,7 +152,7 @@ Notes: For _screen_ video (not camera), use the macOS companion: ```bash -openclaw nodes screen record --node --duration 10s --fps 15 # prints MEDIA: +openclaw nodes screen record --node --duration 10s --fps 15 # prints saved path ``` Notes: diff --git a/docs/nodes/index.md b/docs/nodes/index.md index 047f5437acb8..7d4deac2343c 100644 --- a/docs/nodes/index.md +++ b/docs/nodes/index.md @@ -218,7 +218,7 @@ and approve the new request so the gateway stores the updated command snapshot. If the node is showing the Canvas (WebView), `canvas.snapshot` returns `{ format, base64 }`. -CLI helper (writes to a temp file and prints `MEDIA:`): +CLI helper (writes to a temp file and prints the saved path): ```bash openclaw nodes canvas snapshot --node --format png diff --git a/docs/reference/rich-output-protocol.md b/docs/reference/rich-output-protocol.md index b4a544273999..416c5d4c138f 100644 --- a/docs/reference/rich-output-protocol.md +++ b/docs/reference/rich-output-protocol.md @@ -1,59 +1,51 @@ --- -summary: "Rich output shortcode protocol for embeds, media, audio hints, and replies" +summary: "Rich output protocol for structured media, embeds, audio hints, and replies" read_when: - Changing assistant output rendering in the Control UI - - Debugging `[embed ...]`, `MEDIA:`, reply, or audio presentation directives + - Debugging `[embed ...]`, structured media, reply, or audio presentation directives title: "Rich output protocol" --- Assistant output can carry a small set of delivery/render directives: -- `MEDIA:` for attachment delivery +- structured `mediaUrl` / `mediaUrls` fields for attachment delivery - `[[audio_as_voice]]` for audio presentation hints - `[[reply_to_current]]` / `[[reply_to:]]` for reply metadata - `[embed ...]` for Control UI rich rendering -Remote `MEDIA:` attachments must be public `https:` URLs. Plain `http:`, +Remote media attachments must be public `https:` URLs. Plain `http:`, loopback, link-local, private, and internal hostnames are ignored as attachment directives; server-side media fetchers still enforce their own network guards. -Local `MEDIA:` attachments can use absolute paths, workspace-relative paths, or +Local media attachments can use absolute paths, workspace-relative paths, or home-relative `~/` paths. They still pass through the agent file-read policy and media type checks before delivery. -`MEDIA:` is parsed only as plain text. Wrapping the directive in Markdown -formatting (bold, inline code, fenced code) prevents the parser from -recognizing it, and the attachment is silently dropped from delivery. +Do not emit text commands for attachments from tools, plugins, streaming blocks, +browser output, or message actions. Use structured media fields instead. -Valid: +Valid message-tool payload: -```text -MEDIA:/workspace/image.png +```json +{ "message": "Here is your image.", "mediaUrl": "/workspace/image.png" } ``` -Invalid (parsed as prose, no attachment delivered): - -```text -**MEDIA:/workspace/image.png** -`MEDIA:/workspace/image.png` -Here is your image: MEDIA:/workspace/image.png -``` - -Keep `MEDIA:` on its own line, in plain text, with no surrounding formatting. +Legacy final assistant reply text may still be normalized for compatibility, but +it is not a general plugin/tool protocol. Plain Markdown image syntax stays text by default. Channels that intentionally map Markdown image replies to media attachments opt in at their outbound adapter; Telegram does this so `![alt](url)` can still become a media reply. -These directives are separate. `MEDIA:` and reply/voice tags remain delivery metadata; `[embed ...]` is the web-only rich render path. -Trusted tool-result media uses the same `MEDIA:` / `[[audio_as_voice]]` parser before delivery, so text tool outputs can still mark an audio attachment as a voice note. +These directives are separate. Structured media fields and reply/voice tags are +delivery metadata; `[embed ...]` is the web-only rich render path. -When block streaming is enabled, `MEDIA:` remains single-delivery metadata for a -turn. If the same media URL is sent in a streamed block and repeated in the final -assistant payload, OpenClaw delivers the attachment once and strips the duplicate -from the final payload. +When block streaming is enabled, media must be carried on structured payload +fields. If the same media URL is sent in a streamed block and repeated in the +final assistant payload, OpenClaw delivers the attachment once and strips the +duplicate from the final payload. ## `[embed ...]` @@ -72,7 +64,7 @@ Rules: - Only URL-backed embeds are rendered. Use `ref="..."` or `url="..."`. - Block-form inline HTML embed shortcodes are not rendered. - The web UI strips the shortcode from visible text and renders the embed inline. -- `MEDIA:` is not an embed alias and should not be used for rich embed rendering. +- Structured media is not an embed alias and should not be used for rich embed rendering. ## Stored rendering shape diff --git a/docs/start/openclaw.md b/docs/start/openclaw.md index ca10ac5f191a..71b007b81962 100644 --- a/docs/start/openclaw.md +++ b/docs/start/openclaw.md @@ -196,27 +196,21 @@ Inbound attachments (images/audio/docs) can be surfaced to your command via temp - `{{MediaUrl}}` (pseudo-URL) - `{{Transcript}}` (if audio transcription is enabled) -Outbound attachments from the agent: include `MEDIA:` on its own line (no spaces). The directive must start the line as plain text, outside code fences and without Markdown wrappers such as bold or inline code. Example: +Outbound attachments from the agent use structured media fields on the message tool or reply payload, such as `media`, `mediaUrl`, `mediaUrls`, `path`, or `filePath`. Example message-tool arguments: -``` -Here's the screenshot. -MEDIA:https://example.com/screenshot.png +```json +{ + "message": "Here's the screenshot.", + "mediaUrl": "https://example.com/screenshot.png" +} ``` -OpenClaw extracts these and sends them as media alongside the text. - -These forms are not attachment directives and are sent as normal text: - -```md -**MEDIA:https://example.com/screenshot.png** -`MEDIA:https://example.com/screenshot.png` -Here is the screenshot: MEDIA:https://example.com/screenshot.png -``` +OpenClaw sends structured media alongside the text. Legacy final assistant replies may still be normalized for compatibility, but tool output, browser output, streaming blocks, and message actions do not parse text as attachment commands. Local-path behavior follows the same file-read trust model as the agent: -- If `tools.fs.workspaceOnly` is `true`, outbound `MEDIA:` local paths stay restricted to the OpenClaw temp root, the media cache, agent workspace paths, and sandbox-generated files. -- If `tools.fs.workspaceOnly` is `false`, outbound `MEDIA:` can use host-local files the agent is already allowed to read. +- If `tools.fs.workspaceOnly` is `true`, outbound local media paths stay restricted to the OpenClaw temp root, the media cache, agent workspace paths, and sandbox-generated files. +- If `tools.fs.workspaceOnly` is `false`, outbound local media can use host-local files the agent is already allowed to read. - Local paths can be absolute, workspace-relative, or home-relative with `~/`. - Host-local sends still only allow media and safe document types (images, audio, video, PDF, and Office documents). Plain text and secret-like files are not treated as sendable media. diff --git a/docs/tools/browser-control.md b/docs/tools/browser-control.md index b5e583163fbc..164cb2ddc7ea 100644 --- a/docs/tools/browser-control.md +++ b/docs/tools/browser-control.md @@ -252,7 +252,7 @@ Snapshot flags at a glance: - `--format aria`: accessibility tree with `axN` refs. When Playwright is available, OpenClaw binds refs with backend DOM ids to the live page so follow-up actions can use them; otherwise treat the output as inspection-only. - `--efficient` (or `--mode efficient`): compact role snapshot preset. Set `browser.snapshotDefaults.mode: "efficient"` to make this the default (see [Gateway configuration](/gateway/configuration-reference#browser)). - `--interactive`, `--compact`, `--depth`, `--selector` force a role snapshot with `ref=e12` refs. `--frame "