Commit Graph

57 Commits

Author SHA1 Message Date
Mariano edb7a1692e fix(telegram): expose live location updates to hooks (#122185)
* fix(telegram): expose live location updates to hooks

* test(telegram): keep location coverage focused

* fix(telegram): emit stopped live locations

* test(plugin-sdk): refresh location hook contracts

* docs(changelog): note Telegram live location hooks

* fix(telegram): complete live-location observer path

---------

Co-authored-by: mbelinky <mbelinky@users.noreply.github.com>
2026-08-11 23:26:23 +02:00
Peter Steinberger b5d5ec340f feat(cloud-workers): add desktop apps and browser autonomy (#121475)
* feat(cloud-workers): add desktop apps and browser autonomy

provider-attested Browser/Terminal launchers, shared visible loopback CDP Browser tool, no MCP/cookies/generic command.

* feat(ui): add cloud desktop app launcher

* docs(gateway): document cloud desktop apps and browser autonomy

* perf(ui): trim desktop launcher startup copy

* refactor(ui): simplify desktop launch feedback

* perf(ui): reuse desktop app labels

* fix(ui): keep desktop launch failures actionable

* fix(crabbox): allow browser bootstrap to finish

* fix(cloud-workers): honor provider provision budgets

* fix(cloud-workers): persist browser screenshot receipts

* fix(cloud-workers): bound browser screenshot lifecycle

* fix(cloud-workers): avoid replaying desktop launches

* test(cloud-workers): isolate browser runtime integration

* refactor(cloud-workers): tighten desktop runtime boundaries

* test(cloud-workers): keep browser runtime mock synchronous

* fix(cloud-workers): break gateway type import cycle

* fix(ci): settle admitted setup sessions in tests

* build(plugin-sdk): refresh desktop app contract

* ci: refresh merge-tree validation

* build(plugin-sdk): regenerate desktop app baseline

* style(gateway): format merged method order test
2026-08-10 20:31:07 -07:00
Peter Steinberger 8fdf7570a1 feat(gateway): live Desktop observer for cloud workers (Labs) (#120727)
* feat(gateway): live desktop observer for cloud workers

Adds live observation for cloud worker desktops through the gateway and Crabbox plugin, including desktop provisioning, persisted desktop metadata, tunneled WebSocket proxying, and the worker.desktop.observe protocol method.

The gateway, Crabbox plugin, and gateway protocol surfaces remain off by default behind the cloudWorkers.desktop Labs flag.

* feat(ui): Desktop panel for cloud worker observation

* docs(gateway): document cloud worker desktop lab

* fix(ci): regenerate contract baselines after rebase

* fix(protocol): regenerate Android gateway methods

* fix(ci): align rebased SDK and lint baselines

* fix(gateway): enforce view-only RFB boundary and fence desktop teardown

* fix(gateway): tighten RFB filter surface

* fix(state): keep pre-desktop databases readable and harden view-only RFB

* fix(gateway): fence desktop observer upgrades behind work admission

* fix(gateway): bind desktop observer tokens to their owner epoch

* fix(ci): regenerate config and SDK baselines after rebase

* fix(ci): regenerate native protocol and SDK baselines

* fix(ci): regenerate contracts after main rebase

* fix(state): register desktop metadata as lazy additive

* fix(ci): regenerate SDK baseline after final direct-merge rebase
2026-08-09 09:37:01 -07:00
Peter Steinberger bb799aec0f refactor(plugin-sdk): simplify plugin entry contracts (#120930)
* refactor(plugin-sdk): simplify plugin entry contracts

* fix(plugin-sdk): keep suppression context barrel live
2026-08-08 23:24:44 -07:00
Peter Steinberger 7f3cc6995b refactor(plugin-sdk): collapse plugin entry type aliases (#120814)
* refactor(plugin-sdk): collapse plugin entry type aliases

* chore(plugin-sdk): refresh plugin entry API baseline
2026-08-08 21:33:29 -07:00
Vincent Koc 14940edf15 feat(skills): add Skill Workshop lifecycle hooks (#115606)
* feat(skills): add lifecycle hook contracts

* feat(plugins): expose skill hook contracts

* feat(plugins): identify skill evaluators

* feat(skills): persist proposal evaluation lifecycle

* feat(skills): add agent evaluation action

* feat(skills): emit committed skill lifecycle changes

* feat(gateway): expose skill proposal evaluation lifecycle

* feat(ui): add Skill Workshop evaluations

* fix(skills): bind lifecycle state to proposal revisions

* fix(skills): preserve lifecycle events without artifacts

* feat(cli): evaluate skill proposals

* fix(ui): bind evaluations to proposal revisions

* docs(skills): document lifecycle hook primitives

* chore(plugin-sdk): refresh skill hook surface

* fix(skills): harden proposal evaluator execution

* fix(plugins): isolate skill evaluator inputs

* fix(cli): align skill lifecycle deadlines

* fix(skills): preserve evaluation replay invariants

* test(ui): capture Skill Workshop evaluation proof

* fix(skills): bind apply to evaluated target tree

* fix(skills): preserve evaluation contract edges

* fix(skills): bound evaluation event storage

* chore(skills): keep lifecycle helpers internal

* refactor(skills): isolate evaluation persistence

* fix(skills): satisfy lifecycle validation gates

* chore(protocol): refresh Skill Workshop clients

* docs: refresh Skill Workshop map

* chore: keep release notes in PR metadata

* docs: refresh merged docs map

* fix(ci): type Code Mode catch errors

* fix(skills): freeze lifecycle observation payloads

* fix(protocol): keep proposal inspect backward-decodable

* fix(skills): enforce final evaluator bundle limits

* fix(skills): preserve lifecycle caller attribution

* chore: drop subsumed Code Mode formatting

* test(plugins): adapt lifecycle hook mocks
2026-07-29 15:49:44 +08:00
Peter Steinberger 1e9d918037 feat(sdk): always persist media facts and ship facts-first replacements for legacy Media* surfaces (#113355)
* feat(sdk): always persist media facts and ship facts-first replacements for legacy Media* surfaces

PR 1 of the media legacy retirement program (audit-frozen, 4 PRs).

- Every media-bearing user turn now persists normalized __openclaw.media
  facts unconditionally while continuing to emit the legacy top-level
  Media* projection byte-identically (dual-write bridge; the conditional
  shouldPersistStructuredMediaEntries gate now always includes media).
- New replacement APIs, shipped before any removal: typed hook media
  facts (media[], originalMedia[], mediaStagingPending) on message
  events; {{AttachmentPath}}/{{AttachmentUrl}}/{{AttachmentContentType}}/
  {{AttachmentDir}}/{{AttachmentIndex}} template variables; focused
  openclaw/plugin-sdk/media-local-roots subpath split out of the
  deprecated agent-media-payload facade.
- Every legacy surface carries @deprecated naming its replacement, under
  one named compatibility record media-legacy-projection with the
  operator-approved removeAfter 2026-10-01 (two release trains; deletion
  additionally gates on a clean published-plugin artifact sweep).
- Generic transcript append invariant documented; SDK migration, hooks,
  and configuration docs updated to the facts-first path.

Writer golden matrix proves legacy bytes and model prompt bytes are
unchanged while nested facts become unconditional. 2,189 broad media
tests green; SDK api-baseline regenerated on fresh-env Testbox.

* feat(sdk): register media-local-roots subpath exports and deprecation metadata

Completes PR 1: package export map for openclaw/plugin-sdk/media-local-roots
plus the deprecated-subpath inventory and doc metadata entries for the
media-legacy-projection record.

* chore(sdk): track media-local-roots entrypoint and deprecated-export budgets

* fix(sdk): keep deprecated MSTeams buildMediaPayload re-export through the compat window

Deleting shipped runtime-api re-exports belongs to retirement PR 4 after
the media-legacy-projection window; PR 1 only deprecates. Also formats
the migration-guide schedule table.

* docs: regenerate docs map for media migration additions
2026-07-24 10:42:17 -07:00
Peter Steinberger c7e7ac2728 refactor: remove expired plugin compatibility surfaces (#111451)
* docs(secrets): remove retired web credential paths

* refactor(web): remove retired provider compatibility paths

* refactor(providers): delete retired compatibility routes

* refactor(secrets): remove retired credential aliases

* refactor(plugin-sdk): delete retired compatibility surfaces

* docs(plugin-sdk): remove retired migration guidance

* chore(plugin-sdk): refresh rebased surface budgets

* chore(plugin-sdk): refresh API removal baseline

* refactor(compat): migrate retired internal callers

* chore(plugin-sdk): refresh current-main baselines

* test(config): migrate plugin-owned secret assertions

* test(gateway): narrow plugin secret refs

* fix(plugin-sdk): preserve private boundary type identity

* chore(compat): remove stale sweep references

* chore(lint): lower max-lines budget

* refactor(secrets): remove unused web helper

* build(plugin-sdk): drop removed compat entries

* chore(plugin-sdk): refresh rebased API baseline

* chore(plugin-sdk): use Linux API baseline hash

* fix(plugin-sdk): preserve private bundled build entries

* fix(plugin-sdk): package private runtime facades

* fix(plugins): preserve external credential contracts
2026-07-19 11:04:48 -07:00
Peter Steinberger dde90a345a refactor(plugin-sdk): narrow wildcard barrels to explicit used exports (#108440)
* refactor(plugin-sdk): narrow wildcard barrels to explicit used exports

* refactor(tools): delete dead tool-planning module exposed by barrel narrowing

* fix(plugin-sdk): restore deprecation tag on OpenClawSchemaType alias

* test(agents): drop test for deleted runtime proxy module

* refactor(tools): trim descriptor types to cache consumers

* refactor(deadcode): harvest exports orphaned by barrel narrowing

* refactor(deadcode): harvest exports orphaned by barrel narrowing (rest)

* fix(agents): restore sdk imports and test markers via public predicate

* fix(plugin-sdk): named type re-exports in plugin-entry; trim types barrel precisely

* chore(plugin-sdk): account unmasked deprecated provider types in budgets

* fix(plugins): name star-only type rows for dts bundling

* fix(plugins): restore host-hook surface; unexport internal api compositions

* fix(plugins): named type imports for api composition; restore needed source exports

* fix(plugins): knip-visible type imports for registry surfaces

* test: adapt tests to privatized media and command internals

* fix(qa-lab): re-export snapshot conversation type

* style: format sessions sdk imports

* fix(plugins): restore smoke entry export; pin budgets to exact actuals

* fix(plugins): canonical smoke-entry import; drop orphaned root shims

* fix(plugins): allowlist manifest probe, repoint qa web import, drop dead browser barrels

* fix(plugin-sdk): pin codex auth marker and scaffold provider type

* fix(qa-lab): keep web-facing model-selection shim within boundary rules

* fix(plugin-sdk): preserve merged contracts through narrowed barrels

* chore(plugin-sdk): pin post-rebase surface budgets
2026-07-16 00:45:23 -07:00
Peter Steinberger da59d6ade9 feat(onboarding): detect local inference providers (#108605)
* feat(onboarding): detect local inference providers

* fix(onboarding): satisfy inference choice lint

* chore(i18n): refresh onboarding locales

* chore: leave changelog to release automation

* chore(plugin-sdk): refresh API baseline

* fix(onboarding): preserve source config during local activation
2026-07-15 22:46:16 -07:00
Peter Steinberger 9d79d85f46 feat(workboard): refresh Control UI from live changes (#99051)
* feat(workboard): refresh control ui from live changes

* refactor(workboard): tighten live invalidation path

* chore(workboard): preserve locale catalog state

* refactor(gateway): keep broadcaster runtime object

* refactor(plugin-sdk): own gateway event types

* test(workboard): bind reconciliation mocks

* test(release): declare recovery helpers

* fix(workboard): cancel direct loads on teardown
2026-07-14 05:17:23 -07:00
Peter Steinberger 5ca46a6554 feat(cloud-workers): pinned SSH tunnel runtime and provider-owned key resolution (#104553)
* feat(cloud-workers): add pinned SSH tunnel runtime

* feat(crabbox): resolve cloud worker SSH identities

* docs(cloud-workers): document SSH tunnel contracts
2026-07-11 09:26:08 -07:00
Peter Steinberger 90e465833b feat(gateway): durable cloud worker environments, provider SDK contract, and lifecycle RPCs (#104401)
* docs(plan): add cloud workers design plan

* feat(plugin-sdk): add cloud worker provider foundations

* feat(protocol): add worker environment lifecycle shapes

* feat(gateway): persist worker environment lifecycles

* test(gateway): pin environment inventory assertion for damaged worker store

* style(cloud-workers): satisfy lint and docs formatting

* fix(gateway): narrow worker environment type boundaries

* chore(plugin-sdk): account for WorkerProvider surface growth

* docs: regenerate docs map
2026-07-11 04:54:27 -07:00
Peter Steinberger e1934d968e fix(nodes): stop advertising a disabled browser proxy (#103894)
* fix(nodes): hide disabled browser proxy capability

* chore: defer node fix changelog to release

* chore: ratchet plugin SDK surface budget
2026-07-10 20:06:53 +01:00
Peter Steinberger 08ae0e6d29 refactor: store Zalo hosted media in plugin state
Move Zalo hosted outbound media metadata and expiry into plugin state, add SDK chunked hosted media storage, and keep CI/type/lint gates green after rebase.
2026-06-06 22:56:48 -07:00
Peter Steinberger a16c6ca94b docs: document plugin sdk public helpers 2026-06-04 19:14:41 -04:00
zhang-guiping b6e9473e9f fix(auth): skip Anthropic API keys for usage status
Fixes #85124.

Anthropic standard API keys no longer resolve as provider usage auth for `openclaw status --usage`, so valid inference keys are not sent to Anthropic's OAuth usage endpoint and surfaced as misleading invalid bearer-token errors.

The provider usage-auth SDK result now has an explicit handled/no-token shape so provider hooks can suppress generic fallback without widening the OAuth helper contract. Docs, Plugin SDK API baseline, and extension package-boundary cache inputs were updated with the new contract.

Thanks @zhangguiping-xydt.

Proof:
- node scripts/run-vitest.mjs src/infra/provider-usage.auth.normalizes-keys.test.ts src/infra/provider-usage.auth.plugin.test.ts extensions/anthropic/index.test.ts
- pnpm plugin-sdk:api:check
- pnpm plugin-sdk:check-exports
- git diff --check origin/main...HEAD
- pnpm docs:list
- pnpm run test:extensions:package-boundary:compile
- autoreview clean: no accepted/actionable findings
- PR CI rollup green: 131 success, 22 skipped, 1 neutral, 0 failures

Co-authored-by: 张贵萍0668001030 <zhang.guiping@xydigit.com>
2026-05-31 18:26:03 +01:00
Peter Steinberger b374505e7a refactor: source model catalog types from core
Source model catalog SDK types from @openclaw/model-catalog-core while preserving released compat fields and sanitized routing normalization.
2026-05-30 22:00:51 +01:00
Peter Steinberger cac0b2db18 refactor: move transcripts into core
Move meeting notes into core transcripts, remove the bundled meeting-notes plugin/API, and require explicit transcripts.enabled before exposing the recording-capable tool.
2026-05-26 14:51:11 +01:00
Peter Steinberger 07694c639d feat: add meeting notes plugin
Adds source-only external meeting notes plugin, SDK source-provider contract, CLI access, date-sharded storage, and Discord voice source.
2026-05-23 10:38:09 +01:00
Eva 2a0350b5b4 Separate prompt surfaces by selected harness (#83454)
* fix: scope agent prompt surfaces

* fix(codex): preserve lightweight project doc suppression

* fix(codex): demote openclaw context for native turns

* fix(codex): report demoted prompt context

* fix(codex): align demoted prompt observability

* docs: format codex runtime table

* docs: align codex prompt overlay docs

* test: align codex prompt snapshots

* test: update prompt snapshot contract

---------

Co-authored-by: Eva (agent) <eva+agent-78055@100yen.org>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-05-18 13:00:53 +01:00
Eva 50ee68cdf4 feat(plugin-sdk): consolidate session workflow APIs 2026-05-11 03:24:08 +01:00
Shakker 01741f81f8 test: remove stale unused imports 2026-05-09 11:26:43 +01:00
Peter Steinberger 311e4608d1 feat: unify model catalog registration 2026-05-09 02:34:56 -04:00
Peter Steinberger a3564ae546 perf: optimize plugin schema validation 2026-05-02 16:15:45 +01:00
clawsweeper[bot] 0603c2327d fix(file-transfer): require canonical node policy authorization (#74742)
* feat(file-transfer): add bundled plugin for binary file ops on nodes

New extensions/file-transfer/ plugin exposing four agent tools
(file_fetch, dir_list, dir_fetch, file_write) and four matching
node-host commands (file.fetch, dir.list, dir.fetch, file.write).
Lets agents read and write files on paired nodes by absolute path,
bypassing the bash output cap (200KB) and the live tool-result
text cap that would otherwise truncate base64 payloads.

Public surface
--------------
- file_fetch({ node, path, maxBytes? })
  Image MIMEs return image content blocks; small text (<=8 KB) inlines
  as text content; everything else returns a saved-media-path text
  block. sha256-verified end-to-end.
- dir_list({ node, path, pageToken?, maxEntries? })
  Structured directory listing — name, path, size, mimeType, isDir,
  mtime. Paginated. No content transfer.
- dir_fetch({ node, path, maxBytes?, includeDotfiles? })
  Server-side tar -czf streamed back, unpacked into the gateway media
  store, returns a manifest of saved paths. Single round-trip.
  60s wall-clock timeouts on tar create/unpack. tar -xzf without -P
  rejects absolute paths in archive entries.
- file_write({ node, path, contentBase64, mimeType?, overwrite?,
              createParents? })
  Atomic write (temp + rename). Refuses to overwrite by default.
  Refuses to write through symlinks (lstat check). Buffer-side
  sha256 (no read-back race). Pair with file_fetch to round-trip
  files between nodes — DO NOT use exec/cp for file copies.

All four commands gated by:
  - dangerous-by-default node command policy
    (gateway.nodes.allowCommands opt-in)
  - per-node path policy (gateway.nodes.fileTransfer)
  - optional operator approval prompt (ask: off | on-miss | always)

16 MB raw byte ceiling per single-frame round-trip (25 MB WS frame
with ~33% base64 overhead and JSON envelope). 8 MB defaults.

Path policy and approvals
-------------------------
Default behavior is DENY. The operator must explicitly opt in:

  {
    "gateway": {
      "nodes": {
        "fileTransfer": {
          "<nodeId-or-displayName>": {
            "ask":              "off" | "on-miss" | "always",
            "allowReadPaths":   ["~/Screenshots/**", "/tmp/**"],
            "allowWritePaths":  ["~/Downloads/**"],
            "denyPaths":        ["**/.ssh/**", "**/.aws/**"],
            "maxBytes":         16777216
          },
          "*": { "ask": "on-miss" }
        }
      }
    }
  }

ask modes:
  off       — silent: allow if matched, deny if not (default)
  on-miss   — silent allow if matched; prompt on miss
  always    — prompt every call (denyPaths still hard-deny)

denyPaths always wins. allow-always from the prompt persists the
exact path back into allowReadPaths/allowWritePaths via
mutateConfigFile so subsequent matching calls go silent.

Reuses existing primitives — no new gateway methods:
  plugin.approval.request / plugin.approval.waitDecision
  decision: allow-once | allow-always | deny

Pre-flight against requested path AND post-flight against the
canonicalPath returned by the node — closes symlink-escape attacks
where the requested path matched policy but realpath resolves
somewhere else.

Audit log
---------
JSONL at ~/.openclaw/audit/file-transfer.jsonl. Records every
decision (allow/allowed-once/allowed-always/denied/error) with
timestamp, op, nodeId, displayName, requestedPath, canonicalPath,
decision, error code, sizeBytes, sha256, durationMs. Best-effort
writes; never propagates failure.

Plugin layout
-------------
extensions/file-transfer/
  index.ts                       definePluginEntry, nodeHostCommands
  openclaw.plugin.json           contracts.tools registration
  package.json
  src/node-host/{file-fetch,dir-list,dir-fetch,file-write}.ts
  src/tools/{file-fetch,dir-list,dir-fetch,file-write}-tool.ts
  src/shared/
    mime.ts        single-source extension->MIME map + image/text sets
    errors.ts      shared error code enum and helpers
    params.ts      shared param-validation helpers + GatewayCallOptions
    policy.ts      evaluateFilePolicy, persistAllowAlways
    approval.ts    plugin.approval.request wrapper
    gatekeep.ts    one-stop policy + approval + audit orchestrator
    audit.ts       JSONL audit sink

Core touch points
-----------------
- src/infra/node-commands.ts: NODE_FILE_FETCH_COMMAND,
  NODE_DIR_LIST_COMMAND, NODE_DIR_FETCH_COMMAND,
  NODE_FILE_WRITE_COMMAND, NODE_FILE_COMMANDS array
- src/gateway/node-command-policy.ts: all four added to
  DEFAULT_DANGEROUS_NODE_COMMANDS
- src/security/audit-extra.sync.ts: audit detail mentions file ops
- src/agents/tools/nodes-tool-media.ts: MEDIA_INVOKE_ACTIONS entry
  for file.fetch redirects raw nodes(action=invoke) callers to the
  dedicated file_fetch tool to prevent base64 context bloat
- src/agents/tools/nodes-tool.ts: nodes tool description points to
  the dedicated file_fetch tool

Known limitations / follow-ups
------------------------------
- No tests in this PR. For a security-sensitive surface this is a
  gap; will follow up with a test pass.
- Direct CLI invocation (openclaw nodes invoke --command file.fetch)
  bypasses the plugin policy entirely. Plugin-side gating is the
  realistic threat model (agent on iMessage requesting paths it
  shouldn't), but for true defense-in-depth, policy belongs in the
  gateway-side node.invoke dispatch. Move-policy-to-core is a
  separate PR.
- file_watch (long-lived filesystem event subscription) is not
  included; it needs a new node-protocol primitive for streaming
  event channels and was descoped from this PR.
- dir_fetch includeDotfiles: true is the only supported mode;
  BSD tar exclude patterns reliably collapse dotfile filtering
  to an empty archive. Reliable filtering needs a
  `find ! -name ".*" | tar -T -` pipeline; deferred.
- dir_fetch du -sk preflight is a heuristic (du * 4 vs maxBytes);
  the mid-stream byte cap is the actual safety net.

* test(file-transfer): add unit tests for handlers, policy, and shared utilities

Adds 77 tests covering:
- handleFileFetch: validation, fs errors, sha256, size cap, symlink canonicalization
- handleFileWrite: validation, atomic write, overwrite policy, parent dir handling, symlink refusal, integrity check, size cap
- handleDirList: validation, fs errors, sorted listing, dotfile inclusion, pagination
- handleDirFetch: validation, fs errors, gzipped tar with sha256, mid-stream byte cap
- evaluateFilePolicy: default-deny, denyPaths-wins, allow matching, ask modes (off/on-miss/always), node-id/displayName/'*' resolution
- persistAllowAlways: append, dedupe, create-on-missing
- shared/mime: extension lookup, image/text inline sets
- shared/errors: err helper, classifyFsError, throwFromNodePayload

Also fixes accumulated lint regressions in the prod source flagged once these
files moved into the changed-gate scope (parseInt -> Number.parseInt, redundant
type casts removed, single-statement if bodies wrapped in braces).

* fix(file-transfer): address PR review feedback (security + availability)

Reviewer findings addressed (greptile + aisle):

- policy: persistAllowAlways no longer escalates per-node approvals to the
  '*' wildcard entry; allow-always now writes under the specific node's
  own entry, never the wildcard (greptile P1 SECURITY).
- policy: add literal '..' segment short-circuit in evaluateFilePolicy,
  raised before glob match. Stops "/allowed/../etc/passwd" from passing
  preflight against "/allowed/**" globs (aisle MEDIUM CWE-22).
- file-write: replace no-op base64 try/catch with actual round-trip
  validation. Buffer.from(s, "base64") never throws — invalid input
  silently decoded to garbage bytes. Now re-encodes and compares
  modulo padding/url-variant chars (greptile P1 SECURITY).
- file-write: document the parent-symlink residual risk and rely on the
  existing gateway-side post-flight policy check; full rollback requires
  a node-side file.unlink which is deferred to a follow-up. Initial
  segment-walk attempt was reverted because it false-positives on system
  symlinks like macOS /var → /private/var (aisle HIGH CWE-59).
- dir-fetch tool: add preValidateTarball pass that runs `tar -tzvf` and
  rejects symlinks, hardlinks, absolute paths, '..' traversal,
  uncompressed sizes >64MB, and entry counts >5000 — before any
  extraction. Drops --no-overwrite-dir (GNU-only flag rejected by BSD
  tar on macOS) (aisle HIGH x2 CWE-22 + CWE-409, greptile P2).
- dir-fetch tool: stream-hash files via fs.open + read loop instead of
  fs.readFile to avoid full-buffer reads on large extracted entries.
- dir-fetch handler: replace spawnSync in countTarEntries with async
  spawn + bounded buffer so tar -tzf can't park the node-host event
  loop for up to 10s on a slow filesystem (greptile P1 AVAIL).
- audit: clear auditDirPromise on rejection so a transient mkdir
  failure doesn't permanently silence the audit log (greptile P2).

New tests: wildcard escalation rejection, base64 malformed/url-variant,
'..' traversal short-circuit (3 cases). 84/84 passing.

* fix(file-transfer): CI failures + second-round PR review feedback

CI failures on previous push:

- Declare runtime deps (minimatch, typebox) in package.json — failed the
  extension-runtime-dependencies contract test that scans imports.
- Switch policy.ts and policy.test.ts off the broad
  openclaw/plugin-sdk/config-runtime barrel and onto the narrow
  openclaw/plugin-sdk/config-mutation + runtime-config-snapshot subpaths.
  This satisfies the deprecated-internal-config-api architecture guard.

Second-round Aisle findings:

- policy: traversal-segment check now treats backslash and forward slash
  as equivalent, so a Windows node can't be hit with mixed-separator
  "C:\\allowed\\..\\Windows\\system.ini" (Aisle HIGH CWE-22).
- dir-fetch tool: replace the single fragile `tar -tvzf` parser pass
  (which broke for filenames containing whitespace) with two robust
  passes: `tar -tzf` for paths only (one per line, no parsing of
  fixed columns) and `tar -tzvf` for type chars only (FIRST CHAR of each
  line, never the path column). Also reject backslash-containing entry
  names. Drops the in-process uncompressed-size cap because reliably
  parsing sizes from tar output is fragile and Aisle flagged it as a
  bypass primitive — entry-count cap stays (Aisle HIGH CWE-22, MED).

Tests still 84/84 passing.

* fix(file-transfer): third-round PR review feedback

Aisle's re-analysis on b63daa6a05 surfaced 3 actionable findings:

- nodes.invoke bypass (HIGH CWE-285): generic nodes.action="invoke" let
  agents call dir.list/dir.fetch/file.write directly, skipping the
  file-transfer plugin's gatekeep + policy + approval flow. Only file.fetch
  was redirected to its dedicated tool. Add the other three to
  MEDIA_INVOKE_ACTIONS so the redirect-or-deny logic in
  nodes-tool-commands fires for all four. The dedicated tools enforce
  policy; the generic invoke surface no longer has a way to skip them
  without an explicit allowMediaInvokeCommands opt-in.
- prototype pollution in persistAllowAlways (MED CWE-1321): a paired
  node with displayName "__proto__" / "prototype" / "constructor" would
  mutate the fileTransfer object's prototype when persisting allow-always.
  Reject those keys explicitly. Switch the existing-key lookup to
  Object.prototype.hasOwnProperty.call so a key like "constructor"
  doesn't accidentally match Object.prototype.constructor.
- decompression-bomb cap in dir_fetch (MED CWE-409): compressed tar is
  bounded upstream, but a highly compressible bomb can still expand to
  gigabytes. Enforce DIR_FETCH_MAX_UNCOMPRESSED_BYTES (64MB) summed
  across extracted files and DIR_FETCH_MAX_SINGLE_FILE_BYTES (16MB) per
  entry, both checked during the post-extract walk. On bust, rm -rf the
  rootDir and audit-log + throw UNCOMPRESSED_TOO_LARGE.

Tests: 85/85 passing (added prototype-pollution rejection test).

Aisle's HIGH parent-symlink finding remains documented as deferred — full
rollback requires a node-side file.unlink command which is out of scope
for this PR. The gateway-side post-flight policy check still detects and
loudly errors on canonical-path mismatches.

* fix(file-transfer): refuse symlink traversal by default with followSymlinks opt-in

Closes the deferred Aisle HIGH parent-symlink finding. Instead of
detecting the escape in a post-flight gateway check after the file is
already written, the node-side handler now refuses pre-flight if any
component of the requested path resolves through a symlink.

Behavior:
- Reads (file.fetch / dir.list / dir.fetch): node realpath()s the
  requested path. If canonical != requested AND followSymlinks=false,
  return SYMLINK_REDIRECT { canonicalPath } — no I/O happens.
- Writes (file.write): node realpath()s the parent dir. Same refusal
  rule. The lstat-on-final check is kept to catch the case where the
  target file itself is an existing symlink.
- Opt-in: set gateway.nodes.fileTransfer.<node>.followSymlinks=true to
  bring back the previous "follow + post-flight check" behavior.

Operator UX: the SYMLINK_REDIRECT response includes the canonical path
so the operator can either update their allow list to the canonical form
or set followSymlinks=true on that node. On macOS, /var → /private/var
and /tmp → /private/tmp are system aliases that trip the new check, so
operators using those paths need followSymlinks=true OR canonical-path
allowlists.

Wiring:
- Add followSymlinks?: boolean to NodeFilePolicyConfig.
- evaluateFilePolicy returns followSymlinks (default false) on its
  ok=true branches.
- gatekeep propagates it via GatekeepOutcome.
- Each tool passes it as a node.invoke param.
- Each handler honors it pre-flight before any read/write.

Tests updated: 89/89 passing.
- realpath(mkdtemp()) so existing happy-path tests don't trip the new
  default on macOS where mkdtemp lands under symlinked /var/folders.
- New tests: SYMLINK_REDIRECT refusal for file.fetch and file.write
  parent traversal; opt-in passthrough when followSymlinks=true.
- New policy test: followSymlinks propagation default false / true.

* fix(file-transfer): close two more aisle findings on 069bd66

Aisle re-analysis on 069bd66 surfaced two issues my earlier round-three
fix missed:

- HIGH (CWE-284): file.fetch / dir.fetch / dir.list / file.write were
  still bypassable via the generic nodes.action="invoke" surface when
  the operator had set allowMediaInvokeCommands=true. That flag was
  meant to opt in to base64-bloat for camera/screen, not to disable
  path policy on file-transfer. Split the redirect map: introduce
  POLICY_REDIRECT_INVOKE_COMMANDS (file-transfer only) which ALWAYS
  rerouts to its dedicated tool regardless of the bloat flag. Camera
  and screen continue to use the bloat-only redirect (suppressed by
  allowMediaInvokeCommands=true). Confirmed by clawsweeper P1.
- MED (CWE-276): tar -xzf in dir_fetch unpack preserved archive
  ownership and permissions, so a malicious node could plant
  setuid/setgid or world-writable files on a gateway running with
  elevated privileges. Add --no-same-owner --no-same-permissions
  (both flags are portable across BSD tar / GNU tar).

Tests: 89/89 passing.

* chore(file-transfer): drop file_watch from plugin description

Phase 5 (file_watch) was deferred earlier in this PR. Strip the watch
mention from the plugin description in package.json,
openclaw.plugin.json, and index.ts so the metadata reflects what's
actually shipped (file_fetch, dir_list, dir_fetch, file_write).
Closes clawsweeper P3.

* fix(file-transfer): hash before rename and allow zero-byte round-trip

Two of Peter's review findings on PR #74134:

- P2 (file-write integrity): hash the decoded buffer + compare against
  expectedSha256 BEFORE temp+rename. Previously the rename happened
  first, then the sha check unlinked the target on mismatch — with
  overwrite=true a bad caller hash could replace + delete the original.
  Now a hash mismatch returns INTEGRITY_FAILURE without touching disk.
  Added a regression test that asserts the original file survives.

- P2/P3 (zero-byte round-trip): the tool layer's truthy checks on
  contentBase64 and base64 rejected the empty string, blocking zero-byte
  files from round-tripping through file_fetch -> file_write. Switched
  to type-checks (typeof === "string") and added zero-byte tests at the
  handler layer for both fetch and write (sha matches the known empty
  digest).

Tests: 92/92 passing.

* fix(file-transfer): declare gateway.nodes.fileTransfer in core config schema

Peter's P1/P2 finding: the plugin reads/writes gateway.nodes.fileTransfer
via casts through unknown because the strict zod schema and OpenClawConfig
type didn't declare it. That meant `openclaw config validate` would
reject the very examples in the plugin's own documentation.

- Add fileTransfer block to gateway.nodes in src/config/zod-schema.ts
  with the full per-node entry shape (ask, allowReadPaths,
  allowWritePaths, denyPaths, maxBytes, followSymlinks).
- Add GatewayNodeFileTransferEntry + the fileTransfer field on
  GatewayNodesConfig in src/config/types.gateway.ts.
- Drop the `as unknown` casts in the extension's policy.ts now that
  gateway.nodes.fileTransfer is properly typed end-to-end.
- Regenerate docs/.generated/config-baseline.sha256.

Tests: 92/92 passing. pnpm config:docs:check OK.

* fix(file-transfer): enforce path policy at gateway dispatch

Closes Peter's P1 review finding on PR #74134.

The agent-tool-only redirect added in earlier commits left CLI
(`openclaw nodes invoke`), plugin-runtime, and raw `node.invoke` callers
able to skip the file-transfer path policy entirely. The fix moves the
security boundary down to the gateway: every code path that reaches
`node.invoke` for file.fetch / dir.list / dir.fetch / file.write now
runs the same allow/deny check.

- New: src/gateway/file-transfer-dispatch.ts with
  `evaluateFileTransferDispatchPolicy` and `isFileTransferCommand`. Same
  semantics as the extension-side `evaluateFilePolicy` minus the
  operator-prompt flow (prompts stay at the agent-tool layer; the
  gateway is silent enforcement).
- src/gateway/server-methods/nodes.ts: after the existing command
  allowlist check, run the new gate before forwarding. Denies emit
  INVALID_REQUEST with a structured `{ command, code, reason }`.
- Decision matrix mirrors the extension: NO_POLICY (no entry for
  this node) deny, denyPaths-wins, '..' traversal short-circuit
  (with backslash separator handling), allowPaths match → allow,
  no allow match → deny.
- 19 new unit tests covering each branch including identity
  resolution (nodeId/displayName/'*'), prototype-pollution-safe lookup,
  and read-vs-write allow-list separation.

Note on allow-once approvals: the agent tool's interactive
`allow-once` decision now has to flow through the dedicated tool's
pre-flight (which forwards an approved request); raw `nodes.invoke`
callers cannot benefit from one-time approvals because the gateway is
silent. allow-always (which persists to allowReadPaths/allowWritePaths)
continues to work transparently because by the time the next request
hits the gateway the path is in the persisted allow list.

Tests: 92 extension + 19 gateway = 111 total, all passing.

* fix(file-transfer): enforce node policy in gateway

* fix(file-transfer): use plugin node policy only

* fix(file-transfer): harden node policy edge cases

* fix(file-transfer): close review hardening gaps

* fix(file-transfer): harden node invoke policy

* fix(file-transfer): align runtime dependency versions

* fix(file-transfer): keep minimatch extension-owned

* refactor(file-transfer): remove unused approval gate

* fix(file-transfer): require canonical node policy authorization

Co-authored-by: Omar Shahine <10343873+omarshahine@users.noreply.github.com>

* fix(clawsweeper): address review for automerge-openclaw-openclaw-74134 (1)

Co-authored-by: Omar Shahine <10343873+omarshahine@users.noreply.github.com>

* fix(file-transfer): recheck dir fetch archive policy after fetch

* fix(file-transfer): name file-transfer tool in invoke redirect

---------

Co-authored-by: Omar Shahine <10343873+omarshahine@users.noreply.github.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Co-authored-by: clawsweeper-repair <clawsweeper-repair@users.noreply.github.com>
2026-04-30 04:03:40 +00:00
Shakker c0ec58f4b6 fix: preserve runtime kind install fallback 2026-04-28 18:04:54 +01:00
EVA 1adaa28dc8 [plugin sdk] Add generic plugin host-hook contracts (#72287)
Merged via squash.

Prepared head SHA: 68e5f2ce19
Co-authored-by: 100yenadmin <239388517+100yenadmin@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-04-27 17:07:02 -07:00
Peter Steinberger 67a447c175 refactor: tighten plugin runtime sdk boundaries 2026-04-27 14:15:53 +01:00
Vincent Koc 1fc5b2b703 feat(migrations): add plugin-owned Hermes import
* feat: add migration providers

* feat: offer Hermes migration during onboarding

* feat(hermes): map imported config surfaces

* feat(onboard): require fresh migration imports

* docs(cli): clarify Hermes import coverage

* chore(migrations): rename Hermes importer package

* chore(migrations): rewire Hermes importer id

* fix(migrations): redact migration JSON details

* fix(hermes): use provider runtime for config imports

* test(hermes): cover missing source planning

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-04-27 00:34:29 -07:00
Vincent Koc cb4fc58547 feat(plugins): move Bonjour discovery into bundled plugin
* fix(deps): detect constant dynamic imports in ownership audit

* feat(plugins): move bonjour discovery into bundled plugin

* test(plugins): remove moved bonjour core tests

* fix(plugins): harden bonjour disable and console restore

* fix(plugins): split gateway discovery ids from services

* fix(plugins): harden bonjour advertiser shutdown

* fix(plugins): clean up bonjour split lint
2026-04-23 23:29:51 -07:00
Peter Steinberger 5d0887574b feat(plugin-sdk): add conversation binding hooks 2026-04-24 04:24:07 +01:00
Peter Steinberger f1805ab54d fix: centralize provider thinking profiles 2026-04-21 09:13:35 +01:00
Vincent Koc 61da711b1a fix(cycles): split provider runtime model types 2026-04-11 22:50:45 +01:00
Vincent Koc 74e7b8d47b fix(cycles): bulk extract leaf type surfaces 2026-04-11 13:26:50 +01:00
Peter Steinberger 44ec4d05de feat: add pluggable agent harness registry 2026-04-10 21:22:16 +01:00
Vincent Koc 952862b9e2 perf(plugins): narrow boundary compile import surfaces 2026-04-08 08:52:52 +01:00
Peter Steinberger 471d056e2f refactor: move browser runtime seams behind plugin metadata 2026-04-05 23:13:14 +01:00
Peter Steinberger a81cf1da1f refactor: share sdk lazy config and cli test helpers 2026-04-04 16:55:04 +09:00
Peter Steinberger b167ad052c refactor(providers): move defaults and error policy into plugins 2026-04-04 07:43:14 +01:00
Vincent Koc 26f0c7ee90 refactor(plugin-sdk): lazily resolve plugin config schemas 2026-04-04 15:32:33 +09:00
Peter Steinberger eb9051cc7c refactor(openai): move native transport policy into extension 2026-04-04 04:27:14 +01:00
Peter Steinberger a23ab9b906 refactor: move voice-call realtime providers into extensions 2026-04-04 12:07:23 +09:00
Peter Steinberger 856592cf00 fix(outbound): restore generic delivery and security seams 2026-04-03 19:09:20 +01:00
Josh Lehman 799c6f40aa refactor: move provider replay runtime ownership into plugins (#60126)
* refactor: move provider replay runtime ownership into plugins

* fix(provider-runtime): address review followups

---------

Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
2026-04-03 23:14:37 +09:00
Peter Steinberger 49936f6066 refactor: move ollama synthetic auth precedence into extension 2026-04-03 21:25:02 +09:00
Josh Lehman 71346940ad refactor: add provider replay runtime hook surfaces (#59143)
Merged via squash.

Prepared head SHA: 56b41e87a5
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-04-01 13:45:41 -07:00
Peter Steinberger 8bf8baef87 Revert "refactor: move tasks into bundled plugin"
This reverts commit c75f4695b7.
2026-04-01 01:30:22 +09:00
Peter Steinberger c75f4695b7 refactor: move tasks into bundled plugin 2026-03-31 15:22:08 +01:00
Peter Steinberger d042192c7c refactor(plugins): move provider policy hooks into plugins 2026-03-28 06:11:13 +00:00