diff --git a/docs/concepts/context.md b/docs/concepts/context.md index 068422a7237f..c349f715d4e8 100644 --- a/docs/concepts/context.md +++ b/docs/concepts/context.md @@ -123,7 +123,7 @@ By default, OpenClaw injects a fixed set of workspace files (if present): Large files are truncated per-file using `agents.defaults.bootstrapMaxChars` (default `20000` chars). OpenClaw also enforces a total bootstrap injection cap across files with `agents.defaults.bootstrapTotalMaxChars` (default `60000` chars). `/context` shows **raw vs injected** sizes and whether truncation happened. -When truncation occurs, the runtime can inject an in-prompt warning block under Project Context. Configure this with `agents.defaults.bootstrapPromptTruncationWarning` (`off`, `once`, `always`; default `always`). +When truncation occurs, the runtime injects a concise in-prompt notice under Project Context saying some bootstrap files were truncated; per-file names and sizes stay in `/context` and other diagnostics. This notice is built in and not configurable. ## Skills: injected vs loaded on-demand diff --git a/docs/concepts/session-pruning.md b/docs/concepts/session-pruning.md index 3da4c7a4bfd8..d00e5e11d34c 100644 --- a/docs/concepts/session-pruning.md +++ b/docs/concepts/session-pruning.md @@ -23,12 +23,12 @@ Pruning is especially valuable for **Anthropic prompt caching**. After the cache Pruning runs in `cache-ttl` mode, gated on both a time check and a context-size check: 1. Wait for the cache TTL to expire (default 5 minutes when set manually; see [Smart defaults](#smart-defaults) for the Anthropic auto-default). Before the TTL elapses, pruning is skipped entirely to preserve prompt-cache reuse for nearby turns. -2. Once the TTL has elapsed, estimate total context size against the model's context window. If the ratio is below `softTrimRatio` (default 0.3), skip pruning and keep the TTL clock running. -3. **Soft-trim** oversized tool results above the ratio: keep the head and tail (default 1500 chars each, capped at 4000 chars combined), insert `...` in between. -4. If the ratio is still at or above `hardClearRatio` (default 0.5) and at least `minPrunableToolChars` (default 50,000) of prunable tool content remains, **hard-clear** those results: replace their content with a placeholder (default `[Old tool result content cleared]`). +2. Once the TTL has elapsed, estimate total context size against the model's context window. Below roughly 30% usage, pruning is skipped and the TTL clock keeps running. +3. **Soft-trim** oversized tool results: results over 4,000 characters keep their first and last 1,500 characters with `...` in between. +4. If context usage is still at or above roughly 50% and at least 50,000 characters of prunable tool content remain, **hard-clear** those results: replace their content with a placeholder (default `[Old tool result content cleared]`, configurable via `agents.defaults.contextPruning.hardClear.placeholder`; set `hardClear.enabled: false` to skip this step). 5. Reset the TTL clock only when pruning actually changed the context, so follow-up requests reuse the fresh cache. -Two safety rules apply regardless of thresholds: the most recent `keepLastAssistants` assistant turns (default 3) are never pruned, and nothing before the session's first user message is ever pruned (protects bootstrap reads like `SOUL.md`/`USER.md`). +Two safety rules apply regardless of thresholds: the last three assistant turns are never pruned, and nothing before the session's first user message is ever pruned (protects bootstrap reads like `SOUL.md`/`USER.md`). The size thresholds and trim windows above are built-in behavior, not config keys; the configurable surface is `agents.defaults.contextPruning` (`mode`, `ttl`, `tools`, `hardClear`). Only `toolResult` messages are eligible; normal conversation text is left alone. Use `agents.defaults.contextPruning.tools.{allow,deny}` to scope which tool names are prunable. diff --git a/docs/concepts/system-prompt.md b/docs/concepts/system-prompt.md index 3fa2db8b249d..6648f22c7feb 100644 --- a/docs/concepts/system-prompt.md +++ b/docs/concepts/system-prompt.md @@ -24,7 +24,7 @@ Provider plugins can contribute cache-aware guidance without replacing the OpenC Use provider-owned contributions for model-family-specific tuning. Reserve the legacy `before_prompt_build` hook for compatibility or truly global prompt changes. -The bundled OpenAI/Codex GPT-5-family overlay (`resolveGpt5SystemPromptContribution`) uses this mechanism: a `stablePrefix` behavior contract (execution policy, tool discipline, output contract, completion contract) plus an optional `interaction_style` override for a friendlier tone. It applies to any `gpt-5*` model id routed through the OpenAI or Codex plugins, controlled by `agents.defaults.promptOverlays.gpt5.personality` (`"friendly"`/`"on"` or `"off"`). +The bundled OpenAI/Codex GPT-5-family overlay (`resolveGpt5SystemPromptContribution`) uses this mechanism: a `stablePrefix` behavior contract (execution policy, tool discipline, output contract, completion contract) plus an optional `interaction_style` override for a friendlier tone. It applies to any `gpt-5*` model id routed through the OpenAI or Codex plugins, controlled by the OpenAI plugin setting `plugins.entries.openai.config.personality` (`"friendly"`/`"on"` or `"off"`). ## Structure @@ -112,13 +112,12 @@ On non-Codex harnesses, the remaining bootstrap files compose into the OpenClaw Large files are truncated with a marker: -| Limit | Config key | Default | -| -------------------------------------------- | -------------------------------------------------- | -------- | -| Per-file max characters | `agents.defaults.bootstrapMaxChars` | 20000 | -| Total across all files | `agents.defaults.bootstrapTotalMaxChars` | 60000 | -| Truncation warning (`off`\|`once`\|`always`) | `agents.defaults.bootstrapPromptTruncationWarning` | `always` | +| Limit | Config key | Default | +| ----------------------- | ---------------------------------------- | ------- | +| Per-file max characters | `agents.defaults.bootstrapMaxChars` | 20000 | +| Total across all files | `agents.defaults.bootstrapTotalMaxChars` | 60000 | -Missing files inject a short missing-file marker. Detailed raw/injected counts stay in diagnostics such as `/context`, `/status`, doctor, and logs. +When truncation happens, OpenClaw always injects a concise notice into the system prompt saying some bootstrap files were truncated and to read the affected files directly; this notice is built in and not configurable, and it deliberately omits per-file details. Missing files inject a short missing-file marker. File names and raw/injected counts stay in diagnostics such as `/context`, `/status`, doctor, and logs. For memory files, truncation is not data loss: the file stays intact on disk. On native Codex, `MEMORY.md` is read on demand through memory tools when available, with bounded prompt fallback otherwise. On other harnesses, the model only sees the shortened injected copy until it reads or searches memory directly. If `MEMORY.md` is repeatedly truncated, distill it into a shorter durable summary, move detailed history into `memory/*.md`, or intentionally raise the bootstrap limits. diff --git a/docs/concepts/typebox.md b/docs/concepts/typebox.md index ac9e4fb916ea..199bda4ac965 100644 --- a/docs/concepts/typebox.md +++ b/docs/concepts/typebox.md @@ -184,7 +184,7 @@ Example: add a new `system.echo` request that returns `{ ok: true, text }`. 1. **Schema (source of truth)** -Add to `packages/gateway-protocol/src/schema/system.ts` (or the closest matching feature module): +Add to `packages/gateway-protocol/src/schema/system-info.ts` (or the closest matching feature module): ```ts export const SystemEchoParamsSchema = Type.Object( diff --git a/docs/gateway/config-agents.md b/docs/gateway/config-agents.md index 76d9d9693b0a..8e979515c6d0 100644 --- a/docs/gateway/config-agents.md +++ b/docs/gateway/config-agents.md @@ -153,24 +153,14 @@ injection behavior from the shared defaults. Omitted fields inherit from } ``` -### `agents.defaults.bootstrapPromptTruncationWarning` +### Bootstrap truncation notice -Controls the agent-visible system-prompt notice when bootstrap context is truncated. -Default: `"always"`. - -- `"off"`: never inject truncation notice text into the system prompt. -- `"once"`: inject a concise notice once per unique truncation signature. -- `"always"`: inject a concise notice on every run when truncation exists (recommended). - -Detailed raw/injected counts and config tuning fields stay in diagnostics such -as context/status reports and logs; routine WebChat user/runtime context only -gets the concise recovery notice. - -```json5 -{ - agents: { defaults: { bootstrapPromptTruncationWarning: "always" } }, // off | once | always -} -``` +When bootstrap context is truncated, OpenClaw always injects a concise +agent-visible notice into the system prompt saying some bootstrap files were +truncated and to read the affected files directly. This notice is built in +and not configurable, and it deliberately omits per-file diagnostics: file +names, raw vs injected counts, and limit causes stay in diagnostics such as +context/status reports and logs. ### Context budget ownership map @@ -505,27 +495,19 @@ as shown above. See [CLI backends](/gateway/cli-backends) for operations and [building CLI backend plugins](/plugins/cli-backend-plugins) for command, session, image, and parser registration. -### `agents.defaults.promptOverlays` +### GPT-5 prompt overlay -Provider-independent prompt overlays applied by model family on OpenClaw-assembled prompt surfaces. GPT-5-family model ids receive the shared behavior contract across OpenClaw/provider routes; `personality` controls only the friendly interaction-style layer. Native Codex app-server routes keep Codex-owned base/model instructions instead of this OpenClaw GPT-5 overlay, and OpenClaw disables Codex's built-in personality for native threads. +The GPT-5 prompt overlay is provider-owned, not an agent default. GPT-5-family +model ids on OpenAI-family providers receive a tagged behavior contract on +OpenClaw-assembled prompts, and the OpenAI plugin's +`plugins.entries.openai.config.personality` setting (`"friendly"`/`"on"` or +`"off"`) toggles only the friendly interaction-style layer. Native Codex +app-server routes keep Codex-owned base/model instructions instead of this +overlay. See [GPT-5 prompt contribution](/providers/openai#gpt-5-prompt-contribution). -```json5 -{ - agents: { - defaults: { - promptOverlays: { - gpt5: { - personality: "friendly", // friendly | on | off - }, - }, - }, - }, -} -``` - -- `"friendly"` (default) and `"on"` enable the friendly interaction-style layer. -- `"off"` disables only the friendly layer; the tagged GPT-5 behavior contract remains enabled. -- Legacy `plugins.entries.openai.config.personality` is still read when this shared setting is unset. +The retired `agents.defaults.promptOverlays` key is rejected by config +validation; `openclaw doctor --fix` migrates its personality value into +`plugins.entries.openai.config.personality` when that key is unset. ### `agents.defaults.heartbeat` diff --git a/docs/plan/path3-sqlite-session-artifact-family.md b/docs/plan/path3-sqlite-session-artifact-family.md deleted file mode 100644 index 2f121fa71c97..000000000000 --- a/docs/plan/path3-sqlite-session-artifact-family.md +++ /dev/null @@ -1,103 +0,0 @@ ---- -summary: "Path 3 plan for archiving all SQLite transcript artifacts that belong to a session" -read_when: - - You are implementing clawdbot-d63.2 / clawdbot-04b - - You are touching SQLite session retention, reset, delete, or agent-deletion archival - - You need to distinguish SQLite-era artifact families from legacy JSONL sidecars -title: "Path 3 SQLite session artifact family" ---- - -# Path 3 SQLite Session Artifact Family - -This note scopes `clawdbot-d63.2` while `clawdbot-d63.1` owns the overlapping -reset/delete archive helper in `src/config/sessions/session-accessor.sqlite.ts`. -The implementation file was dirty during this pass, so this artifact records -the exact contract and patch points without racing the sibling worker. - -## Authoritative family - -After the SQLite flip, active session transcripts are SQLite rows. A session's -archive family is: - -- The `transcript_events`, `transcript_event_identities`, and `sessions` rows - for the entry's current `sessionId`. -- The same SQLite transcript row set for every `sessionId` referenced by - `entry.compactionCheckpoints[*].preCompaction.sessionId`. -- The same SQLite transcript row set for every `sessionId` referenced by - `entry.compactionCheckpoints[*].postCompaction.sessionId`. -- The same SQLite transcript row set for every `sessionId` in - `entry.usageFamilySessionIds`. - -Archive only rows that are no longer referenced by any remaining -`session_entries` row or by any remaining entry's compaction or usage-family -metadata. This preserves checkpoint branch/restore and usage rollup state until -the final live reference is gone. - -## Non-family artifacts after the flip - -Generated topic transcript file variants and trajectory sidecars are not active -SQLite runtime state. They are legacy file artifacts: - -- Topic variants such as `-topic-.jsonl` only exist for the - file-backed transcript format. SQLite uses the canonical session id plus - `session_routes`/entry delivery metadata instead of per-topic JSONL files. -- Trajectory sidecars such as `.trajectory.jsonl` and `.trajectory-path.json` - are named from real JSONL `sessionFile` paths. SQLite `sessionFile` values are - `sqlite:::` markers and do not name sidecar - files. -- Archive-tier readers must keep reading legacy archived JSONL files, but - runtime retention must not scan active sessions directories or reopen JSONL - transcript files for SQLite sessions. - -Doctor import remains the migration owner for legacy primary JSONL files and -their adjacent trajectory sidecars. Runtime SQLite retention should not add a -second importer or file fallback. - -## Patch points - -Extend the SQLite archive helper introduced by `clawdbot-d63.1` rather than -adding a parallel path. - -1. Add a local collector near `deleteSqliteSessionStateIfUnreferenced`: - - `collectSqliteSessionArtifactFamily(entry: SessionEntry): Set` - - Include `entry.sessionId`, checkpoint pre/post session ids, and - `usageFamilySessionIds`. - - Filter empty strings and dedupe deterministically. - -2. Add a reference collector for the post-removal store: - - `readReferencedSqliteSessionArtifactFamilyIds(database): Set` - - Iterate current `session_entries`, parse each `entry_json`, and collect - the same family ids from every surviving entry. - -3. Change the reset/delete/maintenance callers that currently archive one - removed `sessionId` to pass the removed entry's full family. - -4. For each family id, archive the SQLite transcript rows with the caller's - reason (`reset` or `deleted`), then delete the `sessions` row only when the - family id is absent from the post-removal reference set. - -5. Keep transcript event deletion centralized through the existing SQLite - session-row cleanup path. Do not add active JSONL reads. - -## Focused tests - -Add SQLite-only tests to `src/config/sessions/session-accessor.conformance.test.ts` -or the sibling lifecycle test after `clawdbot-d63.1` commits: - -- Deleting an entry with a pre-compaction transcript archives both the current - session and the pre-compaction session, then removes both SQLite row sets. -- Deleting one of two entries that share a compaction pre-session archives - nothing for the shared pre-session until the final referencing entry is - removed. -- Deleting an entry with `usageFamilySessionIds` archives predecessor SQLite - transcript rows when no other entry references that usage family. -- A topic-shaped session key with a SQLite marker does not cause any generated - topic JSONL read or sidecar lookup. - -The focused proof should use: - -```bash -node scripts/run-vitest.mjs src/config/sessions/session-accessor.conformance.test.ts -``` - -Broad `pnpm` gates should stay on Crabbox/Testbox for this Codex worktree. diff --git a/docs/plan/swarms.md b/docs/plan/swarms.md deleted file mode 100644 index 804def16b55f..000000000000 --- a/docs/plan/swarms.md +++ /dev/null @@ -1,260 +0,0 @@ -# Swarms — agent fan-out and orchestration in code mode - -Status: Shipped — superseded by `docs/tools/swarm.md`. This document remains as -the implementation design record. - -## 1. What and why - -A **swarm** is many subagents orchestrated deterministically from a code-mode -script: fan out N readers, verify findings adversarially, synthesize through a -stateful prioritizer, loop on decision gates. Control flow (`Promise.all`, -`while`, `if`) _is_ the orchestration — there is deliberately **no graph DSL, -no new mode, no new top-level tool surface**. - -OpenClaw code mode (QuickJS-WASI, snapshot/resume, bridge requests) is the -substrate. A parked bridge call survives VM snapshot, gateway restart, and -resumes exactly where it stopped — stronger than journal-replay designs, with -no determinism constraints on scripts. - -Naming: product/docs name is **Swarm**. Code identifiers stay literal: -`agents.*` guest API, `tools.swarm` config, `swarm` group columns. - -## 2. Decisions (maintainer, 2026-07-17) - -- Cost: enforced config caps; per-swarm token budget optional. No mandatory budget. -- Approvals: children run **fail-closed / non-interactive**. Approval-requiring - actions are denied; the denial is reported in the child result; the script - decides. No operator prompt spam from fan-out. -- v1 is model-written ad-hoc scripts only. Saved/named workflows, CLI/cron - entry: later (headless code mode already exists for cron). -- Child identity: dedicated worker agent by default via `tools.swarm.defaultAgentId` - config (validated against existing subagent target allowlist); per-spawn - `agentId` override. Core ships no bundled agent id; docs recommend a lean - `worker` agent config. -- No Codex source changes. Codex harness uses the spawn/wait idiom (§8). - -## 3. Architecture overview - -``` -code-mode script (QuickJS VM, gateway) Codex V8 script (codex process) - agents.run(...) ── parked bridge call tools.sessions_spawn / tools.agents_wait - │ │ item/tool/call RPC (≤600s each) - ▼ ▼ - CORE (harness-agnostic, this repo) - sessions_spawn {collect:true, outputSchema, fastMode, groupId} - agents_wait {ids, timeoutSeconds} - │ - subagent registry (SQLite): collector completion records, swarm group id - │ - children = ordinary subagent sessions (lane-capped, fail-closed approvals) - │ - sessions.changed SSE ──► Control UI dots / sidebar / channel status message -``` - -One canonical owner of spawn/complete/settle semantics (core tools + registry). -Two await transports: QuickJS parks a bridge call indefinitely (snapshot); -Codex polls `agents_wait` in bounded RPCs. - -## 4. Config gate (v1) - -New `tools.swarm` (global + per-agent override, same merge pattern as -`tools.codeMode`): - -```jsonc -"tools": { - "swarm": { - "enabled": false, // master gate, default OFF - "maxConcurrent": 8, // children running at once (swarm lane cap) - "maxChildrenPerGroup": 50, // live children per swarm group - "maxTotalPerGroup": 200, // lifetime spawn count per group (runaway backstop) - "waitTimeoutSecondsMax": 600, - "defaultAgentId": "" // optional; child agent id when spawn omits agentId - } -} -``` - -- Zod: union `boolean | strict object` like `CodeModeSchema` - (`src/config/zod-schema.agent-runtime.ts`); `swarm: true` → `{enabled: true}`. -- Types in `src/config/types.tools.ts` (both per-agent and top-level `tools`), - labels in `schema.labels.ts`, help in `schema.help.runtime.ts`. -- Resolution helper `resolveSwarmConfig(cfg, agentId)` mirroring - `resolveCodeModeConfig` (`src/agents/code-mode.ts:215`), clamping all numbers. -- Gate effects when disabled: `agents_wait` tool absent from catalogs; - `collect`/`outputSchema`/`fastMode`/`groupId` params on `sessions_spawn` - rejected with a clear error naming the config key. No other behavior change. -- `defaultAgentId` is validated through `resolveSubagentAllowedTargetIds` - (`src/agents/subagent-target-policy.ts`); unknown id → spawn error, not fallback. - -## 5. Core: collector-mode spawn + `agents_wait` (v1) - -### 5.1 `sessions_spawn` additions (all gated on swarm enabled) - -- `collect: boolean` — when true, the child run is registered with - `expectsCompletionMessage: false` and a **collector completion record** - instead of announce/steering delivery. Tool returns `{ runId, sessionKey }` - immediately. No channel/thread binding. -- `outputSchema: object` — JSON Schema. Child gets a synthetic - `structured_output` tool appended to its tool surface; system-prompt addendum - instructs it to call it exactly once with its final result. On validation - failure the child gets one nudge retry; after that the completion record - carries `structured: undefined` plus the raw text and a `schemaError`. -- `fastMode: true | "auto" | false` — threaded into the child session patch - alongside model/thinking via `resolveSubagentModelAndThinkingPlan` - (`src/agents/subagents/spawn/subagent-spawn-plan.ts`), using the existing `FastMode` axis - (`src/shared/fast-mode.ts`). Omitted = inherit. -- `groupId: string` — swarm group stamp. Defaults to - `swarm::`. Persisted on the - registry record and the child session row. Used for caps, listing, batch - archive, and the dots. -- `label: string` already exists — surfaces in dots and `subagents list`. -- Child agent id: `params.agentId` → else `tools.swarm.defaultAgentId` → else - requester agent (existing behavior). - -### 5.2 Approvals fail-closed - -Collector children run with a non-interactive approval context: any tool call -that would require operator approval resolves as a structured denial -(`approval_required`) visible to the child, which is expected to report the -blockage in its result. Implementation: reuse the existing exec/tool approval -policy plumbing with a forced `deny` resolver for collector-mode child runs. -No approval events are emitted to operator surfaces from collector children. - -### 5.3 `agents_wait` tool (new, gated) - -``` -agents_wait({ ids: string[], timeoutSeconds?: number }) -→ { - completed: [{ runId, status: "done"|"failed"|"killed"|"timeout", - result: string, structured?: unknown, schemaError?: string, - sessionKey, label?, usage?: {inputTokens, outputTokens} }], - pending: string[] - } -``` - -- Returns as soon as **at least one** id completes (first-completion / race - semantics, enables pipelines), or on timeout with `completed: []`. -- `timeoutSeconds` default 30, clamped to `waitTimeoutSecondsMax`. -- Idempotent: already-completed ids return their records again (records are - kept until group archive). Unknown id → per-id error entry, not a throw. -- Ownership: only the session that spawned a run (or its parent chain) may wait - on it — same ownership rule as `wait` in code mode (`code-mode.ts:1684`). -- Registry: completion records live in the existing subagent registry SQLite - store (`subagent-registry.store.sqlite.ts`) — new fields, no new store, no - schema-version bump (additive columns only; see §9 constraint). - -### 5.4 Caps enforcement - -- `maxConcurrent`: collector children run on the existing subagent lane but - counted per swarm group; spawns beyond the cap queue FIFO (host-side, in the - spawn path — return runId immediately, run starts when a slot frees). -- `maxChildrenPerGroup` / `maxTotalPerGroup`: spawn rejects with a typed error - once exceeded; the error text names the config key. -- Depth: collector children keep `DEFAULT_SUBAGENT_MAX_SPAWN_DEPTH` semantics - (children are leaves unless nesting explicitly configured). - -## 6. Testing contract (v1, lane A) - -- Unit: config resolution/clamping; gate rejections when disabled; groupId - defaulting; cap enforcement (queue + reject); wait race semantics; wait - idempotency; ownership denial; structured-output validation + nudge retry + - schemaError path; fastMode plumbing into session patch; defaultAgentId - validation. -- Integration (vitest, mock model runtime): spawn 3 collector children, wait - in a loop, assert first-completion ordering and final drain; gateway-restart - simulation: registry reload → wait resolves from persisted completion. -- All tests colocated `*.test.ts`; no live model calls. - -## 7. QuickJS guest surface (lane B, after core) - -- Guest globals installed in `CONTROLLER_SOURCE` - (`src/agents/code-mode.worker.ts:190-374`), reserved names added in - `code-mode-namespaces.ts`: - - `agents.run(prompt, opts) → Promise` — sugar: - collector spawn + parked await on a dedicated bridge method (`agentWait`) - that the host settles on completion (no polling; snapshot-safe). - - `agents.session(system, opts) → Promise`; - `handle.send(input, opts) → Promise<...>`; `handle.close()`. (v1.1 — - ships after run(); uses `mode:"session"` + per-turn collector records.) - - `phase(title)`, `log(message)` — fire-and-forget bridge notifications → - swarm progress events. -- Bridge methods added to `CodeModeBridgeMethod` (`code-mode.ts:91`): - `agentSpawn`, `agentWait`, `swarmNote`. `agentSpawn`/`agentWait` are - replay-safe **by construction**: idempotency key `(codeModeRunId, bridgeId)` - stored on the registry record; restart re-settles from persisted completions - and never double-spawns. -- Pending `agentWait` bridge calls extend the run's snapshot TTL (pending - agent set is the signal; no flag). -- `API.read("agents.d.ts")` virtual file documents the typed surface + the - fan-out / gate / cycle idioms (`createCodeModeApiVirtualFiles`, - `code-mode-namespaces.ts:876`). - -## 8. Codex harness projection (later lane) - -- `sessions_spawn` (with new params) and `agents_wait` flow through the - existing dynamic-tool bridge; inside Codex code-mode scripts they appear as - `tools.*` automatically (verified: `codex-rs/code-mode/src/runtime/globals.rs:14-65`, - `codex-rs/core/src/tools/spec_plan.rs:448-507`). -- `agents_wait` gets the long dynamic-tool timeout class (600s cap; - `extensions/codex/src/app-server/dynamic-tool-execution.ts:37-39`) and is - marked timeout/replay-safe. -- Group key for Codex parents: `swarm::`. -- Codex-native `spawn_agent` subagents coexist; their task-mirror rows feed - the same progress surface. - -## 9. Persistence and retention - -- No new stores. Registry records extend the existing subagent registry - SQLite tables; children are ordinary `sessions` rows. Additive columns only - — **any change requiring a SQLite schema-version bump needs explicit - maintainer sign-off first** (repo policy). -- Swarm group id on registry record + child session metadata. -- Retention: completed collector records survive until **group archive**: - when the parent run finishes (or TTL expires), the group's children archive - as a batch (extend the existing `DEFAULT_SUBAGENT_ARCHIVE_AFTER_MINUTES` - sweep to operate per group). - -## 10. Progress surface ("the dots") — later lane - -- Implicit, harness-driven. Derived from existing `sessions.changed` SSE + - registry; `phase`/`log` notes add semantics. No agent-driven rendering. -- Control UI: `swarm` renderer in the workspace widget family - (`ui/src/lib/workspace/widgets/`) — dot grid grouped by phase, narrator - line, per-dot status/label/model; sidebar child-tree unchanged. -- Channels: one throttled edited status message per group (follow - `docs/concepts/streaming.md`; never per-child messages). - -## 11. Labs page (Control UI, independent lane) - -Settings → **Labs**: experimental feature toggles, first entries **Code Mode** -and **Swarm**. Each row: name, one-line description, docs link, toggle wired -via the existing `config.patch` RPC (RFC 7396 merge-patch — set -`tools.codeMode.enabled` / `tools.swarm.enabled`), plus a "restart required" -hint when applicable. Discoverable, but copy makes the experimental status -clear. i18n: all strings through the normal `en.ts` + sync pipeline. - -## 12. Placement (later) - -- `placement` opt on spawn: `"local"` (default) | `"cloud:"` via - existing worker-environment dispatch (`sessions.dispatch`); pooled placement - later if shared-box SSH-sandbox children prove insufficient. -- Orchestrator VM always stays on the gateway; settle/dots/budget are - placement-blind. - -## 13. Non-goals - -- No graph DSL — control flow is the graph (deliberate, documented). -- No Codex source changes; no reuse of Codex Code Mode internals. -- No saved/named workflows in v1; no CLI entry point. -- No per-child operator approval bubbling. -- No 1:1 cloud provisioning at fan-out scale. -- No steady-state runtime compat shims; swarm is new surface, gated. - -## 14. Build phases / PR slicing - -1. **Lane A (core)**: §4 config + §5 spawn/wait/caps/approvals + §6 tests. -2. **Lane C (Labs page)**: §11 — independent, can land first. -3. **Lane B (QuickJS surface)**: §7 — after A contracts land. -4. Dots renderer (§10), Codex projection (§8), `agents.session` (§7 v1.1), - placement (§12), user docs rewrite — follow-up PRs in that order. - -Each PR: green CI, `$autoreview` clean, gated off by default, main shippable. diff --git a/docs/providers/openai.md b/docs/providers/openai.md index 131e72c11a4f..f90f25c5872b 100644 --- a/docs/providers/openai.md +++ b/docs/providers/openai.md @@ -842,10 +842,10 @@ separate and configurable. ```json5 { - agents: { - defaults: { - promptOverlays: { - gpt5: { personality: "friendly" }, + plugins: { + entries: { + openai: { + config: { personality: "friendly" }, }, }, }, @@ -854,7 +854,7 @@ separate and configurable. ```bash - openclaw config set agents.defaults.promptOverlays.gpt5.personality off + openclaw config set plugins.entries.openai.config.personality off ``` @@ -865,9 +865,9 @@ friendly style layer. -Legacy `plugins.entries.openai.config.personality` is still read as a -compatibility fallback when the shared -`agents.defaults.promptOverlays.gpt5.personality` setting is unset. +The retired `agents.defaults.promptOverlays` key is no longer read; config +validation rejects it, and `openclaw doctor --fix` migrates its personality +value into `plugins.entries.openai.config.personality` when that key is unset. ## Voice and speech diff --git a/docs/reference/path3-live-sqlite-e2e-harness.md b/docs/reference/path3-live-sqlite-e2e-harness.md deleted file mode 100644 index 6fc01e79c853..000000000000 --- a/docs/reference/path3-live-sqlite-e2e-harness.md +++ /dev/null @@ -1,168 +0,0 @@ ---- -summary: "Design for live Gateway proof of the Path 3 SQLite session/transcript flip" -read_when: - - You are proving the Path 3 SQLite storage flip against a live Gateway - - You need to distinguish expected legacy JSONL drift from runtime failures - - You are building or reviewing the agent-driven live SQLite E2E harness -title: "Path 3 live SQLite E2E harness" ---- - -The Path 3 live SQLite E2E harness proves the Gateway is using SQLite as the -canonical session and transcript store while legacy JSONL files remain -migration input or archive material. It is a maintainer proof harness, not a -normal user diagnostic. - -After a Gateway has processed post-migration traffic, legacy JSONL parity is no -longer a valid runtime health signal. A healthy migrated Gateway can have -SQLite transcript rows that differ from legacy JSONL counts because new turns -should advance SQLite only. The live harness must therefore measure Gateway -behavior, SQLite row movement, legacy-file quiescence, and log health at each -step. - -## Command shape - -The intended live command is: - -```bash -node scripts/path3-live-sqlite-e2e.mjs \ - --url http://127.0.0.1:18789 \ - --agent main \ - --session-key agent:main:path3-live-e2e: \ - --json -``` - -The command connects to an already running Gateway. It does not start, stop, -import, or re-run the migration unless an explicit migration mode is added -later. A CI or isolated-local variant can use -`test/helpers/openclaw-test-instance.ts`, but the live proof path should inspect -the actual operator Gateway and its real per-agent SQLite database. - -## Isolated built-CLI proof - -The built-CLI proof runner seeds an isolated legacy session store, starts the -rebuilt Gateway, and proves that startup imports hot legacy sessions into -SQLite before runtime reads begin. It must not run `openclaw doctor --fix` -before the first Gateway start, because that would prove the manual migration -path instead of the upgrade path users receive on first boot after the flip. - -After startup import, the isolated proof may run -`openclaw doctor --session-sqlite inspect` and -`openclaw doctor --session-sqlite validate` as diagnostic evidence. Those -doctor commands are not the migration driver for the startup-upgrade proof. -Separate doctor-import scenarios should seed legacy transcript files plus -trajectory sidecars and verify doctor archives those artifacts while SQLite -remains canonical. - -## Preflight - -Preflight collects a baseline and fails before sending a proof turn if the -Gateway is not usable: - -- `GET /health` and Gateway deep status must report a running, reachable - Gateway. -- The CLI and Gateway versions must match the branch being tested. -- The harness records a log cursor for the active Gateway file log. -- The harness records per-agent SQLite table counts for `sessions`, - `session_entries`, `transcript_events`, `transcript_event_identities`, and - `session_routes`. -- The harness records `mtime`, `size`, and existence for legacy - `sessions.json`, referenced JSONL files, and candidate proof-session JSONL - paths. -- `lsof -p ` must show SQLite DB/WAL/SHM handles and no hot - `.jsonl` or `sessions.json` handles. - -`openclaw doctor --session-sqlite validate` is informational only in live mode. -After post-flip traffic it may report expected drift against legacy files. The -harness should use doctor output for classification and migration inventory, -not as the runtime pass/fail oracle. - -## Agent-driven scenario - -The live scenario uses a dedicated proof session key and drives the Gateway -through public RPC paths wherever possible. One agent turn should be enough to -exercise ordinary persistence, but the full proof should cover the 3.1b seams -that previously required individual live checks: - -- Ordinary chat turn: create or reuse the proof session, send a real agent - prompt, wait for the final assistant result, and verify `chat.history` or - equivalent Gateway projection. -- Transcript identity: verify the same marker appears in Gateway history and in - SQLite transcript rows, including stable event identity rows when present. -- Session metadata accessors: read the proof session and selected existing live - sessions through Gateway/session accessors and compare them to SQLite rows. -- Session patch projection: apply a reversible model/session metadata change on - the proof session, then verify the projected row and Gateway response agree. -- Compaction checkpoint lifecycle: list, branch, and restore a checkpoint only - on the proof session or a synthetic fixture session created by the harness. -- Restart recovery: run the safe recovery marker path against a controlled proof - session or an isolated test instance; live mode may only run this step when - the target session set is explicit and reversible. -- Cleanup lifecycle: delete or reset the proof session, then verify SQLite - lifecycle rows and archived transcript state. - -Transport-specific seams that cannot be exercised safely on the live operator -Gateway, such as WhatsApp or voice-call ingress, should use owner-level runtime -probes against the same SQLite contract rather than fake external transport. - -## Per-step assertions - -Each step snapshots before and after state and writes a structured assertion -record: - -- SQLite row counts advance only where expected. -- Trajectory runtime rows advance for marker-backed proof sessions that record - runtime events. -- The proof session row has the expected `session_id`, status, timestamps, - metadata, and route rows. -- Gateway history/session projection matches the SQLite transcript tail. -- No proof-session JSONL file is created or modified. -- No proof-session `.trajectory.jsonl`, `.trajectory-path.json`, or - marker-derived `trajectory/.jsonl` sidecar is created. -- Existing legacy JSONL files and `sessions.json` remain unchanged unless the - step is explicitly an offline migration or archive operation. -- The Gateway process does not open `.jsonl` or `sessions.json` handles. -- Logs since the previous cursor contain no `ERROR`, `FATAL`, `SQLITE_`, - `no such column`, session-store unavailable, restart-recovery failure, or - transcript-reconcile warning unless the scenario explicitly allowlists it. - -The log scan is part of the pass/fail contract. A Gateway that answers health -checks but emits SQLite schema errors or repeated transcript reconcile failures -is not green for Path 3. - -## Evidence artifact - -The harness should write evidence under `.artifacts/path3-live-e2e//` -and keep it out of git: - -- `summary.json`: command args, Gateway version, result, failed assertion, and - artifact paths. -- `sqlite-before.json` and `sqlite-after.json`: row counts and selected proof - rows. -- `legacy-files.json`: legacy file existence, `mtime`, size, and whether each - file changed. -- `gateway-log-scan.json`: cursor range, matched log lines, and allowlist - decisions. -- `events.jsonl`: ordered per-step observations suitable for PR proof comments. - -The PR proof should summarize these artifacts instead of pasting full -transcripts or private message content. - -## Safety rules - -- Live mode must never re-import legacy JSONL while the Gateway is running. -- Live mode must not mutate non-proof sessions except for explicitly selected, - reversible repair probes. -- Any destructive or broad migration step requires a fresh backup of the - affected SQLite DB and legacy session directory. -- Backups should be scoped to the touched agent DB/session directory and reused - during one proof run to avoid unbounded disk growth. -- The cleanup step must leave no proof session, proof JSONL, or modified legacy - file behind unless the caller passes `--keep-artifacts`. - -## Passing result - -A passing live run means the Gateway accepted a real agent-driven session flow, -all observed canonical state was in SQLite, legacy runtime files stayed -quiescent, and log health stayed clean for the measured window. It does not mean -legacy JSONL parity remains clean after live traffic; live drift is expected -once SQLite is the canonical store. diff --git a/docs/reference/test.md b/docs/reference/test.md index f390be73d1d9..aaaf6b1c9845 100644 --- a/docs/reference/test.md +++ b/docs/reference/test.md @@ -99,7 +99,7 @@ Test wrapper runs end with a short `[test] passed|failed|skipped ... in ...` sum ## Control UI, TUI, and extension lanes -- **Control UI mocked E2E:** `pnpm test:ui:e2e` runs the Vitest + Playwright lane that starts the Vite Control UI and drives a real Chromium page against a mocked Gateway WebSocket. Tests live in `ui/src/**/*.e2e.test.ts`; shared mocks/controls live in `ui/src/test-helpers/control-ui-e2e.ts`. `pnpm test:e2e` includes this lane. Agent runs default to Testbox/Crabbox, including targeted proof; use `node scripts/run-vitest.mjs run --config test/vitest/vitest.ui-e2e.config.ts --configLoader runner ui/src/ui/e2e/chat-flow.e2e.test.ts` only for an explicit local fallback. +- **Control UI mocked E2E:** `pnpm test:ui:e2e` runs the Vitest + Playwright lane that starts the Vite Control UI and drives a real Chromium page against a mocked Gateway WebSocket. Tests live in `ui/src/**/*.e2e.test.ts`; shared mocks/controls live in `ui/src/test-helpers/control-ui-e2e.ts`. `pnpm test:e2e` includes this lane. Agent runs default to Testbox/Crabbox, including targeted proof; use `node scripts/run-vitest.mjs run --config test/vitest/vitest.ui-e2e.config.ts --configLoader runner ui/src/e2e/chat-flow.messaging.e2e.test.ts` only for an explicit local fallback. - **TUI PTY tests:** `node scripts/run-vitest.mjs run --config test/vitest/vitest.tui-pty.config.ts` runs the fast fake-backend PTY lane. `OPENCLAW_TUI_PTY_INCLUDE_LOCAL=1` or `pnpm tui:pty:test:watch --mode local` runs the slower `tui --local` smoke, which mocks only the external model endpoint. CI also sets `OPENCLAW_TUI_PTY_USE_BUILT_CLI=1` after building `dist/`; use that flag only when exact-head built artifacts already exist. Assert stable visible text or fixture calls, not raw ANSI snapshots. - `pnpm test:extensions` and `pnpm test extensions` run all extension/plugin shards. Heavy channel plugins, the browser plugin, and OpenAI run as dedicated shards; other plugin groups stay batched. `pnpm test extensions/` runs one bundled plugin lane. - Source files with sibling tests map to that sibling before falling back to wider directory globs. Helper edits under `src/channels/plugins/contracts/test-helpers`, `src/plugin-sdk/test-helpers`, and `src/plugins/contracts` use a local import graph to run importing tests instead of broad-running every shard when the dependency path is precise. diff --git a/extensions/openai/openclaw.plugin.json b/extensions/openai/openclaw.plugin.json index 17d7797296f2..e5892085a71f 100644 --- a/extensions/openai/openclaw.plugin.json +++ b/extensions/openai/openclaw.plugin.json @@ -306,7 +306,7 @@ "type": "string", "enum": ["friendly", "on", "off"], "default": "friendly", - "description": "Controls the shared GPT-5 friendly interaction-style overlay. `friendly` and `on` enable the style overlay; `off` disables only that style layer." + "description": "Controls the shared GPT-5 friendly interaction-style overlay for OpenAI-family providers. `friendly` and `on` enable the style overlay; `off` disables only that style layer while the tagged GPT-5 behavior contract stays active." } } }