refactor: move session/transcript runtime state to SQLite

Migrate OpenClaw session, transcript, and runtime state off per-file
.jsonl/JSON storage onto the SQLite state and agent databases, brought up to
date with current main. Channel plugins persist via the plugin-state SDK
keyed-store seam rather than the raw DB; storage-heavy plugins keep their own
SQLite stores via SDK path helpers.
This commit is contained in:
Josh Lehman
2026-05-31 09:23:11 -07:00
parent 4b1e5b7943
commit 43ea501f38
3309 changed files with 175707 additions and 136819 deletions
+2 -1
View File
@@ -172,7 +172,8 @@ Notes:
- `main` cannot be deleted.
- Without `--force`, interactive confirmation is required.
- Workspace, agent state, and session transcript directories are moved to Trash, not hard-deleted.
- Workspace and per-agent state directories are moved to Trash, not hard-deleted.
- Session rows for the deleted agent are purged from SQLite.
- When the Gateway is reachable, deletion is sent through the Gateway so config and session-store cleanup share the same writer as runtime traffic. If the Gateway cannot be reached, the CLI falls back to the offline local path.
- If another agent's workspace is the same path, inside this workspace, or contains this workspace,
the workspace is retained and `--json` reports `workspaceRetained`,
+12 -12
View File
@@ -9,7 +9,7 @@ title: "Approvals"
# `openclaw approvals`
Manage exec approvals for the **local host**, **gateway host**, or a **node host**.
By default, commands target the local approvals file on disk. Use `--gateway` to target the gateway, or `--node` to target a specific node.
By default, commands target the local approvals state in SQLite. Use `--gateway` to target the gateway, or `--node` to target a specific node.
Alias: `openclaw exec-approvals`
@@ -21,13 +21,13 @@ Related:
## `openclaw exec-policy`
`openclaw exec-policy` is the local convenience command for keeping the requested
`tools.exec.*` config and the local host approvals file aligned in one step.
`tools.exec.*` config and the local host approvals state aligned in one step.
Use it when you want to:
- inspect the local requested policy, host approvals file, and effective merge
- inspect the local requested policy, host approvals state, and effective merge
- apply a local preset such as YOLO or deny-all
- synchronize local `tools.exec.*` and local `~/.openclaw/exec-approvals.json`
- synchronize local `tools.exec.*` and local exec approvals state
Examples:
@@ -49,10 +49,10 @@ Output modes:
Current scope:
- `exec-policy` is **local-only**
- it updates the local config file and the local approvals file together
- it updates the local config file and the local approvals state together
- it does **not** push policy to the gateway host or a node host
- `--host node` is rejected in this command because node exec approvals are fetched from the node at runtime and must be managed through node-targeted approvals commands instead
- `openclaw exec-policy show` marks `host=node` scopes as node-managed at runtime instead of deriving an effective policy from the local approvals file
- `openclaw exec-policy show` marks `host=node` scopes as node-managed at runtime instead of deriving an effective policy from local approvals state
If you need to edit remote host approvals directly, keep using `openclaw approvals set --gateway`
or `openclaw approvals set --node <id|name|ip>`.
@@ -73,9 +73,9 @@ openclaw approvals get --gateway
Precedence is intentional:
- the host approvals file is the enforceable source of truth
- the host approvals state is the enforceable source of truth
- requested `tools.exec` policy can narrow or broaden intent, but the effective result is still derived from the host rules
- `--node` combines the node host approvals file with gateway `tools.exec` policy, because both still apply at runtime
- `--node` combines the node host approvals state with gateway `tools.exec` policy, because both still apply at runtime
- if gateway config is unavailable, the CLI falls back to the node approvals snapshot and notes that the final runtime policy could not be computed
## Replace approvals from a file
@@ -123,7 +123,7 @@ openclaw approvals set --node <id|name|ip> --stdin <<'EOF'
EOF
```
This changes the **host approvals file** only. To keep the requested OpenClaw policy aligned, also set:
This changes the **host approvals state** only. To keep the requested OpenClaw policy aligned, also set:
```bash
openclaw config set tools.exec.host gateway
@@ -169,8 +169,8 @@ openclaw approvals allowlist remove "~/Projects/**/bin/rg"
Targeting notes:
- no target flags means the local approvals file on disk
- `--gateway` targets the gateway host approvals file
- no target flags means the local approvals state
- `--gateway` targets the gateway host approvals state
- `--node` targets one node host after resolving id, name, IP, or id prefix
`allowlist add|remove` also supports:
@@ -182,7 +182,7 @@ Targeting notes:
- `--node` uses the same resolver as `openclaw nodes` (id, name, ip, or id prefix).
- `--agent` defaults to `"*"`, which applies to all agents.
- The node host must advertise `system.execApprovals.get/set` (macOS app or headless node host).
- Approvals files are stored per host at `~/.openclaw/exec-approvals.json`.
- Approvals are stored per host in the SQLite state database. Legacy `~/.openclaw/exec-approvals.json` files are imported by `openclaw doctor --fix`.
## Related
+15 -10
View File
@@ -1,5 +1,5 @@
---
summary: "CLI reference for `openclaw backup` (create local backup archives)"
summary: "CLI reference for `openclaw backup` (create, verify, and restore local backup archives)"
read_when:
- You want a first-class backup archive for local OpenClaw state
- You want to preview which paths would be included before reset or uninstall
@@ -8,28 +8,34 @@ title: "Backup"
# `openclaw backup`
Create a local backup archive for OpenClaw state, config, auth profiles, channel/provider credentials, sessions, and optionally workspaces.
Create, verify, or restore a local backup archive for OpenClaw state, config,
channel/provider credentials, sessions, auth profiles, and optionally
workspaces.
```bash
openclaw backup create
openclaw backup create --output ~/Backups
openclaw backup create --dry-run --json
openclaw backup create --verify
openclaw backup create --no-verify
openclaw backup create --no-include-workspace
openclaw backup create --only-config
openclaw backup verify ./2026-03-09T08-00-00.000+08-00-openclaw-backup.tar.gz
openclaw backup verify ./2026-03-09T00-00-00.000Z-openclaw-backup.tar.gz
openclaw backup restore ./2026-03-09T00-00-00.000Z-openclaw-backup.tar.gz --dry-run
```
## Notes
- The archive includes a `manifest.json` file with the resolved source paths and archive layout.
- SQLite databases under the state directory are snapshotted with SQLite `VACUUM INTO`; live `*.sqlite-wal` and `*.sqlite-shm` sidecars are not archived directly.
- Default output is a timestamped `.tar.gz` archive in the current working directory.
- Timestamped backup filenames use your machine's local timezone and include the UTC offset.
- If the current working directory is inside a backed-up source tree, OpenClaw falls back to your home directory for the default archive location.
- Existing archive files are never overwritten.
- Output paths inside the source state/workspace trees are rejected to avoid self-inclusion.
- `openclaw backup verify <archive>` validates that the archive contains exactly one root manifest, rejects traversal-style archive paths, and checks that every manifest-declared payload exists in the tarball.
- `openclaw backup create --verify` runs that validation immediately after writing the archive.
- `openclaw backup create` validates the written archive by default: it requires exactly one root manifest, rejects traversal-style archive paths, checks that every manifest-declared payload exists in the tarball, and runs SQLite integrity checks for manifest-declared database snapshots.
- `openclaw backup create --no-verify` skips the post-write archive validation pass.
- `openclaw backup restore <archive> --dry-run` validates the archive and previews the recorded source paths that would be replaced.
- `openclaw backup restore <archive> --yes` restores the archive to the recorded source paths. Restore validates the archive before extracting, then replaces each manifest asset from the verifier-normalized payload.
- `openclaw backup create --only-config` backs up just the active JSON config file.
## What gets backed up
@@ -41,9 +47,8 @@ openclaw backup verify ./2026-03-09T08-00-00.000+08-00-openclaw-backup.tar.gz
- The resolved `credentials/` directory when it exists outside the state directory
- Workspace directories discovered from the current config, unless you pass `--no-include-workspace`
Model auth profiles are already part of the state directory under
`agents/<agentId>/agent/auth-profiles.json`, so they are normally covered by the
state backup entry.
Model auth profiles are stored in SQLite under the state directory, so they are
covered by the database snapshots in the state backup entry.
If you use `--only-config`, OpenClaw skips state, credentials-directory, and workspace discovery and archives only the active config file path.
@@ -86,7 +91,7 @@ Practical limits come from the local machine and destination filesystem:
- Available space for the temporary archive write plus the final archive
- Time to walk large workspace trees and compress them into a `.tar.gz`
- Time to rescan the archive if you use `openclaw backup create --verify` or run `openclaw backup verify`
- Time to rescan the archive after `openclaw backup create`, unless you pass `--no-verify`
- Filesystem behavior at the destination path. OpenClaw prefers a no-overwrite hard-link publish step and falls back to exclusive copy when hard links are unsupported
Large workspaces are usually the main driver of archive size. If you want a smaller or faster backup, use `--no-include-workspace`.
+1 -1
View File
@@ -80,7 +80,7 @@ Text output includes:
- scope
- suggested check-in text
JSON output also includes the commitment store path and full stored records.
JSON output also includes the SQLite state database path and full stored records.
## Related
+4 -6
View File
@@ -2,7 +2,7 @@
summary: "CLI reference for `openclaw completion` (generate/install shell completion scripts)"
read_when:
- You want shell completions for zsh/bash/fish/PowerShell
- You need to cache completion scripts under OpenClaw state
- You need to install shell completion profile hooks
title: "Completion"
---
@@ -17,22 +17,20 @@ openclaw completion
openclaw completion --shell zsh
openclaw completion --install
openclaw completion --shell fish --install
openclaw completion --write-state
openclaw completion --shell bash --write-state
```
## Options
- `-s, --shell <shell>`: shell target (`zsh`, `bash`, `powershell`, `fish`; default: `zsh`)
- `-i, --install`: install completion by adding a source line to your shell profile
- `--write-state`: write completion script(s) to `$OPENCLAW_STATE_DIR/completions` without printing to stdout
- `-y, --yes`: skip install confirmation prompts
## Notes
- `--install` writes a small "OpenClaw Completion" block into your shell profile and points it at the cached script.
- Without `--install` or `--write-state`, the command prints the script to stdout.
- `--install` writes a small "OpenClaw Completion" block into your shell profile that generates completions from the CLI.
- Without `--install`, the command prints the script to stdout.
- Completion generation eagerly loads command trees so nested subcommands are included.
- OpenClaw does not write shell completion cache files under state.
## Related
+1 -1
View File
@@ -133,7 +133,7 @@ you pass `--yes` for a direct command:
Applied writes are recorded in:
```text
~/.openclaw/audit/crestodian.jsonl
SQLite core plugin state: core:crestodian/audit
```
Discovery is not audited. Only applied operations and writes are logged.
+4 -4
View File
@@ -118,7 +118,7 @@ Skipped runs are tracked separately from execution errors. They do not affect re
For isolated jobs that target a local configured model provider, cron runs a lightweight provider preflight before starting the agent turn. Loopback, private-network, and `.local` `api: "ollama"` providers are probed at `/api/tags`; local OpenAI-compatible providers such as vLLM, SGLang, and LM Studio are probed at `/models`. If the endpoint is unreachable, the run is recorded as `skipped` and retried on a later schedule; matching dead endpoints are cached for 5 minutes to avoid many jobs hammering the same local server.
Note: cron jobs, pending runtime state, and run history live in the shared SQLite state database. Legacy `jobs.json`, `jobs-state.json`, and `runs/*.jsonl` files are imported once and renamed with a `.migrated` suffix. After import, edit schedules with `openclaw cron add|edit|remove` instead of editing JSON files.
Note: cron job definitions, pending runtime state, and run history live in the shared SQLite state database. Legacy `jobs.json`, `jobs-state.json`, and `runs/*.jsonl` files are imported and removed by `openclaw doctor --fix`; malformed legacy rows are preserved in SQLite quarantine state during import. After import, edit schedules with `openclaw cron add|edit|remove` instead of editing JSON files.
### Manual runs
@@ -196,15 +196,15 @@ Cron does not classify final-output prose or approval-looking refusal phrases as
## Retention
Retention and pruning are controlled in config:
Cron run-log retention is controlled in config:
- `cron.sessionRetention` (default `24h`) prunes completed isolated run sessions.
- `cron.runLog.keepLines` prunes retained SQLite run-history rows per job. `cron.runLog.maxBytes` remains accepted for compatibility with older file-backed run logs.
- Session rows are SQLite-backed and are not pruned by age/count maintenance.
## Migrating older jobs
<Note>
If you have cron jobs from before the current delivery and store format, run `openclaw doctor --fix`. Doctor normalizes legacy cron fields (`jobId`, `schedule.cron`, top-level delivery fields including legacy `threadId`, payload `provider` delivery aliases) and migrates simple `notify: true` webhook fallback jobs to explicit webhook delivery when `cron.webhook` is configured.
If you have cron jobs from before the current delivery and store format, run `openclaw doctor --fix`. Doctor normalizes legacy cron fields (`jobId`, `schedule.cron`, top-level delivery fields including legacy `threadId`, payload `provider` delivery aliases) and migrates simple `notify: true` webhook fallback jobs to explicit webhook delivery when the deprecated migration fallback `cron.webhook` is configured.
</Note>
## Common edits
+9 -3
View File
@@ -198,8 +198,8 @@ Notes:
- `--fix` (alias for `--repair`) writes a backup to `~/.openclaw/openclaw.json.bak` and drops unknown config keys, listing each removal.
- Modernized health checks can expose a `repair()` path for `doctor --fix`; checks that do not expose one continue through the existing doctor repair flow.
- `doctor --fix --non-interactive` reports missing or stale gateway service definitions but does not install or rewrite them outside update repair mode. Run `openclaw gateway install` for a missing service, or `openclaw gateway install --force` when you intentionally want to replace the launcher.
- State integrity checks now detect orphan transcript files in the sessions directory. Archiving them as `.deleted.<timestamp>` requires an interactive confirmation; `--fix`, `--yes`, and headless runs leave them in place.
- Doctor also scans `~/.openclaw/cron/jobs.json` (or `cron.store`) for legacy cron job shapes and can rewrite them in place before the scheduler has to auto-normalize them at runtime.
- State integrity checks now detect orphan legacy transcript files in old sessions directories. Deleting those leftovers requires an interactive confirmation; `--fix`, `--yes`, and headless runs leave them in place unless a migration step imports and removes them.
- Doctor also imports legacy `~/.openclaw/cron/jobs.json` / `jobs-state.json` cron stores into SQLite and normalizes old job shapes before the scheduler sees them.
- Doctor reports cron jobs with explicit `payload.model` overrides, including provider namespace counts and mismatches against `agents.defaults.model`, so scheduled jobs that do not inherit the default model are visible during auth or billing investigations.
- On Linux, doctor warns when the user's crontab still runs legacy `~/.openclaw/bin/ensure-whatsapp.sh`; that script is no longer maintained and can log false WhatsApp gateway outages when cron lacks the systemd user-bus environment.
- When WhatsApp is enabled, doctor checks for a degraded Gateway event loop with local `openclaw-tui` clients still running. `doctor --fix` stops only verified local TUI clients so WhatsApp replies are not queued behind stale TUI refresh loops.
@@ -217,9 +217,15 @@ Notes:
- Doctor removes retired `plugins.entries.codex.config.codexDynamicToolsProfile`; Codex app-server always keeps Codex-native workspace tools native.
- Doctor warns when skills allowed for the default agent are unavailable in the current runtime environment because bins, env vars, config, or OS requirements are missing. `doctor --fix` can disable those unavailable skills with `skills.entries.<skill>.enabled=false`; install/configure the missing requirement instead when you want to keep the skill active.
- If sandbox mode is enabled but Docker is unavailable, doctor reports a high-signal warning with remediation (`install Docker` or `openclaw config set agents.defaults.sandbox.mode off`).
- If legacy sandbox registry files (`~/.openclaw/sandbox/containers.json` or `~/.openclaw/sandbox/browsers.json`) are present, doctor reports them; `openclaw doctor --fix` migrates valid entries into sharded registry directories and quarantines invalid legacy files.
- If legacy sandbox registry files (`~/.openclaw/sandbox/containers.json`, `~/.openclaw/sandbox/browsers.json`, or old registry shard JSON files) are present, doctor reports them; `openclaw doctor --fix` migrates valid entries into SQLite and quarantines invalid legacy files.
- Legacy session state (`sessions.json`, transcript JSONL files, compaction checkpoints, and related session sidecars) is a doctor/migrate input only. Repair imports valid data into the global/per-agent SQLite databases and removes successfully imported sources; runtime code no longer keeps compatibility readers for those files.
- If `gateway.auth.token`/`gateway.auth.password` are SecretRef-managed and unavailable in the current command path, doctor reports a read-only warning and does not write plaintext fallback credentials. For exec-backed SecretRefs, doctor skips execution unless `--allow-exec` is present.
- If channel SecretRef inspection fails in a fix path, doctor continues and reports a warning instead of exiting early.
- Extension-owned state migrations run through doctor without loading full
channel runtimes. BlueBubbles, Discord, Feishu, Matrix, Microsoft Teams,
QQBot, and Telegram import their legacy JSON sidecars into SQLite plugin
state/blob tables from their own setup/doctor migration files, then remove the
imported sources.
- After state-directory migrations, doctor warns when enabled default Telegram or Discord accounts depend on env fallback and `TELEGRAM_BOT_TOKEN` or `DISCORD_BOT_TOKEN` is unavailable to the doctor process.
- Telegram `allowFrom` username auto-resolution (`doctor --fix`) requires a resolvable Telegram token in the current command path. If token inspection is unavailable, doctor reports a warning and skips auto-resolution for that pass.
+5 -8
View File
@@ -104,10 +104,7 @@ openclaw gateway run
Alias for `--ws-log compact`.
</ParamField>
<ParamField path="--raw-stream" type="boolean">
Log raw model stream events to jsonl.
</ParamField>
<ParamField path="--raw-stream-path <path>" type="string">
Raw stream jsonl path.
Log raw model stream events to SQLite diagnostics.
</ParamField>
## Restart the Gateway
@@ -127,11 +124,11 @@ openclaw gateway restart --force
Inline `--password` can be exposed in local process listings. Prefer `--password-file`, env, or a SecretRef-backed `gateway.auth.password`.
</Warning>
### Gateway profiling
### Startup profiling
- Set `OPENCLAW_GATEWAY_STARTUP_TRACE=1` to log phase timings during Gateway startup, including per-phase `eventLoopMax` delay and plugin lookup-table timings for installed-index, manifest registry, startup planning, and owner-map work.
- Set `OPENCLAW_GATEWAY_RESTART_TRACE=1` to log restart-scoped `restart trace:` lines for restart signal handling, active-work drain, shutdown phases, next start, ready timing, and memory metrics.
- Set `OPENCLAW_DIAGNOSTICS=timeline` with `OPENCLAW_DIAGNOSTICS_TIMELINE_PATH=<path>` to write a best-effort JSONL startup diagnostics timeline for external QA harnesses. You can also enable the flag with `diagnostics.flags: ["timeline"]` in config; the path is still env-provided. Add `OPENCLAW_DIAGNOSTICS_EVENT_LOOP=1` to include event-loop samples.
- Set `OPENCLAW_DIAGNOSTICS=timeline` to write a best-effort startup diagnostics timeline into the shared SQLite state database for external QA harnesses. You can also enable the flag with `diagnostics.flags: ["timeline"]` in config. Add `OPENCLAW_DIAGNOSTICS_EVENT_LOOP=1` to include event-loop samples.
- Run `pnpm build` first, then `pnpm test:startup:gateway -- --runs 5 --warmup 1` to benchmark Gateway startup against the built CLI entry. The benchmark records first process output, `/healthz`, `/readyz`, startup trace timings, event-loop delay, and plugin lookup-table timing details.
- Run `pnpm build` first, then `pnpm test:restart:gateway -- --case skipChannels --runs 1 --restarts 5` to benchmark in-process Gateway restart against the built CLI entry on macOS or Linux. The restart benchmark uses SIGUSR1, enables both startup and restart traces in the child process, and records next `/healthz`, next `/readyz`, downtime, ready timing, CPU, RSS, and restart trace metrics.
- Treat `/healthz` as liveness and `/readyz` as usable readiness. Trace lines and benchmark output are for owner attribution; do not treat one trace span or one sample as a complete performance conclusion.
@@ -171,7 +168,7 @@ The HTTP `/healthz` endpoint is a liveness probe: it returns once the server can
### `gateway usage-cost`
Fetch usage-cost summaries from session logs.
Fetch usage-cost summaries from session transcripts.
```bash
openclaw gateway usage-cost
@@ -217,7 +214,7 @@ openclaw gateway stability --json
<AccordionGroup>
<Accordion title="Privacy and bundle behavior">
- Records keep operational metadata: event names, counts, byte sizes, memory readings, queue/session state, channel/plugin names, and redacted session summaries. They do not keep chat text, webhook bodies, tool outputs, raw request or response bodies, tokens, cookies, secret values, hostnames, or raw session ids. Set `diagnostics.enabled: false` to disable the recorder entirely.
- On fatal Gateway exits, shutdown timeouts, and restart startup failures, OpenClaw writes the same diagnostic snapshot to `~/.openclaw/logs/stability/openclaw-stability-*.json` when the recorder has events. Inspect the newest bundle with `openclaw gateway stability --bundle latest`; `--limit`, `--type`, and `--since-seq` also apply to bundle output.
- On fatal Gateway exits, shutdown timeouts, and restart startup failures, OpenClaw writes the same diagnostic snapshot to the shared SQLite state database when the recorder has events. Inspect the newest bundle with `openclaw gateway stability --bundle latest`; `--limit`, `--type`, and `--since-seq` also apply to bundle output.
</Accordion>
</AccordionGroup>
+5 -5
View File
@@ -300,7 +300,7 @@ openclaw hooks enable bootstrap-extra-files
### command-logger
Logs all command events to a centralized audit file.
Logs all command events to the shared SQLite state database.
**Enable:**
@@ -308,19 +308,19 @@ Logs all command events to a centralized audit file.
openclaw hooks enable command-logger
```
**Output:** `~/.openclaw/logs/commands.log`
**Output:** `~/.openclaw/state/openclaw.sqlite`, table `command_log_entries`
**View logs:**
```bash
# Recent commands
tail -n 20 ~/.openclaw/logs/commands.log
sqlite3 ~/.openclaw/state/openclaw.sqlite 'select datetime(timestamp_ms / 1000, "unixepoch"), action, session_key, sender_id, source from command_log_entries order by timestamp_ms desc limit 20;'
# Pretty-print
cat ~/.openclaw/logs/commands.log | jq .
sqlite3 -json ~/.openclaw/state/openclaw.sqlite 'select entry_json from command_log_entries order by timestamp_ms desc limit 20;' | jq .
# Filter by action
grep '"action":"new"' ~/.openclaw/logs/commands.log | jq .
sqlite3 ~/.openclaw/state/openclaw.sqlite 'select entry_json from command_log_entries where action = "new" order by timestamp_ms desc;'
```
**See:** [command-logger documentation](/automation/hooks#command-logger)
-1
View File
@@ -203,7 +203,6 @@ openclaw [--dev] [--profile <name>] <command>
status
health
sessions
cleanup
tasks
list
audit
+1 -1
View File
@@ -55,7 +55,7 @@ openclaw memory index --agent main --verbose
- `--deep`: probe local vector-store readiness, embedding-provider readiness, and semantic vector-search readiness. Plain `memory status` stays fast and does not run live embedding or provider discovery work; unknown vector-store or semantic-vector state means it was not probed in that command. QMD lexical `searchMode: "search"` skips semantic vector probes and embedding maintenance even with `--deep`.
- `--index`: run a reindex if the store is dirty (implies `--deep`).
- `--fix`: repair stale recall locks and normalize promotion metadata.
- `--fix`: normalize short-term promotion metadata.
- `--json`: print JSON output.
If `memory status` shows `Dreaming status: blocked`, the managed dreaming cron is enabled but the heartbeat that drives it is not firing for the default agent. See [Dreaming never runs](/concepts/dreaming#dreaming-never-runs-status-shows-blocked) for the two common causes.
+5 -1
View File
@@ -10,6 +10,10 @@ title: "Migrate"
Import state from another agent system through a plugin-owned migration provider. Bundled providers cover Codex CLI state, [Claude](/install/migrating-claude), and [Hermes](/install/migrating-hermes); third-party plugins can register additional providers.
Legacy OpenClaw file-to-database imports are doctor-owned. Run
`openclaw doctor --fix` after upgrading an older state directory so doctor can
create the database and import legacy files in one migration pass.
<Tip>
For user-facing walkthroughs, see [Migrating from Claude](/install/migrating-claude) and [Migrating from Hermes](/install/migrating-hermes). The [migration hub](/install/migrating) lists all paths.
</Tip>
@@ -202,7 +206,7 @@ For migrated source-installed curated plugins, apply writes:
- `plugins.entries.codex.enabled: true`
- `plugins.entries.codex.config.codexPlugins.enabled: true`
- `plugins.entries.codex.config.codexPlugins.allow_destructive_actions: true`
- `plugins.entries.codex.config.codexPlugins.allow_destructive_actions: false`
- one explicit plugin entry with `marketplaceName: "openai-curated"` and
`pluginName` for each selected plugin
+3 -3
View File
@@ -39,7 +39,7 @@ Probes are real requests (may consume tokens and trigger rate limits).
Use `--agent <id>` to inspect a configured agent's model/auth state. When omitted,
the command uses `OPENCLAW_AGENT_DIR` if set, otherwise the
configured default agent.
Probe rows can come from auth profiles, env credentials, or `models.json`.
Probe rows can come from auth profiles, env credentials, or the stored model catalog.
For OpenAI ChatGPT/Codex OAuth troubleshooting, `openclaw models status`,
`openclaw models auth list --provider openai`, and
`openclaw config get agents.defaults.model --json` are the quickest way to
@@ -50,8 +50,8 @@ Notes:
- `models set <model-or-alias>` accepts `provider/model` or an alias.
- `models list` is read-only: it reads config, auth profiles, existing catalog
state, and provider-owned catalog rows, but it does not rewrite
`models.json`.
state, and provider-owned catalog rows, but it does not rewrite the stored
model catalog.
- The `Auth` column is provider-level and read-only. It is computed from local
auth profile metadata, env markers, configured provider keys, local-provider
markers, AWS Bedrock env/profile markers, and plugin synthetic-auth metadata;
+2 -2
View File
@@ -156,13 +156,13 @@ the previous pending request is superseded and a new `requestId` is created.
Run `openclaw devices list` again before approval.
The node host stores its node id, token, display name, and gateway connection info in
`~/.openclaw/node.json`.
the SQLite state database.
## Exec approvals
`system.run` is gated by local exec approvals:
- `~/.openclaw/exec-approvals.json`
- host-local SQLite approvals state
- [Exec approvals](/tools/exec-approvals)
- `openclaw approvals --node <id|name|ip>` (edit from the Gateway)
+1 -1
View File
@@ -336,7 +336,7 @@ Use `--pin` on npm installs to save the resolved exact spec (`name@version`) in
### Plugin index
Plugin install metadata is machine-managed state, not user config. Installs and updates write it to `plugins/installs.json` under the active OpenClaw state directory. Its top-level `installRecords` map is the durable source of install metadata, including records for broken or missing plugin manifests. The `plugins` array is the manifest-derived cold registry cache. The file includes a do-not-edit warning and is used by `openclaw plugins update`, uninstall, diagnostics, and the cold plugin registry.
Plugin install metadata is machine-managed state, not user config. Installs and updates write it to the global SQLite database at `state/openclaw.sqlite` under the active OpenClaw state directory. The typed `installed_plugin_index` row keeps the durable `installRecords` map, including records for broken or missing plugin manifests, plus the manifest-derived cold registry cache in `plugins`. Legacy `plugins/installs.json` files are doctor migration inputs only.
When OpenClaw sees shipped legacy `plugins.installs` records in config, runtime reads treat them as compatibility input without rewriting `openclaw.json`. Explicit plugin writes and `openclaw doctor --fix` move those records into the plugin index and remove the config key when config writes are allowed; if either write fails, the config records are kept so the install metadata is not lost.
+2
View File
@@ -78,6 +78,8 @@ semantics.
- `start` defaults to `127.0.0.1` unless `--host` is set.
- `run` starts a local debug proxy and then runs the command after `--`.
- Captures are stored in the shared state database
(`~/.openclaw/state/openclaw.sqlite`).
- The debug proxy's direct upstream forwarding opens upstream sockets for diagnostics. When OpenClaw managed proxy mode is active, direct forwarding for proxy requests and CONNECT tunnels is disabled by default; set `OPENCLAW_DEBUG_PROXY_ALLOW_DIRECT_CONNECT_WITH_MANAGED_PROXY=1` only for approved local diagnostics.
- `validate` exits with code 1 when proxy config or destination checks fail.
- Captures are local debugging data; use `openclaw proxy purge` when finished.
+5 -2
View File
@@ -166,12 +166,15 @@ Prefer `openclaw sandbox recreate` over manual backend-specific cleanup. It uses
## Registry migration
OpenClaw stores sandbox runtime metadata as one JSON shard per container/browser entry under the sandbox state directory. Older installs may still have monolithic legacy files:
OpenClaw stores sandbox runtime metadata in the shared SQLite state database.
Older installs may still have JSON registry files:
- `~/.openclaw/sandbox/containers.json`
- `~/.openclaw/sandbox/browsers.json`
- `~/.openclaw/sandbox/containers/*.json`
- `~/.openclaw/sandbox/browsers/*.json`
Regular sandbox runtime reads do not rewrite those files. Run `openclaw doctor --fix` to migrate valid legacy entries into the sharded registry directories. Invalid legacy files are quarantined so one bad old registry cannot hide current runtime entries.
Regular sandbox runtime reads do not rewrite those files. Run `openclaw doctor --fix` to migrate valid legacy entries into SQLite and remove the legacy files. Invalid legacy files are quarantined so one bad old registry cannot hide current runtime entries.
## Configuration
+6 -6
View File
@@ -71,8 +71,8 @@ Scan OpenClaw state for:
- plaintext secret storage
- unresolved refs
- precedence drift (`auth-profiles.json` credentials shadowing `openclaw.json` refs)
- generated `agents/*/agent/models.json` residues (provider `apiKey` values and sensitive provider headers)
- precedence drift (SQLite auth-profile credentials shadowing `openclaw.json` refs)
- stored model catalog residues (provider `apiKey` values and sensitive provider headers)
- legacy residues (legacy auth store entries, OAuth reminders)
Header residue note:
@@ -126,15 +126,15 @@ Flags:
- `--providers-only`: configure `secrets.providers` only, skip credential mapping.
- `--skip-provider-setup`: skip provider setup and map credentials to existing providers.
- `--agent <id>`: scope `auth-profiles.json` target discovery and writes to one agent store.
- `--agent <id>`: scope SQLite auth-profile target discovery and writes to one agent store.
- `--allow-exec`: allow exec SecretRef checks during preflight/apply (may execute provider commands).
Notes:
- Requires an interactive TTY.
- You cannot combine `--providers-only` with `--skip-provider-setup`.
- `configure` targets secret-bearing fields in `openclaw.json` plus `auth-profiles.json` for the selected agent scope.
- `configure` supports creating new `auth-profiles.json` mappings directly in the picker flow.
- `configure` targets secret-bearing fields in `openclaw.json` plus SQLite auth-profile rows for the selected agent scope.
- `configure` supports creating new auth-profile mappings directly in the picker flow.
- Canonical supported surface: [SecretRef Credential Surface](/reference/secretref-credential-surface).
- It performs preflight resolution before apply.
- If preflight/apply includes exec refs, keep `--allow-exec` set for both steps.
@@ -176,7 +176,7 @@ Plan contract details (allowed target paths, validation rules, and failure seman
What `apply` may update:
- `openclaw.json` (SecretRef targets + provider upserts/deletes)
- `auth-profiles.json` (provider-target scrubbing)
- SQLite auth-profile rows (provider-target scrubbing)
- legacy `auth.json` residues
- `~/.openclaw/.env` known secret keys whose values were migrated
+3 -3
View File
@@ -114,12 +114,12 @@ openclaw security audit --fix --json | jq '{fix: .fix.ok, summary: .report.summa
- flips common `groupPolicy="open"` to `groupPolicy="allowlist"` (including account variants in supported channels)
- when WhatsApp group policy flips to `allowlist`, seeds `groupAllowFrom` from
the stored `allowFrom` file when that list exists and config does not already
the stored pairing allowlist when that list exists and config does not already
define `allowFrom`
- sets `logging.redactSensitive` from `"off"` to `"tools"`
- tightens permissions for state/config and common sensitive files
(`credentials/*.json`, `auth-profiles.json`, `sessions.json`, session
`*.jsonl`)
(`state/openclaw.sqlite`, `credentials/*.json` legacy doctor inputs,
legacy runtime/session JSON files, session `*.jsonl`)
- also tightens config include files referenced from `openclaw.json`
- uses `chmod` on POSIX hosts and `icacls` resets on Windows
+30 -89
View File
@@ -10,24 +10,19 @@ title: "Sessions"
List stored conversation sessions.
Session lists are not channel/provider liveness checks. They show persisted
conversation rows from session stores. A quiet Discord, Slack, Telegram, or
other channel can reconnect successfully without creating a new session row
until a message is processed. Use `openclaw channels status --probe`,
`openclaw status --deep`, or `openclaw health --verbose` when you need live
channel connectivity.
conversation rows from the per-agent SQLite databases. A quiet Discord, Slack,
Telegram, or other channel can reconnect successfully without creating a new
session row until a message is processed. Use `openclaw channels status
--probe`, `openclaw status --deep`, or `openclaw health --verbose` when you need
live channel connectivity.
`openclaw sessions` and Gateway `sessions.list` responses are bounded by
default so large long-lived stores cannot monopolize the CLI process or Gateway
event loop. The CLI returns the newest 100 sessions by default; pass
default so large long-lived databases cannot monopolize the CLI process or
Gateway event loop. The CLI returns the newest 100 sessions by default; pass
`--limit <n>` for a smaller/larger window or `--limit all` when you intentionally
need the full store. JSON responses include `totalCount`, `limitApplied`, and
`hasMore` when callers need to show that more rows exist.
RPC clients can pass `configuredAgentsOnly: true` to keep the broad combined
discovery source but return only rows for agents currently present in config.
Control UI uses that mode by default so deleted or disk-only agent stores do
not reappear in the Sessions view.
```bash
openclaw sessions
openclaw sessions --agent work
@@ -40,11 +35,17 @@ openclaw sessions --json
Scope selection:
- default: configured default agent store
- default: configured default agent database
- `--verbose`: verbose logging
- `--agent <id>`: one configured agent store
- `--all-agents`: aggregate all configured agent stores
- `--store <path>`: explicit store path (cannot be combined with `--agent` or `--all-agents`)
- `--agent <id>`: one configured agent database
- `--all-agents`: aggregate all configured agent databases
Canonical per-agent session rows live in `openclaw-agent.sqlite` under each
agent. Existing `sessions.json` indexes are imported by the `openclaw doctor`
fix mode, then removed after SQLite has the rows. Gateway startup does not
import or rewrite legacy session indexes; run doctor when you intentionally want
that migration.
- `--limit <n|all>`: max rows to output (default `100`; `all` restores full output)
Tail human-readable trajectory progress for stored sessions:
@@ -72,11 +73,9 @@ This is the command path used by the `/export-trajectory` slash command after
the owner approves the exec request. The output directory is always resolved
inside `.openclaw/trajectory-exports/` under the selected workspace.
`openclaw sessions --all-agents` reads configured agent stores. Gateway and ACP
session discovery are broader: they also include disk-only stores found under
the default `agents/` root or a templated `session.store` root. Those
discovered stores must resolve to regular `sessions.json` files inside the
agent root; symlinks and out-of-root paths are skipped.
`openclaw sessions --all-agents` reads configured agent databases plus
registered agent databases. Legacy `sessions.json` files are migration inputs
only and should disappear after doctor imports them.
JSON examples:
@@ -84,10 +83,10 @@ JSON examples:
```json
{
"path": null,
"stores": [
{ "agentId": "main", "path": "/home/user/.openclaw/agents/main/sessions/sessions.json" },
{ "agentId": "work", "path": "/home/user/.openclaw/agents/work/sessions/sessions.json" }
"databasePath": null,
"databases": [
{ "agentId": "main", "path": "/home/user/.openclaw/agents/main/agent/openclaw-agent.sqlite" },
{ "agentId": "work", "path": "/home/user/.openclaw/agents/work/agent/openclaw-agent.sqlite" }
],
"allAgents": true,
"count": 2,
@@ -102,71 +101,13 @@ JSON examples:
}
```
## Cleanup maintenance
## Repair
Run maintenance now (instead of waiting for the next write cycle):
```bash
openclaw sessions cleanup --dry-run
openclaw sessions cleanup --agent work --dry-run
openclaw sessions cleanup --all-agents --dry-run
openclaw sessions cleanup --enforce
openclaw sessions cleanup --enforce --active-key "agent:main:telegram:direct:123"
openclaw sessions cleanup --dry-run --fix-dm-scope
openclaw sessions cleanup --json
```
`openclaw sessions cleanup` uses `session.maintenance` settings from config:
- Scope note: `openclaw sessions cleanup` maintains session stores, transcripts, and trajectory sidecars. It does not prune cron run history, which is managed by `cron.runLog.keepLines` in [Cron configuration](/automation/cron-jobs#configuration) and explained in [Cron maintenance](/automation/cron-jobs#maintenance).
- Cleanup also prunes unreferenced primary transcripts, compaction checkpoints, and trajectory sidecars older than `session.maintenance.pruneAfter`; files still referenced by `sessions.json` are preserved.
- `--dry-run`: preview how many entries would be pruned/capped without writing.
- In text mode, dry-run prints a per-session action table (`Action`, `Key`, `Age`, `Model`, `Flags`) so you can see what would be kept vs removed.
- `--enforce`: apply maintenance even when `session.maintenance.mode` is `warn`.
- `--fix-missing`: remove entries whose transcript files are missing or header-only/empty, even if they would not normally age/count out yet.
- `--fix-dm-scope`: when `session.dmScope` is `main`, retire stale peer-keyed direct-DM rows left behind by earlier `per-peer`, `per-channel-peer`, or `per-account-channel-peer` routing. Use `--dry-run` first; applying the cleanup removes those rows from `sessions.json` and preserves their transcripts as deleted archives.
- `--active-key <key>`: protect a specific active key from disk-budget eviction. Durable external conversation pointers, such as group sessions and thread-scoped chat sessions, are also kept by age/count/disk-budget maintenance.
- `--agent <id>`: run cleanup for one configured agent store.
- `--all-agents`: run cleanup for all configured agent stores.
- `--store <path>`: run against a specific `sessions.json` file.
- `--json`: print a JSON summary. With `--all-agents`, output includes one summary per store.
When a Gateway is reachable, non-dry-run cleanup for configured agent stores is
sent through the Gateway so it shares the same session-store writer as runtime
traffic. Use `--store <path>` for explicit offline repair of a store file.
`openclaw sessions cleanup --all-agents --dry-run --json`:
```json
{
"allAgents": true,
"mode": "warn",
"dryRun": true,
"stores": [
{
"agentId": "main",
"storePath": "/home/user/.openclaw/agents/main/sessions/sessions.json",
"beforeCount": 120,
"afterCount": 80,
"missing": 0,
"dmScopeRetired": 0,
"pruned": 40,
"capped": 0
},
{
"agentId": "work",
"storePath": "/home/user/.openclaw/agents/work/sessions/sessions.json",
"beforeCount": 18,
"afterCount": 18,
"missing": 0,
"dmScopeRetired": 0,
"pruned": 0,
"capped": 0
}
]
}
```
Legacy JSON import belongs to `openclaw doctor --fix`. Runtime commands do not
prune, cap, import, or rewrite session databases. If doctor reports session rows
whose transcript events are missing, rerun doctor to import any remaining legacy
sources; if the source transcript is gone, reset or delete the affected session
explicitly.
Related:
-8
View File
@@ -38,14 +38,6 @@ the heartbeat immediately; `next-heartbeat` waits for the next scheduled tick.
Pass `--session-key` to target a specific session (for example to relay an
async-task completion back to the channel that started it).
> **Timing exception with `--session-key`:** when `--session-key` is supplied,
> `--mode next-heartbeat` collapses to an immediate targeted wake instead of
> waiting for the next scheduled tick. Targeted wakes use heartbeat intent
> `immediate` so they bypass the runner's not-due gate that would otherwise
> defer (and effectively drop) an `event`-intent wake. If you want delayed
> delivery, omit `--session-key` so the event lands on the main session and
> rides the next regular heartbeat.
Flags:
- `--text <text>`: required system event text.
+3 -3
View File
@@ -113,10 +113,10 @@ the packaged `dist` inventory there, then swaps that clean package tree into the
real global prefix. If verification fails, post-update doctor, plugin sync, and
restart work do not run from the suspect tree. Even when the installed version
already matches the target, the command refreshes the global package install,
then runs plugin sync, a core-command completion refresh, and restart work. This
then runs plugin sync, shell-completion profile checks, and restart work. This
keeps packaged sidecars and channel-owned plugin records aligned with the
installed OpenClaw build while leaving full plugin-command completion rebuilds to
explicit `openclaw completion --write-state` runs.
installed OpenClaw build without writing completion cache files under OpenClaw
state.
When a local managed Gateway service is installed and restart is enabled,
package-manager updates stop the running service before replacing the package
+13 -15
View File
@@ -24,8 +24,8 @@ openclaw voicecall speak --call-id <id> --message <text>
openclaw voicecall dtmf --call-id <id> --digits <digits>
openclaw voicecall end --call-id <id>
openclaw voicecall status [--call-id <id>] [--json]
openclaw voicecall tail [--file <path>] [--since <n>] [--poll <ms>]
openclaw voicecall latency [--file <path>] [--last <n>]
openclaw voicecall tail [--since <n>] [--poll <ms>]
openclaw voicecall latency [--last <n>]
openclaw voicecall expose [--mode <m>] [--path <p>] [--port <port>] [--serve-path <p>]
```
@@ -40,8 +40,8 @@ openclaw voicecall expose [--mode <m>] [--path <p>] [--port <port>] [--serve-p
| `dtmf` | Send DTMF digits to an active call. |
| `end` | Hang up an active call. |
| `status` | Inspect active calls (or one by `--call-id`). |
| `tail` | Tail `calls.jsonl` (useful during provider tests). |
| `latency` | Summarize turn-latency metrics from `calls.jsonl`. |
| `tail` | Tail SQLite-backed call records (useful during provider tests). |
| `latency` | Summarize turn-latency metrics from SQLite-backed call records. |
| `expose` | Toggle Tailscale serve/funnel for the webhook endpoint. |
## Setup and smoke
@@ -158,22 +158,20 @@ openclaw voicecall status --call-id <id>
### `tail`
Tail the voice-call JSONL log. Prints the last `--since` lines on start, then streams new lines as they are written.
Tail SQLite-backed voice-call records. Prints the last `--since` records on start, then streams newly written records.
| Flag | Default | Description |
| --------------- | -------------------------- | ------------------------------ |
| `--file <path>` | resolved from plugin store | Path to `calls.jsonl`. |
| `--since <n>` | `25` | Lines to print before tailing. |
| `--poll <ms>` | `250` (minimum 50) | Poll interval in milliseconds. |
| Flag | Default | Description |
| ------------- | ------------------ | ------------------------------ |
| `--since <n>` | `25` | Lines to print before tailing. |
| `--poll <ms>` | `250` (minimum 50) | Poll interval in milliseconds. |
### `latency`
Summarize turn-latency and listen-wait metrics from `calls.jsonl`. Output is JSON with `recordsScanned`, `turnLatency`, and `listenWait` summaries.
Summarize turn-latency and listen-wait metrics from SQLite-backed call records. Output is JSON with `recordsScanned`, `turnLatency`, and `listenWait` summaries.
| Flag | Default | Description |
| --------------- | -------------------------- | ------------------------------------ |
| `--file <path>` | resolved from plugin store | Path to `calls.jsonl`. |
| `--last <n>` | `200` (minimum 1) | Number of recent records to analyze. |
| Flag | Default | Description |
| ------------ | ----------------- | ------------------------------------ |
| `--last <n>` | `200` (minimum 1) | Number of recent records to analyze. |
## Exposing webhooks
+3 -5
View File
@@ -106,12 +106,10 @@ Notes:
### `wiki compile`
Rebuild indexes, related blocks, dashboards, and compiled digests.
Rebuild indexes, related blocks, dashboards, and SQLite-backed compiled digests.
This writes stable machine-facing artifacts under:
- `.openclaw-wiki/cache/agent-digest.json`
- `.openclaw-wiki/cache/claims.jsonl`
The stable machine-facing digests live in OpenClaw's SQLite plugin state so
agents and runtime code do not have to scrape Markdown pages.
If `render.createDashboards` is enabled, compile also refreshes report pages.