Peter Steinberger
1ea2640f54
* refactor(state): make cron and subagent rows JSON-canonical * refactor(state): make gateway origin device tokens canonical at v13 The lazy ensure predates the table joining the canonical schema; at the v13 bump the schema owns creation, so the feature-local DDL, WeakSet dedupe, and lazy-list entry retire. The legacy-file guard the ensure carried stays at each call site. * test: drop obsolete lazy-ensure coverage for origin device tokens The table is canonical at v13; same-version lazy creation no longer exists to protect. Origin CRUD, isolation, and rotation coverage remains in the surviving cases. * refactor(state): fold installed_plugin_index into config_machine_state The singleton index row becomes one JSON value under plugins.installedIndex with its rollback-fencing revision inside the value; reads, CAS restore, and the lease-held write transactions use direct Kysely on config_machine_state so the state_leases assertion stays in-transaction. The v13 migration imports the row and drops the table; the additive workspace_dir entry folds with it. Doctor guidance, docker staging, and the e2e probes name the machine-state row. * refactor(state): merge workspace_attestations into workspace_setup_state One row per workspace now carries both setup milestones and the attestation clock: nullable setup columns represent attestation-only workspaces (replaceWorkspaceAttestation can precede any setup write) and setupExists derives from a non-null version. The bootstrap-hash FK repoints to the merged table; migration receipts keep the historical workspace_attestations discriminator string. The v13 migration grows and rebuilds the table, merges attestation rows (orphans without a path alias drop — their hashes re-derive at the next bootstrap attestation), and the consolidation kind is renamed state-consolidation-v13 to cover the batch. * test(state): cover the workspace merge and consolidation fallout The v12-to-v13 regression seeds merged, attestation-only, and orphan attestation workspaces; the 13-to-12 downgrade fixture recreates workspace_attestations and installed_plugin_index from the folded data; the fold-in migration gates the additive workspace_dir column for pre-additive rows; the workspace merge now triggers on the setup table's own shape so stable-era databases without an attestations table still reshape; the consolidation applied-message covers the batch. * refactor(state): fold shared auth profile singletons into config_machine_state The shared-state auth_profile_stores/auth_profile_state rows (fixed key 'shared') become authProfiles.store/authProfiles.state machine-state values; the agent-DB tables of the same names are untouched. Git-backup redaction moves from table-drop to the authProfiles. secret prefix with seeded-secret absence proof; migration receipts keep the historical table-name discriminators; the shared-auth relocation and receipt verification project the KV cells back to the receipt-era row shapes so persisted digests stay byte-compatible. mcp_oauth_stores stays a table — its multi-key fold is a named follow-up. * test(state): finish shared-auth fold coverage and annotate boundary casts Auth seeders and assertions across the e2e/scripts/secrets suites target the authProfiles machine-state cells; the v12-to-v13 regression proves payload-byte fidelity, non-shared-row drop, and insert-if-absent precedence; the downgrade fixture recreates and repopulates both v12 tables. Boundary type assertions in the plugin-index store carry SAFETY invariants per the ratchet. * chore: shrink assertion-safety baseline for plugin-index store * refactor(doctor): delete the dead onboarding-recommendations migration Its input — the unscoped 'primary' onboarding row — existed only between9a93a52a8aand473962b7de, a two-day beta window; no shipped stable can produce it and the runtime table folded away at v12. The audit backup list keeps recognizing system-agent.jsonl artifacts because beta installs that ran that import may still carry its backups. * docs: sync the 13-to-12 downgrade example with the executable fixture * style: format the synced downgrade example * style: drop unused import and duplicate union constituent * fix(state): keep orphan attestations across the v13 workspace merge The merged workspace_setup_state required a workspace path, but legacy orphan hashed-key attestations never recorded one. workspace_path is now nullable (setup rows still enforce it via CHECK), the v13 migration and the doctor file import keep orphans with a NULL path that heals on the next live access, and the 13-to-12 downgrade keeps attestation-owned hashes. Doctor test seeds move to the folded KV row. * perf(state): retire unused cron indexes * fix(state): preserve v13 migration recovery * fix(state): preserve v12 lazy-table upgrade * docs(state): document v13 auth relocation --------- Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
25 KiB
summary, read_when, title
| summary | read_when | title | |||||
|---|---|---|---|---|---|---|---|
| CLI reference for `openclaw backup` (archives, SQLite snapshots, and Git history) |
|
Backup |
openclaw backup
Create a local backup archive for OpenClaw state, config, auth profiles, channel/provider credentials, sessions, and optionally workspaces.
openclaw backup create
openclaw backup create --output ~/Backups
openclaw backup create --dry-run --json
openclaw backup create --verify
openclaw backup create --no-include-workspace
openclaw backup create --only-config
openclaw backup verify ./2026-03-09T08-00-00.000+08-00-openclaw-backup.tar.gz
openclaw backup restore ./2026-03-09T08-00-00.000+08-00-openclaw-backup.tar.gz --target ./restored-openclaw
openclaw backup sqlite create --global --repository ~/Backups/openclaw-sqlite
openclaw backup sqlite create --agent main --repository ~/Backups/openclaw-sqlite
openclaw backup sqlite list --repository ~/Backups/openclaw-sqlite
openclaw backup sqlite verify ~/Backups/openclaw-sqlite/<snapshot-id>
openclaw backup sqlite verify ~/Backups/openclaw-sqlite/<snapshot-id> --scratch ~/Private/openclaw-scratch
openclaw backup sqlite restore ~/Backups/openclaw-sqlite/<snapshot-id> --target ./restored/openclaw.sqlite
openclaw backup git init --repository ~/Backups/openclaw-git --remote <private-git-url>
openclaw backup git create --repository ~/Backups/openclaw-git --all --push
openclaw backup git log --repository ~/Backups/openclaw-git
openclaw backup git verify --repository ~/Backups/openclaw-git --global
openclaw backup git restore --repository ~/Backups/openclaw-git --agent main --target ./restored/agent.sqlite
openclaw backup enable --repository ~/Backups/openclaw-git --every 24h --push
openclaw backup disable
Archive create, verify, and restore, plus SQLite create, list, verify, and
restore, accept --json for one machine-readable result on stdout.
Notes
- The archive embeds a schema-version-1
manifest.jsonwith the resolved source paths and archive layout. Additive ownership metadata records configured agent ids and roots, including agent roots already covered by another asset; existing archive layout and older archives remain supported. - Default output is a timestamped
.tar.gzarchive in the current working directory. Timestamped filenames use your machine's local timezone and include the UTC offset. If the current working directory is inside a backed-up source tree, OpenClaw falls back to your home directory for the default archive location. - Existing archive files are never overwritten. Output paths inside the source state/workspace trees are rejected to avoid self-inclusion.
openclaw backup verify <archive>checks that the archive contains exactly one root manifest, rejects traversal-style archive paths, unsafe symbolic links, and SQLite sidecars, confirms every manifest-declared payload exists, validates every SQLite snapshot's file shape, and runs full integrity and role checks on canonical OpenClaw databases. Dedicated plugin schemas remain opaque because they may require owner-defined SQLite capabilities.openclaw backup create --verifyruns that validation immediately after writing the archive.openclaw backup create --only-configbacks up just the active JSON config file.
Restore a full archive
Restore a complete archive into a fresh staging directory without touching the live state directory:
openclaw backup restore <archive.tar.gz> --target <fresh-directory>
The target must not exist or must be an empty directory, and it cannot be inside
the live state directory or any configured live agent directory. Restore
verifies the archive and its SQLite databases before creating or writing the
target, refuses a non-empty target, and removes an incomplete extraction if
anything fails. It never restores in place and has no --force mode. The
extracted layout retains the archive root, manifest, and payload/ paths
exactly as recorded in the archive.
Activation is a separate offline operator step. Stop the Gateway, move the
restored state asset into place or point OPENCLAW_STATE_DIR at that asset,
then run openclaw doctor before restarting. Use manifest.json as the source
of truth for the state, config, credentials, workspace, and configured agent
paths. Restore custom agent roots to the locations configured by agentDir, or
update those settings to their new locations before restarting. See
Restore a full archive for the full
disaster-recovery sequence.
SQLite snapshots
Use openclaw backup sqlite when you need a portable artifact for one OpenClaw-owned SQLite database instead of a broad state archive.
Snapshot creation accepts exactly one named source. Agent sources always use
the current configuration's resolved <agentDir>/openclaw-agent.sqlite, even
when agentDir is outside the state directory:
| Command | Database |
|---|---|
openclaw backup sqlite create --global --repository <dir> |
Shared OpenClaw state |
openclaw backup sqlite create --agent <id> --repository <dir> |
One per-agent database |
The repository contains one directory per committed snapshot. Each snapshot directory contains exactly:
manifest.jsondatabase.sqlite
Snapshot creation verifies the live database before reading it, uses SQLite's online backup API to capture committed WAL state without holding one long read transaction, closes the live database, compacts the private copy with VACUUM, verifies the generated database again, and publishes the completed directory without overwriting existing paths. Global snapshots remove every delivery queue row before compaction, including pending work, failed ownership fences, and completion or idempotency receipts, so neither payload detail nor ownership tombstones are published or retained in free pages. Restoring this sanitized, portable snapshot is therefore not an exactly-once delivery continuation boundary. This is an intentional privacy and no-replay portability tradeoff.
Do not copy live .sqlite, -wal, -shm, or -journal files as a portability artifact. Copy only completed snapshot directories.
SQLite snapshots can contain auth profiles, session state, plugin state, and other sensitive records. Protect repositories with the same permissions, encryption, retention policy, and destination restrictions as the live OpenClaw state directory.
Verify and restore
openclaw backup sqlite verify <snapshot-directory>
openclaw backup sqlite restore <snapshot-directory> --target <new-database-path>
Verification checks the strict manifest shape, artifact size and SHA-256, SQLite integrity, foreign keys, schema version, database role and owner, and OpenClaw-owned index definitions.
Verification validates a private content-pinned copy so pathname races cannot swap the bytes SQLite inspects. By default, that temporary copy is created beside the snapshot repository and removed before the command returns. The staging root and its ancestor chain must prevent other users from replacing it. POSIX roots must be current-user-owned and not group/world writable; sticky ancestors such as /tmp are accepted for user-owned children. macOS ACL grants that expose or make staging replaceable are rejected. Windows roots and ancestors must be owned by the current user or a trusted OS principal, with ACLs that deny untrusted staging access. For a read-only mount or network share, pass --scratch <existing-private-directory> on storage with equivalent encryption and destination controls.
Snapshot creation applies the same owner, ACL, ancestor, and path-identity checks to the repository before staging or publishing database bytes. Newly created directory edges and final publication metadata are synchronized through the shared fs-safe durability boundary before success is reported on supported filesystems.
Restore repeats verification and writes only to a fresh target. It refuses an existing target, -wal, -shm, or -journal sidecar and never performs an in-place replacement of a live OpenClaw database. The target parent has the same path-security requirements as verification scratch. Activating a restored database remains an explicit offline operator step.
Snapshot repositories are local directories. Scheduling, upload, retention, incremental WAL bundles, failover, and restore-on-boot behavior are intentionally outside this command.
Versioned Git backups
openclaw backup git stores deterministic, per-table JSONL dumps in a plain Git repository owned by the operator. One repository can hold the shared database and every per-agent database:
global/manifest.json
global/schema.sql
global/tables/<table>.jsonl
agents/<agentId>/manifest.json
agents/<agentId>/schema.sql
agents/<agentId>/tables/<table>.jsonl
Initialize the repository, then create a snapshot of the shared database and all configured agent databases:
openclaw backup git init --repository ~/Backups/openclaw-git --remote <private-git-url>
openclaw backup git create --repository ~/Backups/openclaw-git --all --push
The repository root must be owned by the current user and must not be group- or
world-writable. OpenClaw checks this when initializing or adopting a repository
and before every create. On POSIX systems, repair unsafe permissions with
chmod 700 <repository> after confirming its ownership.
The repository must be dedicated to OpenClaw backups. An existing global/ or
agents/<agentId>/ scope is backup-owned only when it is empty or contains a
valid schema-version-1 manifest.json. OpenClaw refuses to replace any other
scope. With --all, it validates every existing entry under agents/ before
removing stale backup-owned agent scopes, so an unowned entry aborts the cleanup
before anything is deleted.
You can also select --global, repeat --agent <id>, or combine the shared database with selected agents. Explicit agent selections, --all, and scheduled backups resolve each database from its configured agentDir; historical artifact verification and restore use the artifact's recorded agent id without requiring that agent to remain in the current configuration. Snapshot creation uses the same online backup, sanitizer, VACUUM, owner validation, and integrity checks as backup sqlite create; it never reads live SQLite files directly. Rows and schema entries have deterministic ordering, and integers and blobs use lossless encodings. The command creates one commit named openclaw backup <ISO8601>. If the database content is unchanged, it prints no changes and creates no commit.
Git staging is restricted to the backup-owned global and agents paths;
unrelated files elsewhere in an adopted repository are never staged.
--push pushes the current branch to origin. A push failure after a successful local commit is a warning and does not discard or mark the local backup as failed.
src/state/secret-state-tables.ts is the source of truth for redaction. At this revision, --exclude-secrets omits these shared-state tables:
audit_identity_keysapns_registrationschannel_ingress_eventschannel_pairing_requestsclawhub_promotion_claimsconfig_revision_keysdevice_auth_tokensdevice_bootstrap_tokensdevice_identitiesdevice_pairing_join_codesdevice_pairing_pairedgateway_origin_device_tokensmcp_oauth_pending_authorizationsmcp_oauth_storesnative_hook_relay_bridgessecret_store_entriesweb_push_subscriptionsworker_environment_credentials
It also omits config_machine_state rows whose keys begin with authProfiles.,
nodeHost., or webPush.vapidKeys, while retaining other machine-state rows.
It omits these per-agent tables:
auth_profile_stateauth_profile_storesession_suggestions
The backup manifest records omitted tables in excludedTables and omitted
machine-state prefixes in excludedConfigStateKeyPrefixes. Restore reports
omitted tables and machine-state prefixes so a redacted snapshot cannot be
mistaken for a complete credential backup.
Inspect or verify history without changing the live databases:
openclaw backup git log --repository ~/Backups/openclaw-git --limit 20
openclaw backup git verify --repository ~/Backups/openclaw-git --ref <commit> --global
openclaw backup git verify --repository ~/Backups/openclaw-git --ref <commit> --agent main
Verification restores the selected snapshot into private scratch space, checks each table's row count and SHA-256, runs PRAGMA integrity_check and PRAGMA foreign_key_check, and removes the scratch copy. Restore writes only to a fresh target and refuses existing -wal, -shm, and -journal sidecars:
openclaw backup git restore --repository ~/Backups/openclaw-git --ref <commit> --global --target ./restored/openclaw.sqlite
Restore rebuilds content-backed FTS5 indexes after loading their content tables. It deliberately omits the derived session_transcript_index_state projection so Gateway startup reconciliation rebuilds transcript search. vec0 virtual tables are not materialized because the extension is unavailable in the restore process; memory indexing recreates them and schedules a full reindex.
Schedule backups
Provision one Gateway-owned automation with a fixed name:
openclaw backup enable --repository ~/Backups/openclaw-git --every 24h --push
The default scope is every database. Use --global-only or --agent <id> to narrow it, and add --exclude-secrets for a redacted history. Pushed schedules (--push) redact credential-bearing tables and secret-prefixed machine-state rows by default because an unattended recurring push retains them durably in remote history; pass --include-secrets for explicit full-fidelity remote backups (restores from redacted history need device re-pairing and provider re-authentication). --push also requires the repository to already have an origin remote. Re-running backup enable updates the existing automation instead of creating a duplicate. openclaw backup disable removes it; disabling an already-missing job is a successful no-op. Backup scheduling currently requires a local Gateway because the command job runs on the Gateway host; for a remote Gateway, create the cron job manually with openclaw cron add.
Recorded runs and freshness
Every real archive, SQLite snapshot, and Git create attempt records a compact outcome in the existing shared state database. Dry runs are not recorded. The log retains the newest 200 attempts, so frequent schedules remain bounded.
openclaw status shows one Backups overview row, and openclaw status --json includes the latest attempt and latest successful run. openclaw doctor prints an informational hint when no successful backup is recorded or the newest successful backup is more than 14 days old. Recording is best-effort: a record-write failure prints a warning but never changes a successful backup into a failed command.
What gets backed up
openclaw backup create plans sources from your local OpenClaw install:
- The state directory (usually
~/.openclaw) - The active config file path
- The resolved
credentials/directory when it exists outside the state directory - Every configured agent directory, including custom
agentDirroots outside the state directory - Workspace directories discovered from the current config, unless you pass
--no-include-workspace - Durable resources declared by effectively activated, loadable plugin manifests
Auth profiles and other per-agent runtime state live in
<agentDir>/openclaw-agent.sqlite. The default agent root is
<stateDir>/agents/<agentId>/agent, but a custom root remains authoritative
whether it is outside the state directory, inside a workspace, or nested under
an otherwise regenerable managed state root. --no-include-workspace omits
ordinary workspace sources, not configured agent directories.
--only-config skips state, agent, credentials-directory, workspace, and
plugin-resource discovery and archives only the active config file path.
OpenClaw builds one immutable, configuration-derived ownership inventory before
planning sources, SQLite snapshots, exclusions, results, and the embedded
manifest. Paths are canonicalized: config, credentials, workspaces, and agents
already covered by another included root are not duplicated as top-level
sources. A custom agent root becomes a distinct agent asset only when no
existing asset covers it; the manifest still records its agent id and root when
another asset contains it. Missing paths are reported as skipped.
During archive creation, OpenClaw excludes known live-mutation paths before tar reads them. This avoids races between a file's recorded size and concurrent writes. The filter applies these state-relative rules under each backed-up state directory:
| State-relative scope | Skipped entries |
|---|---|
sessions/** |
.jsonl, .log |
agents/<agentId>/sessions/** |
.jsonl, .log |
cron/runs/** |
.jsonl, .log |
logs/** |
.jsonl, .log |
delivery-queue/** |
.json, .delivered, .tmp |
session-delivery-queue/** |
.json, .delivered, .tmp |
browser/<profile>/user-data/ |
SingletonCookie, SingletonLock, SingletonSocket |
sandbox/skills-workspaces/** |
All entries |
| Any path under the backed-up state directory | .sock, .pid, .tmp |
These rules do not filter workspace files outside the state directory. They also omit completed transcript and log files that match the table, so retain those records separately when needed. The JSON result's skippedVolatileCount reports intentionally omitted volatile entries; regenerable agent temporary roots are listed separately in skipped and are not included in that count.
Chromium singleton entries coordinate one running browser on one host and are recreated when that profile starts; the rest of the profile's user-data/ remains in the archive. Sandbox skills workspaces are generated copies of current skill sources and are materialized again when OpenClaw prepares the next sandbox context after restore; adjacent sandbox registry and other durable state remain included.
SQLite databases owned by the state directory or any configured agent directory
are captured with SQLite's online backup API and compacted offline with
VACUUM, including custom agent roots covered by a workspace or managed state
asset. Committed WAL changes are included, deleted-page remnants and transient
leases are removed, sidecars are omitted, and canonical OpenClaw databases must
match their expected role and agent owner. Unsafe aliasing or an owner mismatch
fails closed. A plugin-owned database that requires unavailable owner-defined
SQLite capabilities also fails closed rather than falling back to a direct file
copy. Other workspace SQLite files outside configured agent roots remain raw
workspace files and do not receive the SQLite snapshot or compaction guarantee.
Installed plugin source and manifest files under the state directory's extensions/ tree are included, but their nested node_modules/ dependency trees are skipped as rebuildable install artifacts. After restoring an archive, use openclaw plugins update <id> or reinstall with openclaw plugins install <spec> --force if a restored plugin reports missing dependencies.
The state directory's plugin-skills/ root is a generated, OpenClaw-owned symlink index, not authoritative state. Backup creation reports and omits that root because its absolute targets are specific to the source installation. After activating restored state, run openclaw skills list or start an agent session to rebuild the links from current plugin metadata.
Agent-scoped temporary trees under agents/<agentId>/agent/**/{tmp,.tmp}/ are also omitted and reported as regenerable. This includes temporary files directly below an agent directory and temporary trees inside agent runtime homes; durable sibling directories remain included. An explicitly configured config file, credentials directory, or workspace nested below an omitted temporary root remains included.
Symbolic links are archived as link metadata and are never followed. Relative links are retained only when both the link and its lexical target remain within backup assets declared in manifest.json; links between declared assets and dangling links within an asset are allowed. Absolute links, links containing backslashes, and links escaping the archive root or every declared asset are rejected during both creation and verification.
Installer-managed and rebuildable runtime roots under the state directory are
also skipped: dev/, git/, npm/, legacy npm-runtime/, tmp/, and
tools/. These contain managed checkouts, package trees, compiler caches,
temporary files, and downloaded runtimes rather than authoritative user state;
reinstall or update the corresponding runtime or plugin after restore.
Effectively activated, loadable plugins can declare additional durable or
regenerable state- or agent-relative roots through
backupResources. Disabled or
unloadable plugins cannot exclude data. Explicit config, credentials, workspace,
agent, and plugin-included paths override exclusions, and any excluded parent
remains traversable to reach those protected descendants. Names such as tmp
and .tmp are not blanket exclusions in custom agent directories; only an
applicable owner declaration can omit their durable-looking siblings.
Local edits inside a managed dev/ checkout are developer source, not OpenClaw product state, and are not included. Commit and push those edits or copy the checkout separately before relying on a state backup.
Invalid config behavior
openclaw backup bypasses the normal config preflight so it can still help during recovery. Workspace discovery depends on a valid config, so openclaw backup create fails fast when the config file exists but is invalid and workspace backup is still enabled.
For a partial backup in that situation, rerun with
--no-include-workspace: it keeps state, config, and the external credentials
directory in scope without workspace discovery. Because malformed configuration
also prevents resolving custom agent ownership and effectively activated plugin
resources, the result records those unresolved scopes as skipped diagnostics;
do not treat that recovery archive as a complete backup.
--only-config also works when the config is malformed, since it does not parse the config for workspace discovery.
Size and performance
OpenClaw does not enforce a built-in maximum backup size or per-file size limit. An archive write that produces no data for five minutes fails and removes its partial temporary file instead of hanging indefinitely. Practical limits otherwise come from:
- Available space for the temporary archive write plus the final archive
- Time to walk large workspace trees and compress them into a
.tar.gz - Time to rescan the archive with
--verifyoropenclaw backup verify - Destination filesystem behavior: OpenClaw requires no-overwrite hard-link publication so a final archive path never exposes an in-progress copy; unsupported filesystems fail with an actionable error
If final-directory durability confirmation fails after publication, the command reports failure but preserves the complete final entry rather than risk deleting a concurrent replacement.
Large workspaces are usually the main driver of archive size. Use --no-include-workspace for a smaller/faster backup, or --only-config for the smallest archive.