mirror of
https://github.com/openclaw/openclaw.git
synced 2026-08-12 21:53:00 -06:00
Peter Steinberger
37b4fc8621
* refactor(infra): extract shared git exec and verified snapshot-copy helpers Moves the worktrees git wrapper to src/infra/git-exec.ts (with optional maxOutputBytes for large buffered reads) and the online-backup/sanitize/ VACUUM/verify snapshot step into src/snapshot/openclaw-snapshot-copy.ts so snapshot backends share one hardened copy path. Behavior-identical moves; all importers updated. * feat(snapshot): git-backed versioned SQLite snapshot engine Deterministic per-table JSONL dumps (PK-ordered, lossless bigint/blob encoding), verbatim DDL preservation, virtual/shadow-table skipping with FTS rebuild on restore, secret-table redaction policy, manifest with per-table row counts and content hashes, and restore verification by re-serialization. Unchanged data produces no commit. * feat(backup): recorded runs, freshness surfacing, and scheduled git backups Every backup attempt is recorded in the previously writer-less backup_runs table (bounded to 200 rows). openclaw status gains a Backups overview row and JSON payload; doctor prints an informational hint when no successful backup is recorded or the newest is stale. New commands: backup git init/create/log/verify/restore and backup enable/disable, which provision one idempotent gateway cron job running scheduled git backups. * fix(state): stop bumping schema_meta.updated_at on unchanged opens updated_at now records when schema metadata actually changed instead of when the database was last opened; unconditional bumps dirtied the row on every open and defeated no-change backup detection. * docs: document versioned git backups, scheduling, and backup freshness * fix(backup): satisfy CI ownership checks * fix(backup): complete CI contract coverage * fix(backup): complete credential table redaction * fix(backup): isolate git repository ownership * fix(backup): persist push degradation * fix(backup): atomically converge schedules * fix(status): isolate backup freshness environment * fix(status): carry scan environment to freshness reads * fix(backup): harden Git repository ownership * docs(backup): document Git repository safety * fix(backup): non-creating outcome log and origin preflight for pushed schedules Recording a backup outcome never bootstraps an absent state database (a failed backup on a fresh host would otherwise create a blank DB that a retry treats as real input), and backup enable --push now requires the repository to have an origin remote, pointing at backup git init --remote instead of scheduling permanently degraded pushes. * refactor(worktrees): use shared git exec helpers * refactor(worktrees): remove unused git buffer wrapper * refactor(worktrees): consume buffered git helper * feat(backup): redact pushed schedules by default Unattended recurring pushes retain credential-bearing tables durably in remote Git history, so backup enable --push now defaults to --exclude-secrets; --include-secrets is the explicit full-fidelity override (still warned). Local non-push schedules keep full fidelity for complete restores. * fix(backup): redact audit HMAC and OAuth pending state; tolerate absent backup_runs Adds audit_identity_keys (audit HMAC key) and mcp_oauth_pending_authorizations (live OAuth callback state) to the redaction inventory, and makes read-only backup freshness treat a same-version database without the additive backup_runs table as no recorded backups instead of failing before a writable open converges the schema. * fix(backup): restrict schedules to local gateways * fix(snapshot): harden Git restore and redaction * fix(backup): block pushes of adopted history * fix(backup): contain commits and pairing secrets
summary, read_when, title
| summary | read_when | title | |||
|---|---|---|---|---|---|
| Repository script entry points and compatibility notes |
|
Scripts Directory |
Scripts Directory
The scripts/ directory contains repository tooling used by local development,
CI, docs publishing, releases, Docker proof, and maintainer operations. Prefer
the package-script entry points in package.json when one exists, then read the
underlying script before running it directly.
Compatibility
Many scripts are stable paths referenced by package.json, GitHub Actions,
docs, and maintainer runbooks. Do not move, rename, or regroup scripts only to
improve taxonomy. A directory migration needs an explicit maintainer-approved
compatibility plan for package scripts, workflows, docs snippets, and any raw
script paths users may have copied.
This index is a discovery aid for the current flat layout. It does not define a new directory taxonomy.
Common Entry Points
| Area | Prefer | Notes |
|---|---|---|
| Build | pnpm build |
Runs scripts/build-all.mts; use specific build scripts only when debugging a build stage. |
| Changed checks | pnpm changed:lanes --json, pnpm check:changed |
Lane classification lives in scripts/changed-lanes.mjs; changed-file checks live in scripts/check-changed.mjs. |
| Docs | pnpm docs:list, pnpm docs:check-mdx, pnpm docs:check-links |
Backed by scripts/docs-list.js, scripts/check-docs-mdx.mjs, and scripts/docs-link-audit.mjs. |
| Formatting docs | pnpm format:docs:check |
Uses scripts/format-docs.mts; use write mode only when intentionally formatting docs. |
| Lint | pnpm lint, pnpm lint:core, pnpm lint:all |
Wrapper scripts keep oxlint behavior aligned with repo config. |
| Targeted tests | pnpm test <path-or-filter> or node scripts/run-vitest.mjs <path-or-filter> |
Avoid bare vitest; it can start watch mode. |
| Changed tests | pnpm test:changed |
Uses the repo's changed-test resolver instead of a broad Vitest run. |
| Docker proof | pnpm test:docker:all, pnpm test:docker:rerun, pnpm test:docker:timings |
Use the planner/rerun helpers before launching broad Docker work. |
| Live proof | pnpm test:live |
Live checks require the matching environment and credentials. |
| Release checks | pnpm release:check, pnpm release:beta, pnpm release:candidate |
Release scripts are maintainer workflows; read release docs before use. |
| GitHub reads | scripts/gh-read |
Uses a GitHub App read token when configured, leaving normal gh login for writes. |
| Commits | git add <files...> then git commit -m "<message>" |
Stage only the intended files for each commit. |
| Remote proof | node scripts/crabbox-wrapper.mjs ... |
Agent default for tests and heavy work; pre-warm by source trust, sync each run, reuse the lease. |
Script Families
check-*.mts/check-*.ts/ retainedcheck-*.mjs: guardrails for architecture, docs, package contents, boundaries, workflows, and generated artifacts.run-*.mjs/run-*.mts: stable wrappers and typed implementations for Node, Vitest, oxlint, tsgo, and environment setup.test-*.mts/ retainedtest-*.mjs/test-*.sh/test-*.ts: test planners, Docker lanes, live checks, and focused validation helpers.docs-*andcheck-docs-*: docs listing, link auditing, MDX checks, spellcheck, sync, and i18n glossary checks.release-*,openclaw-npm-*, andplugin-*-release-*: release preparation, package verification, and publishing helpers.docker-*,test-docker-*, andtest-live-*-docker.sh: Docker E2E planning, rerun, timing, and live/package lane helpers.gh-read*,label-*,sync-labels.ts, and PR helpers: GitHub read, labeling, and maintainer workflow support.generate-*,write-*,copy-*, andsync-*: generated docs, metadata, package surfaces, and build artifact support.lib/: shared helpers imported by script entry points.
Maintenance Rules
- Read
scripts/AGENTS.mdbefore changing scripts. - Keep package scripts, generators, generated-artifact checks, docs references, and workflow references aligned when touching a script path.
- Prefer existing wrappers instead of introducing a raw tool invocation.
- Add or update focused tests under
test/scripts/when changing script behavior.
See also Scripts for public-facing script guidance.