Commit Graph

12 Commits

Author SHA1 Message Date
Peter Steinberger 37b4fc8621 feat(backup): recorded runs, scheduled backups, and git-backed versioned snapshots (#122485)
* refactor(infra): extract shared git exec and verified snapshot-copy helpers

Moves the worktrees git wrapper to src/infra/git-exec.ts (with optional
maxOutputBytes for large buffered reads) and the online-backup/sanitize/
VACUUM/verify snapshot step into src/snapshot/openclaw-snapshot-copy.ts so
snapshot backends share one hardened copy path. Behavior-identical moves;
all importers updated.

* feat(snapshot): git-backed versioned SQLite snapshot engine

Deterministic per-table JSONL dumps (PK-ordered, lossless bigint/blob
encoding), verbatim DDL preservation, virtual/shadow-table skipping with
FTS rebuild on restore, secret-table redaction policy, manifest with
per-table row counts and content hashes, and restore verification by
re-serialization. Unchanged data produces no commit.

* feat(backup): recorded runs, freshness surfacing, and scheduled git backups

Every backup attempt is recorded in the previously writer-less backup_runs
table (bounded to 200 rows). openclaw status gains a Backups overview row
and JSON payload; doctor prints an informational hint when no successful
backup is recorded or the newest is stale. New commands: backup git
init/create/log/verify/restore and backup enable/disable, which provision
one idempotent gateway cron job running scheduled git backups.

* fix(state): stop bumping schema_meta.updated_at on unchanged opens

updated_at now records when schema metadata actually changed instead of
when the database was last opened; unconditional bumps dirtied the row on
every open and defeated no-change backup detection.

* docs: document versioned git backups, scheduling, and backup freshness

* fix(backup): satisfy CI ownership checks

* fix(backup): complete CI contract coverage

* fix(backup): complete credential table redaction

* fix(backup): isolate git repository ownership

* fix(backup): persist push degradation

* fix(backup): atomically converge schedules

* fix(status): isolate backup freshness environment

* fix(status): carry scan environment to freshness reads

* fix(backup): harden Git repository ownership

* docs(backup): document Git repository safety

* fix(backup): non-creating outcome log and origin preflight for pushed schedules

Recording a backup outcome never bootstraps an absent state database (a
failed backup on a fresh host would otherwise create a blank DB that a
retry treats as real input), and backup enable --push now requires the
repository to have an origin remote, pointing at backup git init --remote
instead of scheduling permanently degraded pushes.

* refactor(worktrees): use shared git exec helpers

* refactor(worktrees): remove unused git buffer wrapper

* refactor(worktrees): consume buffered git helper

* feat(backup): redact pushed schedules by default

Unattended recurring pushes retain credential-bearing tables durably in
remote Git history, so backup enable --push now defaults to
--exclude-secrets; --include-secrets is the explicit full-fidelity
override (still warned). Local non-push schedules keep full fidelity for
complete restores.

* fix(backup): redact audit HMAC and OAuth pending state; tolerate absent backup_runs

Adds audit_identity_keys (audit HMAC key) and mcp_oauth_pending_authorizations
(live OAuth callback state) to the redaction inventory, and makes read-only
backup freshness treat a same-version database without the additive
backup_runs table as no recorded backups instead of failing before a
writable open converges the schema.

* fix(backup): restrict schedules to local gateways

* fix(snapshot): harden Git restore and redaction

* fix(backup): block pushes of adopted history

* fix(backup): contain commits and pairing secrets
2026-08-12 08:11:22 -07:00
Peter Steinberger 3856b4fa1b refactor(cli): unify gateway RPC transport (#117601) 2026-08-01 14:06:30 -07:00
Peter Steinberger 433bb3f954 fix(cli): make commands, completion, and JSON output reliable (#116033)
* fix(cli): make commands, completion, and JSON output reliable

* fix(cli): reconcile completion coverage with current main

* test(cli): keep test routing stable across isolation lanes
2026-07-31 16:20:34 -07:00
JC bd5bf4820a fix(cli): retry logs.tail after journal fallback in logs follow (#88159)
* fix(cli): retry logs.tail after journal fallback in logs follow

Rebase #88159 onto current main and keep systemd journal fallback temporary in follow mode. Preserve the journal cursor across repeated fallback outages, but retry logs.tail on the next loop so recovered Gateway RPC returns to normal log output.

This refresh also replaces the stale red checks-node-core-fast result from the old head with a current-head CI run.

* Keep log source metadata explicit

* ci: retrigger checks for PR #88159

* docs: clarify logs follow JSON source transitions

* fix(cli): keep journal logs responsive during recovery

---------

Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
2026-07-01 06:55:11 -07:00
openclaw-clownfish[bot] 99d0bdc23a fix(cli): validate gateway RPC timeout inputs
Reject malformed or explicit empty Gateway RPC timeout values before opening Gateway calls, align the shared Gateway RPC omitted-timeout fallback with the 30000 ms CLI default, and validate explicit `cron add --timeout-seconds` values at the CLI boundary.

Carries forward the useful source work from #54646 and the earlier timeout-validation context from #40953. #60661 remains separate accepted-run timeout semantics work and is intentionally not folded into this change.

Validation:
- `npm run review-results -- /tmp/clownfish-check-27341769444`
- `git diff --check`
- OpenClaw PR checks on `ce7bd8b9388a5689b14ddc2b3a984f7b4647e5ca`: 132 pass, 0 pending, 0 failing
- ClawSweeper re-review: https://github.com/openclaw/clawsweeper/actions/runs/27344244608

Co-authored-by: RayRuan <43744645+ruanrrn@users.noreply.github.com>
Co-authored-by: Homeran <11574611+comeran@users.noreply.github.com>
2026-06-11 20:52:07 +09:00
Peter Steinberger 69c27677f6 docs: document gateway cli helpers 2026-06-04 10:56:59 -04:00
Peter Steinberger b1117d9862 refactor: extract gateway client package (#87797)
* refactor: extract gateway client package

* chore: drop generated gateway package artifacts

* refactor: move gateway protocol package

* refactor: remove old gateway protocol tree

* test: keep auth compat split in run mode

* test: expose gateway wrapper options for internals

* fix: watch moved gateway package sources

* test: normalize slash command import guard

* chore: teach knip gateway package entries

* ci: route gateway client package checks

* fix: reuse ipaddr for gateway client hosts

* fix: sync gateway protocol usage schema
2026-05-29 02:23:42 +01:00
Peter Steinberger e61f175203 fix(cli): reject malformed gateway timeouts 2026-05-27 08:57:03 -04:00
Peter Steinberger 4a24b23e3e fix(ci): stabilize full release validation 2026-04-28 20:14:14 +01:00
Vincent Koc 0f7d9c9570 fix(runtime): split approval and gateway client seams 2026-04-11 18:36:48 +01:00
Vincent Koc dfdc281f55 fix(cycles): split small runtime seams 2026-04-10 09:00:19 +01:00
Shakker 23422ccb68 refactor: lazy load cli gateway rpc runtime 2026-04-04 01:07:28 +09:00