refactor(state): consolidate wide rows, plugin index, workspace attestations, and shared auth singletons at schema v13 (#130466)

* refactor(state): make cron and subagent rows JSON-canonical

* refactor(state): make gateway origin device tokens canonical at v13

The lazy ensure predates the table joining the canonical schema; at the
v13 bump the schema owns creation, so the feature-local DDL, WeakSet
dedupe, and lazy-list entry retire. The legacy-file guard the ensure
carried stays at each call site.

* test: drop obsolete lazy-ensure coverage for origin device tokens

The table is canonical at v13; same-version lazy creation no longer
exists to protect. Origin CRUD, isolation, and rotation coverage remains
in the surviving cases.

* refactor(state): fold installed_plugin_index into config_machine_state

The singleton index row becomes one JSON value under
plugins.installedIndex with its rollback-fencing revision inside the
value; reads, CAS restore, and the lease-held write transactions use
direct Kysely on config_machine_state so the state_leases assertion
stays in-transaction. The v13 migration imports the row and drops the
table; the additive workspace_dir entry folds with it. Doctor guidance,
docker staging, and the e2e probes name the machine-state row.

* refactor(state): merge workspace_attestations into workspace_setup_state

One row per workspace now carries both setup milestones and the
attestation clock: nullable setup columns represent attestation-only
workspaces (replaceWorkspaceAttestation can precede any setup write) and
setupExists derives from a non-null version. The bootstrap-hash FK
repoints to the merged table; migration receipts keep the historical
workspace_attestations discriminator string. The v13 migration grows and
rebuilds the table, merges attestation rows (orphans without a path
alias drop — their hashes re-derive at the next bootstrap attestation),
and the consolidation kind is renamed state-consolidation-v13 to cover
the batch.

* test(state): cover the workspace merge and consolidation fallout

The v12-to-v13 regression seeds merged, attestation-only, and orphan
attestation workspaces; the 13-to-12 downgrade fixture recreates
workspace_attestations and installed_plugin_index from the folded data;
the fold-in migration gates the additive workspace_dir column for
pre-additive rows; the workspace merge now triggers on the setup table's
own shape so stable-era databases without an attestations table still
reshape; the consolidation applied-message covers the batch.

* refactor(state): fold shared auth profile singletons into config_machine_state

The shared-state auth_profile_stores/auth_profile_state rows (fixed key
'shared') become authProfiles.store/authProfiles.state machine-state
values; the agent-DB tables of the same names are untouched. Git-backup
redaction moves from table-drop to the authProfiles. secret prefix with
seeded-secret absence proof; migration receipts keep the historical
table-name discriminators; the shared-auth relocation and receipt
verification project the KV cells back to the receipt-era row shapes so
persisted digests stay byte-compatible. mcp_oauth_stores stays a table —
its multi-key fold is a named follow-up.

* test(state): finish shared-auth fold coverage and annotate boundary casts

Auth seeders and assertions across the e2e/scripts/secrets suites target
the authProfiles machine-state cells; the v12-to-v13 regression proves
payload-byte fidelity, non-shared-row drop, and insert-if-absent
precedence; the downgrade fixture recreates and repopulates both v12
tables. Boundary type assertions in the plugin-index store carry SAFETY
invariants per the ratchet.

* chore: shrink assertion-safety baseline for plugin-index store

* refactor(doctor): delete the dead onboarding-recommendations migration

Its input — the unscoped 'primary' onboarding row — existed only between
9a93a52a8a and 473962b7de, a two-day beta window; no shipped stable
can produce it and the runtime table folded away at v12. The audit
backup list keeps recognizing system-agent.jsonl artifacts because beta
installs that ran that import may still carry its backups.

* docs: sync the 13-to-12 downgrade example with the executable fixture

* style: format the synced downgrade example

* style: drop unused import and duplicate union constituent

* fix(state): keep orphan attestations across the v13 workspace merge

The merged workspace_setup_state required a workspace path, but legacy
orphan hashed-key attestations never recorded one. workspace_path is now
nullable (setup rows still enforce it via CHECK), the v13 migration and
the doctor file import keep orphans with a NULL path that heals on the
next live access, and the 13-to-12 downgrade keeps attestation-owned
hashes. Doctor test seeds move to the folded KV row.

* perf(state): retire unused cron indexes

* fix(state): preserve v13 migration recovery

* fix(state): preserve v12 lazy-table upgrade

* docs(state): document v13 auth relocation

---------

Co-authored-by: Vincent Koc <vincentkoc@ieee.org>

scripts/e2e/lib/auth-profile-store-assertions.mjs

@@ -14,17 +14,17 @@ export function readSharedAuthProfileStoreText(stateDir) {
     db = new DatabaseSync(dbPath, { readOnly: true });
     const schema = db
       .prepare("SELECT type FROM sqlite_schema WHERE name = ? LIMIT 1")
-      .get("auth_profile_stores");
+      .get("config_machine_state");
     if (!schema) {
       return "";
     }
     if (schema.type !== "table") {
-      throw new Error(`auth_profile_stores is ${String(schema.type)}, not a table`);
+      throw new Error(`config_machine_state is ${String(schema.type)}, not a table`);
     }
     const row = db
-      .prepare("SELECT store_json FROM auth_profile_stores WHERE store_key = ?")
-      .get("shared");
-    return typeof row?.store_json === "string" ? row.store_json : "";
+      .prepare("SELECT value_json FROM config_machine_state WHERE state_key = ?")
+      .get("authProfiles.store");
+    return typeof row?.value_json === "string" ? row.value_json : "";
   } catch (error) {
     const detail = error instanceof Error ? error.message : String(error);
     throw new Error(`could not read the shared auth profile store: ${detail}`, {

scripts/e2e/lib/plugin-index-sqlite.mjs

@@ -5,7 +5,7 @@ import { DatabaseSync } from "node:sqlite";
 import { readPositiveIntEnv } from "./env-limits.mjs";
 import { readTextFileBounded } from "./text-file-utils.mjs";
 
-const INDEX_KEY = "installed-plugin-index";
+const STATE_KEY = "plugins.installedIndex";
 const ERROR_DETAIL_TAIL_BYTES = 16 * 1024;
 const JSON_ARTIFACT_MAX_BYTES = readPositiveIntEnv(
   "OPENCLAW_PLUGIN_INDEX_JSON_MAX_BYTES",
@@ -80,53 +80,24 @@ function readSqlitePluginIndex(root = stateDir()) {
     const lengths = db
       .prepare(
         `
-          SELECT octet_length(install_records_json) AS install_records_json_bytes,
-                 octet_length(plugins_json) AS plugins_json_bytes,
-                 octet_length(diagnostics_json) AS diagnostics_json_bytes
-            FROM installed_plugin_index
-           WHERE index_key = ?
+          SELECT octet_length(value_json) AS value_json_bytes
+            FROM config_machine_state
+           WHERE state_key = ?
         `,
       )
-      .get(INDEX_KEY);
+      .get(STATE_KEY);
     if (!lengths) {
       return {};
     }
-    assertIndexJsonByteLength(
-      lengths.install_records_json_bytes,
-      "plugin index install_records_json",
-    );
-    assertIndexJsonByteLength(lengths.plugins_json_bytes, "plugin index plugins_json");
-    assertIndexJsonByteLength(lengths.diagnostics_json_bytes, "plugin index diagnostics_json");
+    assertIndexJsonByteLength(lengths.value_json_bytes, "plugin index value_json");
     const row = db
-      .prepare(
-        `
-          SELECT version, warning, host_contract_version, compat_registry_version,
-                 migration_version, policy_hash, generated_at_ms, refresh_reason,
-                 install_records_json, plugins_json, diagnostics_json
-            FROM installed_plugin_index
-           WHERE index_key = ?
-        `,
-      )
-      .get(INDEX_KEY);
+      .prepare("SELECT value_json FROM config_machine_state WHERE state_key = ?")
+      .get(STATE_KEY);
     if (!row) {
       return {};
     }
-    return {
-      version: Number(row.version),
-      ...(row.warning ? { warning: row.warning } : {}),
-      hostContractVersion: row.host_contract_version,
-      compatRegistryVersion: row.compat_registry_version,
-      migrationVersion: Number(row.migration_version),
-      policyHash: row.policy_hash,
-      generatedAtMs: Number(row.generated_at_ms),
-      ...(row.refresh_reason ? { refreshReason: row.refresh_reason } : {}),
-      installRecords: parseIndexJsonText(
-        row.install_records_json,
-        "plugin index install_records_json",
-      ),
-      plugins: parseIndexJsonText(row.plugins_json, "plugin index plugins_json"),
-      diagnostics: parseIndexJsonText(row.diagnostics_json, "plugin index diagnostics_json"),
-    };
+    const value = parseIndexJsonText(row.value_json, "plugin index value_json");
+    return value?.index && typeof value.index === "object" ? value.index : {};
   } catch (error) {
     if (error?.code === "ETOOBIG") {
       throw error;
@@ -168,59 +139,40 @@ export function writePluginInstallIndexForE2E(index, options = {}) {
   const db = new DatabaseSync(dbPath);
   try {
     db.exec(`
-      CREATE TABLE IF NOT EXISTS installed_plugin_index (
-        index_key TEXT NOT NULL PRIMARY KEY,
-        version INTEGER NOT NULL,
-        host_contract_version TEXT NOT NULL,
-        compat_registry_version TEXT NOT NULL,
-        migration_version INTEGER NOT NULL,
-        policy_hash TEXT NOT NULL,
-        generated_at_ms INTEGER NOT NULL,
-        refresh_reason TEXT,
-        install_records_json TEXT NOT NULL,
-        plugins_json TEXT NOT NULL,
-        diagnostics_json TEXT NOT NULL,
-        warning TEXT,
+      CREATE TABLE IF NOT EXISTS config_machine_state (
+        state_key TEXT NOT NULL PRIMARY KEY,
+        value_json TEXT NOT NULL,
         updated_at_ms INTEGER NOT NULL
       );
     `);
     const now = Date.now();
+    const persisted = {
+      revision: now,
+      index: {
+        version: index.version ?? 1,
+        warning:
+          index.warning ??
+          "DO NOT EDIT. This row is generated by OpenClaw plugin registry commands.",
+        hostContractVersion: index.hostContractVersion ?? "docker-e2e",
+        compatRegistryVersion: index.compatRegistryVersion ?? "docker-e2e",
+        migrationVersion: index.migrationVersion ?? 1,
+        policyHash: index.policyHash ?? "docker-e2e",
+        generatedAtMs: index.generatedAtMs ?? now,
+        ...(index.refreshReason ? { refreshReason: index.refreshReason } : {}),
+        installRecords: index.installRecords ?? {},
+        plugins: index.plugins ?? [],
+        diagnostics: index.diagnostics ?? [],
+      },
+    };
     db.prepare(
       `
-        INSERT INTO installed_plugin_index (
-          index_key, version, host_contract_version, compat_registry_version,
-          migration_version, policy_hash, generated_at_ms, refresh_reason,
-          install_records_json, plugins_json, diagnostics_json, warning, updated_at_ms
-        ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
-        ON CONFLICT(index_key) DO UPDATE SET
-          version = excluded.version,
-          host_contract_version = excluded.host_contract_version,
-          compat_registry_version = excluded.compat_registry_version,
-          migration_version = excluded.migration_version,
-          policy_hash = excluded.policy_hash,
-          generated_at_ms = excluded.generated_at_ms,
-          refresh_reason = excluded.refresh_reason,
-          install_records_json = excluded.install_records_json,
-          plugins_json = excluded.plugins_json,
-          diagnostics_json = excluded.diagnostics_json,
-          warning = excluded.warning,
+        INSERT INTO config_machine_state (state_key, value_json, updated_at_ms)
+        VALUES (?, ?, ?)
+        ON CONFLICT(state_key) DO UPDATE SET
+          value_json = excluded.value_json,
           updated_at_ms = excluded.updated_at_ms
       `,
-    ).run(
-      INDEX_KEY,
-      index.version ?? 1,
-      index.hostContractVersion ?? "docker-e2e",
-      index.compatRegistryVersion ?? "docker-e2e",
-      index.migrationVersion ?? 1,
-      index.policyHash ?? "docker-e2e",
-      index.generatedAtMs ?? now,
-      index.refreshReason ?? null,
-      JSON.stringify(index.installRecords ?? {}),
-      JSON.stringify(index.plugins ?? []),
-      JSON.stringify(index.diagnostics ?? []),
-      index.warning ?? "DO NOT EDIT. This row is generated by OpenClaw plugin registry commands.",
-      now,
-    );
+    ).run(STATE_KEY, JSON.stringify(persisted), now);
   } finally {
     db.close();
   }

scripts/e2e/lib/upgrade-survivor/sqlite-volume.mjs

@@ -427,9 +427,8 @@ export function assertUpgradeVolumeMigrated(stateDir, stage) {
   assertHealthySqlite(stateDatabasePath, (db) => {
     const rows = db
       .prepare(
-        `SELECT job_id, job_json, state_json, enabled, schedule_kind, every_ms, anchor_ms,
-                payload_kind, payload_message, delivery_mode, next_run_at_ms, running_at_ms,
-                last_run_status, last_error, updated_at, runtime_updated_at_ms
+        `SELECT job_id, job_json, state_json, enabled, payload_kind, updated_at,
+                runtime_updated_at_ms
          FROM cron_jobs
          WHERE job_id LIKE 'volume-cron-%'`,
       )
@@ -468,26 +467,23 @@ export function assertUpgradeVolumeMigrated(stateDir, stage) {
         row?.enabled === (expected.enabled ? 1 : 0),
         `volume cron enabled column changed: ${index}`,
       );
-      assert(row?.schedule_kind === "every", `volume cron schedule column changed: ${index}`);
-      assert(row?.every_ms === expected.schedule.everyMs, `volume cron interval changed: ${index}`);
-      assert(row?.anchor_ms === expected.schedule.anchorMs, `volume cron anchor changed: ${index}`);
       assert(row?.payload_kind === "agentTurn", `volume cron payload kind changed: ${index}`);
       assert(
-        row?.payload_message === expected.payload.message,
-        `volume cron payload changed: ${index}`,
-      );
-      assert(row?.delivery_mode === "none", `volume cron delivery mode changed: ${index}`);
-      assert(
-        row?.next_run_at_ms === (expected.enabled ? expected.state.nextRunAtMs : null),
+        (actualState?.nextRunAtMs ?? null) ===
+          (expected.enabled ? expected.state.nextRunAtMs : null),
         `volume cron next-run state changed: ${index}`,
       );
-      assert(row?.running_at_ms === null, `volume cron running state changed: ${index}`);
       assert(
-        row?.last_run_status === (expected.state.lastStatus ?? null),
+        (actualState?.runningAtMs ?? null) === null,
+        `volume cron running state changed: ${index}`,
+      );
+      assert(
+        (actualState?.lastRunStatus ?? actualState?.lastStatus ?? null) ===
+          (expected.state.lastStatus ?? null),
         `volume cron status state changed: ${index}`,
       );
       assert(
-        row?.last_error === (expected.state.lastError ?? null),
+        (actualState?.lastError ?? null) === (expected.state.lastError ?? null),
         `volume cron error state changed: ${index}`,
       );
     }