refactor(state): consolidate wide rows, plugin index, workspace attestations, and shared auth singletons at schema v13 (#130466)

* refactor(state): make cron and subagent rows JSON-canonical

* refactor(state): make gateway origin device tokens canonical at v13

The lazy ensure predates the table joining the canonical schema; at the
v13 bump the schema owns creation, so the feature-local DDL, WeakSet
dedupe, and lazy-list entry retire. The legacy-file guard the ensure
carried stays at each call site.

* test: drop obsolete lazy-ensure coverage for origin device tokens

The table is canonical at v13; same-version lazy creation no longer
exists to protect. Origin CRUD, isolation, and rotation coverage remains
in the surviving cases.

* refactor(state): fold installed_plugin_index into config_machine_state

The singleton index row becomes one JSON value under
plugins.installedIndex with its rollback-fencing revision inside the
value; reads, CAS restore, and the lease-held write transactions use
direct Kysely on config_machine_state so the state_leases assertion
stays in-transaction. The v13 migration imports the row and drops the
table; the additive workspace_dir entry folds with it. Doctor guidance,
docker staging, and the e2e probes name the machine-state row.

* refactor(state): merge workspace_attestations into workspace_setup_state

One row per workspace now carries both setup milestones and the
attestation clock: nullable setup columns represent attestation-only
workspaces (replaceWorkspaceAttestation can precede any setup write) and
setupExists derives from a non-null version. The bootstrap-hash FK
repoints to the merged table; migration receipts keep the historical
workspace_attestations discriminator string. The v13 migration grows and
rebuilds the table, merges attestation rows (orphans without a path
alias drop — their hashes re-derive at the next bootstrap attestation),
and the consolidation kind is renamed state-consolidation-v13 to cover
the batch.

* test(state): cover the workspace merge and consolidation fallout

The v12-to-v13 regression seeds merged, attestation-only, and orphan
attestation workspaces; the 13-to-12 downgrade fixture recreates
workspace_attestations and installed_plugin_index from the folded data;
the fold-in migration gates the additive workspace_dir column for
pre-additive rows; the workspace merge now triggers on the setup table's
own shape so stable-era databases without an attestations table still
reshape; the consolidation applied-message covers the batch.

* refactor(state): fold shared auth profile singletons into config_machine_state

The shared-state auth_profile_stores/auth_profile_state rows (fixed key
'shared') become authProfiles.store/authProfiles.state machine-state
values; the agent-DB tables of the same names are untouched. Git-backup
redaction moves from table-drop to the authProfiles. secret prefix with
seeded-secret absence proof; migration receipts keep the historical
table-name discriminators; the shared-auth relocation and receipt
verification project the KV cells back to the receipt-era row shapes so
persisted digests stay byte-compatible. mcp_oauth_stores stays a table —
its multi-key fold is a named follow-up.

* test(state): finish shared-auth fold coverage and annotate boundary casts

Auth seeders and assertions across the e2e/scripts/secrets suites target
the authProfiles machine-state cells; the v12-to-v13 regression proves
payload-byte fidelity, non-shared-row drop, and insert-if-absent
precedence; the downgrade fixture recreates and repopulates both v12
tables. Boundary type assertions in the plugin-index store carry SAFETY
invariants per the ratchet.

* chore: shrink assertion-safety baseline for plugin-index store

* refactor(doctor): delete the dead onboarding-recommendations migration

Its input — the unscoped 'primary' onboarding row — existed only between
9a93a52a8a and 473962b7de, a two-day beta window; no shipped stable
can produce it and the runtime table folded away at v12. The audit
backup list keeps recognizing system-agent.jsonl artifacts because beta
installs that ran that import may still carry its backups.

* docs: sync the 13-to-12 downgrade example with the executable fixture

* style: format the synced downgrade example

* style: drop unused import and duplicate union constituent

* fix(state): keep orphan attestations across the v13 workspace merge

The merged workspace_setup_state required a workspace path, but legacy
orphan hashed-key attestations never recorded one. workspace_path is now
nullable (setup rows still enforce it via CHECK), the v13 migration and
the doctor file import keep orphans with a NULL path that heals on the
next live access, and the 13-to-12 downgrade keeps attestation-owned
hashes. Doctor test seeds move to the folded KV row.

* perf(state): retire unused cron indexes

* fix(state): preserve v13 migration recovery

* fix(state): preserve v12 lazy-table upgrade

* docs(state): document v13 auth relocation

---------

Co-authored-by: Vincent Koc <vincentkoc@ieee.org>

scripts/bench-agent-concurrency-worker.ts

@@ -265,7 +265,15 @@ async function readDurableRows() {
     path: database.path,
     subagentRows: executeSqliteQuerySync(
       database.db,
-      db.selectFrom("subagent_runs").select(["run_id", "ended_at"]).orderBy("run_id"),
+      db
+        .selectFrom("subagent_runs")
+        .select((eb) => [
+          "run_id",
+          eb
+            .fn<number | null>("json_extract", ["payload_json", eb.val("$.execution.endedAt")])
+            .as("ended_at"),
+        ])
+        .orderBy("run_id"),
     ).rows,
     taskRows: executeSqliteQuerySync(
       database.db,

scripts/bench-sqlite-state.ts

@@ -195,51 +195,67 @@ function seedStateDatabase(db: DatabaseSync, config: ProfileConfig): void {
 function seedCronJobs(db: DatabaseSync, count: number): void {
   const insert = db.prepare(`
     INSERT INTO cron_jobs (
-      store_key, job_id, name, description, enabled, delete_after_run, created_at_ms,
-      agent_id, session_key, schedule_kind, schedule_expr, schedule_tz, every_ms,
-      anchor_ms, at, stagger_ms, session_target, wake_mode, payload_kind,
-      payload_message, payload_model, payload_fallbacks_json, payload_thinking,
-      payload_timeout_seconds, payload_allow_unsafe_external_content,
-      payload_external_content_source_json, payload_light_context, payload_tools_allow_json,
-      delivery_mode, delivery_channel, delivery_to, delivery_thread_id, delivery_account_id,
-      delivery_best_effort, delivery_completion_mode, delivery_completion_to,
-      failure_delivery_mode, failure_delivery_channel, failure_delivery_to,
-      failure_delivery_account_id, failure_alert_disabled, failure_alert_after,
-      failure_alert_channel, failure_alert_to, failure_alert_cooldown_ms,
-      failure_alert_include_skipped, failure_alert_mode, failure_alert_account_id,
-      next_run_at_ms, running_at_ms, last_run_at_ms, last_run_status, last_error,
-      last_duration_ms, consecutive_errors, consecutive_skipped, schedule_error_count,
-      last_delivery_status, last_delivery_error, last_delivered, last_failure_alert_at_ms,
+      store_key, job_id, name, enabled, agent_id, payload_kind,
       job_json, state_json, runtime_updated_at_ms, schedule_identity, sort_order, updated_at
-    ) VALUES (
-      ?, ?, ?, NULL, ?, NULL, ?, ?, ?, 'every', NULL, NULL, ?, ?, NULL, NULL,
-      'isolated', 'now', 'agentTurn', ?, 'openai/gpt-5.6-luna', NULL, NULL, 60,
-      0, NULL, 1, NULL, 'announce', 'telegram', ?, NULL, 'bench-account',
-      1, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL,
-      NULL, NULL, NULL, ?, NULL, ?, 'completed', NULL, ?, 0, 0, 0, 'sent',
-      NULL, 1, NULL, ?, '{}', ?, ?, ?, ?
-    )
+    ) VALUES (?, ?, ?, ?, ?, 'agentTurn', ?, ?, ?, ?, ?, ?)
   `);
   for (let i = 0; i < count; i += 1) {
     const jobId = `job-${String(i).padStart(8, "0")}`;
     const storeKey = `/state/cron/jobs-${i % 8}.json`;
     const updatedAt = 1_700_000_000_000 + i;
+    const name = `Benchmark job ${i}`;
+    const enabled = i % 5 !== 0;
+    const agentId = `agent-${i % 16}`;
+    const job = {
+      id: jobId,
+      name,
+      enabled,
+      createdAtMs: updatedAt - 100_000,
+      agentId,
+      sessionKey: `agent:${agentId}:main`,
+      schedule: {
+        kind: "every",
+        everyMs: 60_000 + (i % 120) * 1_000,
+        anchorMs: updatedAt - 60_000,
+      },
+      sessionTarget: "isolated",
+      wakeMode: "now",
+      payload: {
+        kind: "agentTurn",
+        message: `Benchmark payload ${i}`,
+        model: "openai/gpt-5.6-luna",
+        timeoutSeconds: 60,
+        allowUnsafeExternalContent: false,
+        lightContext: true,
+      },
+      delivery: {
+        mode: "announce",
+        channel: "telegram",
+        to: `chat-${i % 32}`,
+        accountId: "bench-account",
+        bestEffort: true,
+      },
+      state: {},
+    };
+    const state = {
+      nextRunAtMs: updatedAt + (i % 2_000) * 1_000,
+      lastRunAtMs: updatedAt - 1_000,
+      lastRunStatus: "completed",
+      lastDurationMs: 50 + (i % 500),
+      consecutiveErrors: 0,
+      consecutiveSkipped: 0,
+      scheduleErrorCount: 0,
+      lastDeliveryStatus: "sent",
+      lastDelivered: true,
+    };
     insert.run(
       storeKey,
       jobId,
-      `Benchmark job ${i}`,
-      i % 5 === 0 ? 0 : 1,
-      updatedAt - 100_000,
-      `agent-${i % 16}`,
-      `agent:agent-${i % 16}:main`,
-      60_000 + (i % 120) * 1_000,
-      updatedAt - 60_000,
-      `Benchmark payload ${i}`,
-      `chat-${i % 32}`,
-      updatedAt + (i % 2_000) * 1_000,
-      updatedAt - 1_000,
-      50 + (i % 500),
-      JSON.stringify({ id: jobId, seed: i }),
+      name,
+      enabled ? 1 : 0,
+      agentId,
+      JSON.stringify(job),
+      JSON.stringify(state),
       updatedAt,
       `schedule-${i % 512}`,
       i,
@@ -491,21 +507,10 @@ function runHotQueries(params: {
   return [
     runTimedQuery(
       params.stateDb,
-      `SELECT job_id, name, updated_at
+      `SELECT *
          FROM cron_jobs
         WHERE store_key = ?
-        ORDER BY sort_order ASC, updated_at ASC, job_id
-        LIMIT 50`,
-      ["/state/cron/jobs-0.json"],
-      params.config.queryRuns,
-    ),
-    runTimedQuery(
-      params.stateDb,
-      `SELECT job_id, next_run_at_ms
-         FROM cron_jobs
-        WHERE store_key = ? AND enabled = 1 AND next_run_at_ms IS NOT NULL
-        ORDER BY next_run_at_ms ASC, job_id
-        LIMIT 50`,
+        ORDER BY sort_order ASC, updated_at ASC, job_id ASC`,
       ["/state/cron/jobs-0.json"],
       params.config.queryRuns,
     ),

scripts/check-kysely-guardrails.mts

@@ -67,6 +67,7 @@ const rawSqliteAllowPathGroups = {
     "src/state/openclaw-state-db-schema-helpers.ts",
     "src/state/openclaw-state-db-schema-repair.ts",
     "src/state/openclaw-state-db-schema-v12-foldin.ts",
+    "src/state/openclaw-state-db-schema-v13-widerow.ts",
     "src/state/openclaw-state-db-startup-checkpoint.ts",
     "src/state/openclaw-state-db-table-retirements.ts",
     "src/state/openclaw-state-db-fast-path.ts",

scripts/e2e/lib/auth-profile-store-assertions.mjs

@@ -14,17 +14,17 @@ export function readSharedAuthProfileStoreText(stateDir) {
     db = new DatabaseSync(dbPath, { readOnly: true });
     const schema = db
       .prepare("SELECT type FROM sqlite_schema WHERE name = ? LIMIT 1")
-      .get("auth_profile_stores");
+      .get("config_machine_state");
     if (!schema) {
       return "";
     }
     if (schema.type !== "table") {
-      throw new Error(`auth_profile_stores is ${String(schema.type)}, not a table`);
+      throw new Error(`config_machine_state is ${String(schema.type)}, not a table`);
     }
     const row = db
-      .prepare("SELECT store_json FROM auth_profile_stores WHERE store_key = ?")
-      .get("shared");
-    return typeof row?.store_json === "string" ? row.store_json : "";
+      .prepare("SELECT value_json FROM config_machine_state WHERE state_key = ?")
+      .get("authProfiles.store");
+    return typeof row?.value_json === "string" ? row.value_json : "";
   } catch (error) {
     const detail = error instanceof Error ? error.message : String(error);
     throw new Error(`could not read the shared auth profile store: ${detail}`, {

scripts/e2e/lib/plugin-index-sqlite.mjs

@@ -5,7 +5,7 @@ import { DatabaseSync } from "node:sqlite";
 import { readPositiveIntEnv } from "./env-limits.mjs";
 import { readTextFileBounded } from "./text-file-utils.mjs";
 
-const INDEX_KEY = "installed-plugin-index";
+const STATE_KEY = "plugins.installedIndex";
 const ERROR_DETAIL_TAIL_BYTES = 16 * 1024;
 const JSON_ARTIFACT_MAX_BYTES = readPositiveIntEnv(
   "OPENCLAW_PLUGIN_INDEX_JSON_MAX_BYTES",
@@ -80,53 +80,24 @@ function readSqlitePluginIndex(root = stateDir()) {
     const lengths = db
       .prepare(
         `
-          SELECT octet_length(install_records_json) AS install_records_json_bytes,
-                 octet_length(plugins_json) AS plugins_json_bytes,
-                 octet_length(diagnostics_json) AS diagnostics_json_bytes
-            FROM installed_plugin_index
-           WHERE index_key = ?
+          SELECT octet_length(value_json) AS value_json_bytes
+            FROM config_machine_state
+           WHERE state_key = ?
         `,
       )
-      .get(INDEX_KEY);
+      .get(STATE_KEY);
     if (!lengths) {
       return {};
     }
-    assertIndexJsonByteLength(
-      lengths.install_records_json_bytes,
-      "plugin index install_records_json",
-    );
-    assertIndexJsonByteLength(lengths.plugins_json_bytes, "plugin index plugins_json");
-    assertIndexJsonByteLength(lengths.diagnostics_json_bytes, "plugin index diagnostics_json");
+    assertIndexJsonByteLength(lengths.value_json_bytes, "plugin index value_json");
     const row = db
-      .prepare(
-        `
-          SELECT version, warning, host_contract_version, compat_registry_version,
-                 migration_version, policy_hash, generated_at_ms, refresh_reason,
-                 install_records_json, plugins_json, diagnostics_json
-            FROM installed_plugin_index
-           WHERE index_key = ?
-        `,
-      )
-      .get(INDEX_KEY);
+      .prepare("SELECT value_json FROM config_machine_state WHERE state_key = ?")
+      .get(STATE_KEY);
     if (!row) {
       return {};
     }
-    return {
-      version: Number(row.version),
-      ...(row.warning ? { warning: row.warning } : {}),
-      hostContractVersion: row.host_contract_version,
-      compatRegistryVersion: row.compat_registry_version,
-      migrationVersion: Number(row.migration_version),
-      policyHash: row.policy_hash,
-      generatedAtMs: Number(row.generated_at_ms),
-      ...(row.refresh_reason ? { refreshReason: row.refresh_reason } : {}),
-      installRecords: parseIndexJsonText(
-        row.install_records_json,
-        "plugin index install_records_json",
-      ),
-      plugins: parseIndexJsonText(row.plugins_json, "plugin index plugins_json"),
-      diagnostics: parseIndexJsonText(row.diagnostics_json, "plugin index diagnostics_json"),
-    };
+    const value = parseIndexJsonText(row.value_json, "plugin index value_json");
+    return value?.index && typeof value.index === "object" ? value.index : {};
   } catch (error) {
     if (error?.code === "ETOOBIG") {
       throw error;
@@ -168,59 +139,40 @@ export function writePluginInstallIndexForE2E(index, options = {}) {
   const db = new DatabaseSync(dbPath);
   try {
     db.exec(`
-      CREATE TABLE IF NOT EXISTS installed_plugin_index (
-        index_key TEXT NOT NULL PRIMARY KEY,
-        version INTEGER NOT NULL,
-        host_contract_version TEXT NOT NULL,
-        compat_registry_version TEXT NOT NULL,
-        migration_version INTEGER NOT NULL,
-        policy_hash TEXT NOT NULL,
-        generated_at_ms INTEGER NOT NULL,
-        refresh_reason TEXT,
-        install_records_json TEXT NOT NULL,
-        plugins_json TEXT NOT NULL,
-        diagnostics_json TEXT NOT NULL,
-        warning TEXT,
+      CREATE TABLE IF NOT EXISTS config_machine_state (
+        state_key TEXT NOT NULL PRIMARY KEY,
+        value_json TEXT NOT NULL,
         updated_at_ms INTEGER NOT NULL
       );
     `);
     const now = Date.now();
+    const persisted = {
+      revision: now,
+      index: {
+        version: index.version ?? 1,
+        warning:
+          index.warning ??
+          "DO NOT EDIT. This row is generated by OpenClaw plugin registry commands.",
+        hostContractVersion: index.hostContractVersion ?? "docker-e2e",
+        compatRegistryVersion: index.compatRegistryVersion ?? "docker-e2e",
+        migrationVersion: index.migrationVersion ?? 1,
+        policyHash: index.policyHash ?? "docker-e2e",
+        generatedAtMs: index.generatedAtMs ?? now,
+        ...(index.refreshReason ? { refreshReason: index.refreshReason } : {}),
+        installRecords: index.installRecords ?? {},
+        plugins: index.plugins ?? [],
+        diagnostics: index.diagnostics ?? [],
+      },
+    };
     db.prepare(
       `
-        INSERT INTO installed_plugin_index (
-          index_key, version, host_contract_version, compat_registry_version,
-          migration_version, policy_hash, generated_at_ms, refresh_reason,
-          install_records_json, plugins_json, diagnostics_json, warning, updated_at_ms
-        ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
-        ON CONFLICT(index_key) DO UPDATE SET
-          version = excluded.version,
-          host_contract_version = excluded.host_contract_version,
-          compat_registry_version = excluded.compat_registry_version,
-          migration_version = excluded.migration_version,
-          policy_hash = excluded.policy_hash,
-          generated_at_ms = excluded.generated_at_ms,
-          refresh_reason = excluded.refresh_reason,
-          install_records_json = excluded.install_records_json,
-          plugins_json = excluded.plugins_json,
-          diagnostics_json = excluded.diagnostics_json,
-          warning = excluded.warning,
+        INSERT INTO config_machine_state (state_key, value_json, updated_at_ms)
+        VALUES (?, ?, ?)
+        ON CONFLICT(state_key) DO UPDATE SET
+          value_json = excluded.value_json,
           updated_at_ms = excluded.updated_at_ms
       `,
-    ).run(
-      INDEX_KEY,
-      index.version ?? 1,
-      index.hostContractVersion ?? "docker-e2e",
-      index.compatRegistryVersion ?? "docker-e2e",
-      index.migrationVersion ?? 1,
-      index.policyHash ?? "docker-e2e",
-      index.generatedAtMs ?? now,
-      index.refreshReason ?? null,
-      JSON.stringify(index.installRecords ?? {}),
-      JSON.stringify(index.plugins ?? []),
-      JSON.stringify(index.diagnostics ?? []),
-      index.warning ?? "DO NOT EDIT. This row is generated by OpenClaw plugin registry commands.",
-      now,
-    );
+    ).run(STATE_KEY, JSON.stringify(persisted), now);
   } finally {
     db.close();
   }

scripts/e2e/lib/upgrade-survivor/sqlite-volume.mjs

@@ -427,9 +427,8 @@ export function assertUpgradeVolumeMigrated(stateDir, stage) {
   assertHealthySqlite(stateDatabasePath, (db) => {
     const rows = db
       .prepare(
-        `SELECT job_id, job_json, state_json, enabled, schedule_kind, every_ms, anchor_ms,
-                payload_kind, payload_message, delivery_mode, next_run_at_ms, running_at_ms,
-                last_run_status, last_error, updated_at, runtime_updated_at_ms
+        `SELECT job_id, job_json, state_json, enabled, payload_kind, updated_at,
+                runtime_updated_at_ms
          FROM cron_jobs
          WHERE job_id LIKE 'volume-cron-%'`,
       )
@@ -468,26 +467,23 @@ export function assertUpgradeVolumeMigrated(stateDir, stage) {
         row?.enabled === (expected.enabled ? 1 : 0),
         `volume cron enabled column changed: ${index}`,
       );
-      assert(row?.schedule_kind === "every", `volume cron schedule column changed: ${index}`);
-      assert(row?.every_ms === expected.schedule.everyMs, `volume cron interval changed: ${index}`);
-      assert(row?.anchor_ms === expected.schedule.anchorMs, `volume cron anchor changed: ${index}`);
       assert(row?.payload_kind === "agentTurn", `volume cron payload kind changed: ${index}`);
       assert(
-        row?.payload_message === expected.payload.message,
-        `volume cron payload changed: ${index}`,
-      );
-      assert(row?.delivery_mode === "none", `volume cron delivery mode changed: ${index}`);
-      assert(
-        row?.next_run_at_ms === (expected.enabled ? expected.state.nextRunAtMs : null),
+        (actualState?.nextRunAtMs ?? null) ===
+          (expected.enabled ? expected.state.nextRunAtMs : null),
         `volume cron next-run state changed: ${index}`,
       );
-      assert(row?.running_at_ms === null, `volume cron running state changed: ${index}`);
       assert(
-        row?.last_run_status === (expected.state.lastStatus ?? null),
+        (actualState?.runningAtMs ?? null) === null,
+        `volume cron running state changed: ${index}`,
+      );
+      assert(
+        (actualState?.lastRunStatus ?? actualState?.lastStatus ?? null) ===
+          (expected.state.lastStatus ?? null),
         `volume cron status state changed: ${index}`,
       );
       assert(
-        row?.last_error === (expected.state.lastError ?? null),
+        (actualState?.lastError ?? null) === (expected.state.lastError ?? null),
         `volume cron error state changed: ${index}`,
       );
     }

scripts/lib/live-docker-stage.sh

@@ -135,7 +135,7 @@ try {
   db = new DatabaseSync(dbPath);
   try {
     db.exec("PRAGMA secure_delete = ON;");
-    db.prepare("DELETE FROM installed_plugin_index WHERE index_key = ?").run("installed-plugin-index");
+    db.prepare("DELETE FROM config_machine_state WHERE state_key = ?").run("plugins.installedIndex");
     db.exec("PRAGMA wal_checkpoint(TRUNCATE);");
     db.exec("VACUUM;");
   } catch (err) {
@@ -158,7 +158,7 @@ openclaw_live_stage_state_dir() {
     # Sandbox workspaces can accumulate root-owned artifacts from prior Docker
     # runs. Persisted plugin registry state contains host-absolute paths that
     # are not portable into Linux containers. Live-test auth/config staging does
-    # not need the old JSON source or the SQLite installed_plugin_index row.
+    # not need the old JSON source or the SQLite plugins.installedIndex machine-state row.
     set +e
     tar -C "$source_dir" \
       --warning=no-file-changed \