feat(doctor): relocate shared auth store into state DB (#123349)

* feat(doctor): relocate shared auth store into state DB

* fix(doctor): make shared auth relocation crash-safe

* fix(doctor): skip disabled shared auth inspection

* fix(doctor): break shared auth migration cycle

* fix(doctor): remove unused migration type export

* fix(test): remove duplicate Codex attempt shard
This commit is contained in:
Peter Steinberger
2026-08-13 17:44:02 -07:00
committed by GitHub
parent f0076b0ca6
commit d2dad76ecd
44 changed files with 2622 additions and 800 deletions
+2 -2
View File
@@ -49,7 +49,7 @@ Token credentials (`type: "token"`) support inline `token` and/or `tokenRef`.
## Agent copy portability
Agent auth inheritance is read-through. When an agent has no local profile, it resolves profiles from the default/main agent store at runtime without copying secret material into its own credential store (`agents/<agentId>/agent/openclaw-agent.sqlite`).
Agent auth inheritance is read-through. When an agent has no local profile, it resolves profiles from the shared auth store at runtime without copying secret material into its own credential store (`agents/<agentId>/agent/openclaw-agent.sqlite`). The shared store lives in `state/openclaw.sqlite` after `openclaw doctor --fix` performs the one-time relocation. Until then, doctor reports the legacy `agents/main/agent/openclaw-agent.sqlite` owner and leaves that agent undeletable.
Explicit copy flows, such as `openclaw agents add`, use this portability policy:
@@ -57,7 +57,7 @@ Explicit copy flows, such as `openclaw agents add`, use this portability policy:
- `oauth` profiles are not portable by default because refresh tokens can be single-use or rotation-sensitive.
- Provider-owned OAuth flows may opt in with `copyToAgents: true` only when copying refresh material across agents is known safe; the opt-in only applies when the profile carries inline access/refresh material.
Non-portable profiles remain available through read-through inheritance unless the target agent signs in separately and creates its own local profile.
Non-portable profiles remain available through the shared read-through base unless the target agent signs in separately and creates its own local profile.
## Config-only auth routes
+3 -2
View File
@@ -44,7 +44,7 @@ Options: `--workspace <dir>`, `--model <id>`, `--agent-dir <dir>`, `--bind <chan
- Passing any explicit add flag switches the command into the non-interactive path.
- Non-interactive mode requires both an agent name and `--workspace`.
- `main` is reserved and cannot be used as the new agent id.
- Interactive mode seeds auth by copying only portable static credentials (`api_key` and static `token` profiles) unless a credential opts out with `copyToAgents: false`; OAuth refresh-token profiles are not copied unless a provider opts in with `copyToAgents: true`. Without a copy, OAuth stays available only through read-through inheritance from the real `main` agent store. If the configured default agent is not `main`, sign in separately for OAuth profiles on the new agent.
- Interactive mode seeds auth by copying only portable static credentials (`api_key` and static `token` profiles) unless a credential opts out with `copyToAgents: false`; OAuth refresh-token profiles are not copied unless a provider opts in with `copyToAgents: true`. Without a copy, OAuth stays available through the shared auth base. If the configured default agent has its own local OAuth profile, sign in separately for the new agent.
### `agents bindings`
@@ -66,9 +66,10 @@ Options: `--agent <id>`, `--workspace <dir>`, `--identity-file <path>`, `--from-
Options: `--force`, `--json`.
- `main` cannot be deleted.
- The only configured agent cannot be deleted.
- Without `--force`, interactive confirmation is required (fails in a non-TTY session; re-run with `--force`).
- Workspace, agent state, and session transcript directories move to Trash, not hard-deleted. If Trash is unavailable, agent config deletion still succeeds and reports paths requiring manual cleanup.
- On installations that have not migrated shared auth yet, the legacy owner cannot be deleted. Run `openclaw doctor --fix`; after relocation into shared state SQLite, `main` follows the same deletion rules as any other agent.
- When the Gateway is reachable, deletion routes through the Gateway so config and session-store cleanup share the same writer as runtime traffic. If the Gateway is unreachable, the CLI falls back to the offline local path.
- If another agent's workspace is the same path, inside this workspace, or contains this workspace, the workspace is retained, and `--json` reports `workspaceRetained`, `workspaceRetainedReason`, and `workspaceSharedWith`.
+2
View File
@@ -211,6 +211,8 @@ the container normally.
`openclaw doctor --fix` is the only owner for persistent file-to-SQLite migrations. It validates and claims each recognized source, writes and verifies canonical rows, records a migration receipt, then removes the retired source. Runtime code does not perform lazy imports or fallback reads.
Doctor also reports when shared auth still uses the legacy `agents/main/agent/openclaw-agent.sqlite` owner. `openclaw doctor --fix` copies its auth profile and runtime-state rows into `state/openclaw.sqlite`, verifies the exact payloads, removes the source rows, and records the new ownership only after the transaction succeeds. Auth resolution has no dual-read fallback: before migration the legacy database is complete; after migration the shared state database is complete. Once relocated, deleting `main` no longer risks fleet credentials.
For the retired QMD memory backend, including config rewrites and derived
workspace cleanup, see [Migrating from QMD](/concepts/memory-builtin#migrating-from-qmd).
+3 -3
View File
@@ -517,10 +517,10 @@ children:
Sub-agent auth is resolved by **agent id**, not by session type:
- The sub-agent session key is `agent:<agentId>:subagent:<uuid>`.
- The auth store is loaded from that agent's `agentDir`.
- The main agent's auth profiles are merged in as a **fallback**; agent profiles override main profiles on conflicts.
- The local auth overlay is loaded from that agent's `agentDir`.
- The shared auth profiles are merged in as a **fallback**; agent profiles override shared profiles on conflicts.
The merge is additive, so main profiles are always available as
The merge is additive, so shared profiles are always available as
fallbacks. Fully isolated auth per agent is not supported yet.
## Announce
@@ -11,6 +11,7 @@ import { DatabaseSync } from "node:sqlite";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import * as kyselySync from "../infra/kysely-sync.js";
import * as nodeSqlite from "../infra/node-sqlite.js";
import { writeConfigMachineState } from "../state/config-machine-state.js";
import {
closeOpenClawAgentDatabasesForTest,
OPENCLAW_AGENT_SCHEMA_VERSION,
@@ -125,6 +126,33 @@ describe("auth profile sqlite store", () => {
});
});
it("persists the relocated shared store through the shared-state adapter", async () => {
await withAgentDirEnv("openclaw-auth-shared-state-", () => {
writeConfigMachineState("auth.sharedStore", { location: "state-db" });
saveAuthProfileStore({
...apiKeyStore("sk-shared"),
order: { openai: ["openai:default"] },
});
expect(ensureAuthProfileStore(undefined, { syncExternalCli: false })).toMatchObject({
profiles: { "openai:default": { key: "sk-shared" } },
order: { openai: ["openai:default"] },
});
const database = new DatabaseSync(resolveOpenClawStateSqlitePath());
expect(
database
.prepare("SELECT store_key FROM auth_profile_stores WHERE store_key = 'shared'")
.get(),
).toEqual({ store_key: "shared" });
expect(
database
.prepare("SELECT store_key FROM auth_profile_state WHERE store_key = 'shared'")
.get(),
).toEqual({ store_key: "shared" });
database.close();
});
});
it("does not read legacy auth-profiles.json at runtime", async () => {
await withAgentDirEnv("openclaw-auth-no-json-fallback-", (agentDir) => {
fs.writeFileSync(
@@ -4,6 +4,7 @@ import { resolveOAuthDir } from "../../config/paths.js";
import { createSubsystemLogger } from "../../logging/subsystem.js";
import { shortenHomePath } from "../../utils.js";
import { resolveSharedAuthStorePath } from "./path-resolve.js";
import { resolveSharedMainAuthAgentDir } from "./shared-main-dir.js";
import { resolveAuthProfileDatabasePath } from "./sqlite.js";
const AUTH_PROFILE_MIGRATION_REQUIRED_CODE = "AUTH_PROFILE_MIGRATION_REQUIRED" as const;
@@ -29,8 +30,13 @@ function resolveAuthProfileOwnerPath(agentDir?: string): string {
return agentDir ? resolveAuthProfileDatabasePath(agentDir) : resolveSharedAuthStorePath();
}
function resolveAgentDir(agentDir?: string): string {
return path.dirname(resolveAuthProfileOwnerPath(agentDir));
function resolveLegacySourceAgentDir(
agentDir: string | undefined,
env: NodeJS.ProcessEnv = process.env,
): string {
return agentDir
? path.dirname(resolveAuthProfileOwnerPath(agentDir))
: resolveSharedMainAuthAgentDir(env);
}
/** Detects retired auth files by name only; runtime code must never read their contents. */
@@ -38,13 +44,13 @@ export function listLegacyAuthProfileSources(params: {
agentDir?: string;
env?: NodeJS.ProcessEnv;
}): LegacyAuthProfileSource[] {
const agentDir = resolveAgentDir(params.agentDir);
const agentDir = resolveLegacySourceAgentDir(params.agentDir, params.env);
const candidates: LegacyAuthProfileSource[] = [
{ kind: "auth-profiles", path: path.join(agentDir, "auth-profiles.json") },
{ kind: "auth-state", path: path.join(agentDir, "auth-state.json") },
{ kind: "legacy-auth", path: path.join(agentDir, "auth.json") },
];
const sharedMainDir = path.dirname(resolveSharedAuthStorePath(params.env));
const sharedMainDir = resolveSharedMainAuthAgentDir(params.env);
if (path.resolve(agentDir) === path.resolve(sharedMainDir)) {
candidates.push({ kind: "legacy-oauth", path: resolveLegacyOAuthPath(params.env) });
}
@@ -98,7 +104,7 @@ function listStartupLegacyAuthProfileSources(params: {
sources: LegacyAuthProfileSource[];
credentialSources: LegacyAuthProfileSource[];
}> {
const sharedMainDir = path.dirname(resolveSharedAuthStorePath(params.env));
const sharedMainDir = resolveSharedMainAuthAgentDir(params.env);
return [...new Set([...params.agentDirs, sharedMainDir])].map((agentDir) => {
const sources = listLegacyAuthProfileSources({ agentDir, env: params.env });
return { agentDir, sources, credentialSources: sources.filter(isCredentialSource) };
@@ -3,6 +3,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { useAutoCleanupTempDirTracker } from "../../../test/helpers/temp-dir.js";
import { writeConfigMachineState } from "../../state/config-machine-state.js";
import { closeOpenClawStateDatabaseForTest } from "../../state/openclaw-state-db.js";
import { resolveOpenClawStateSqlitePath } from "../../state/openclaw-state-db.paths.js";
const tempDirs = useAutoCleanupTempDirTracker(afterEach);
@@ -22,12 +23,10 @@ describe("shared auth store path resolution", () => {
it("keeps the absent ownership record pinned to the shipped legacy-main path", async () => {
const env = makeStateEnv();
const { resolveSharedAuthStoreDir, resolveSharedAuthStorePath } =
await import("./path-resolve.js");
const { resolveSharedAuthStorePath } = await import("./path-resolve.js");
const { resolveSharedMainAuthAgentDir } = await import("./shared-main-dir.js");
const legacyDir = resolveSharedMainAuthAgentDir(env);
expect(resolveSharedAuthStoreDir(env)).toBe(legacyDir);
expect(resolveSharedAuthStorePath(env)).toBe(path.join(legacyDir, "openclaw-agent.sqlite"));
writeConfigMachineState("auth.sharedStore", { location: "state-db" }, { env });
@@ -36,37 +35,19 @@ describe("shared auth store path resolution", () => {
OPENCLAW_STATE_DIR: path.join(env.OPENCLAW_STATE_DIR ?? "", "."),
};
expect(resolveSharedAuthStoreDir(aliasEnv)).toBe(legacyDir);
expect(resolveSharedAuthStorePath(aliasEnv)).toBe(
path.join(legacyDir, "openclaw-agent.sqlite"),
);
});
it("fails closed when the ownership record says state-db", async () => {
it("resolves the relocated store to the canonical shared state database", async () => {
const env = makeStateEnv();
writeConfigMachineState("auth.sharedStore", { location: "state-db" }, { env });
const {
resolveSharedAuthStoreDir,
resolveSharedAuthStoreOwnership,
resolveSharedAuthStorePath,
} = await import("./path-resolve.js");
const { resolveSharedAuthStoreOwnership, resolveSharedAuthStorePath } =
await import("./path-resolve.js");
expect(resolveSharedAuthStoreOwnership(env)).toEqual({ location: "state-db" });
for (const resolvePath of [resolveSharedAuthStoreDir, resolveSharedAuthStorePath]) {
try {
resolvePath(env);
throw new Error("expected relocated shared auth resolution to fail");
} catch (error) {
expect(error).toBeInstanceOf(Error);
expect(error).toMatchObject({
name: "SharedAuthStoreRelocatedUnsupportedError",
code: "SHARED_AUTH_STORE_RELOCATED_UNSUPPORTED",
action: "openclaw doctor --fix",
location: "state-db",
message: expect.stringContaining("this build cannot serve it"),
});
}
}
expect(resolveSharedAuthStorePath(env)).toBe(resolveOpenClawStateSqlitePath(env));
});
it("caches ownership independently for each canonical state root", async () => {
@@ -81,7 +62,13 @@ describe("shared auth store path resolution", () => {
{ env: secondEnv },
);
expect(() => resolveSharedAuthStoreOwnership(secondEnv)).toThrow("auth.sharedStore is invalid");
expect(() => resolveSharedAuthStoreOwnership(secondEnv)).toThrow(
expect.objectContaining({
name: "InvalidSharedAuthStoreOwnershipError",
code: "INVALID_SHARED_AUTH_STORE_OWNERSHIP",
action: "openclaw doctor --fix",
}),
);
expect(resolveSharedAuthStoreOwnership(firstEnv)).toEqual({ location: "legacy-main" });
});
});
+17 -15
View File
@@ -10,7 +10,7 @@ import { resolveOpenClawStateSqlitePath } from "../../state/openclaw-state-db.pa
import { resolveUserPath } from "../../utils.js";
import { resolveSharedMainAuthAgentDir } from "./shared-main-dir.js";
const SHARED_AUTH_STORE_STATE_KEY = "auth.sharedStore";
export const SHARED_AUTH_STORE_STATE_KEY = "auth.sharedStore";
const SHARED_AUTH_STORE_OWNERSHIP_CACHE_LIMIT = 256;
export type SharedAuthStoreOwnership = { location: "legacy-main" } | { location: "state-db" };
@@ -19,16 +19,16 @@ export type SharedAuthStoreOwnership = { location: "legacy-main" } | { location:
// Pin each root once so later row changes require an owner-controlled restart.
const sharedAuthStoreOwnershipByDatabasePath = new Map<string, SharedAuthStoreOwnership>();
class SharedAuthStoreRelocatedUnsupportedError extends Error {
readonly code = "SHARED_AUTH_STORE_RELOCATED_UNSUPPORTED" as const;
class InvalidSharedAuthStoreOwnershipError extends Error {
readonly code = "INVALID_SHARED_AUTH_STORE_OWNERSHIP" as const;
readonly action = "openclaw doctor --fix" as const;
readonly location = "state-db" as const;
readonly stateKey = SHARED_AUTH_STORE_STATE_KEY;
constructor() {
constructor(value: unknown) {
super(
"Shared auth store is recorded as relocated, but this build cannot serve it; run openclaw doctor --fix.",
`Config machine state ${SHARED_AUTH_STORE_STATE_KEY} has an invalid shared auth store location (${JSON.stringify(value)}); run openclaw doctor --fix.`,
);
this.name = "SharedAuthStoreRelocatedUnsupportedError";
this.name = "InvalidSharedAuthStoreOwnershipError";
}
}
@@ -43,9 +43,7 @@ function parseSharedAuthStoreOwnership(value: unknown): SharedAuthStoreOwnership
) {
return { location: value.location };
}
throw new Error(
`Config machine state ${SHARED_AUTH_STORE_STATE_KEY} is invalid; run openclaw doctor --fix.`,
);
throw new InvalidSharedAuthStoreOwnershipError(value);
}
/** Resolve the process-stable owner of the shared auth store. */
@@ -69,15 +67,19 @@ export function resolveSharedAuthStoreOwnership(
return ownership;
}
/** Resolve the legacy agent directory containing the shared auth store. */
export function resolveSharedAuthStoreDir(env: NodeJS.ProcessEnv = process.env): string {
return path.dirname(resolveSharedAuthStorePath(env));
/** Update the process-stable cache after this process commits the ownership row. */
export function noteCommittedSharedAuthStoreOwnership(
ownership: SharedAuthStoreOwnership,
env: NodeJS.ProcessEnv = process.env,
): void {
const databasePath = path.resolve(resolveOpenClawStateSqlitePath(env));
sharedAuthStoreOwnershipByDatabasePath.set(databasePath, ownership);
}
/** Resolve the shared auth database path, failing closed for unserved relocation. */
/** Resolve the canonical shared auth database path. */
export function resolveSharedAuthStorePath(env: NodeJS.ProcessEnv = process.env): string {
if (resolveSharedAuthStoreOwnership(env).location === "state-db") {
throw new SharedAuthStoreRelocatedUnsupportedError();
return resolveOpenClawStateSqlitePath(env);
}
return path.join(resolveSharedMainAuthAgentDir(env), "openclaw-agent.sqlite");
}
+25 -3
View File
@@ -8,7 +8,6 @@ import { isRecord } from "@openclaw/normalization-core/record-coerce";
import { readNonBlankString } from "@openclaw/normalization-core/string-coerce";
import { uniqueStrings } from "@openclaw/normalization-core/string-normalization";
import { coerceSecretRef } from "../../config/types.secrets.js";
import type { OpenClawAgentDatabase } from "../../state/openclaw-agent-db.js";
import { asBoolean } from "../../utils/boolean.js";
import { AUTH_STORE_VERSION, authProfilesLog } from "./constants.js";
import { hasUsableOAuthCredential } from "./credential-state.js";
@@ -23,7 +22,12 @@ import {
getRuntimeExternalCliProfileIds,
setRuntimeExternalCliProfileIds,
} from "./runtime-external-profile-references.js";
import { readPersistedAuthProfileStoreRaw } from "./sqlite.js";
import {
readPersistedAuthProfileStoreRaw,
readPersistedSharedAuthProfileStateRaw,
readPersistedSharedAuthProfileStoreRaw,
type AuthProfileDatabase,
} from "./sqlite.js";
import {
coerceAuthProfileState,
loadPersistedAuthProfileState,
@@ -42,7 +46,7 @@ type LegacyAuthStore = Record<string, AuthProfileCredential>;
type LoadPersistedAuthProfileStoreOptions = {
allowKeychainPrompt?: boolean;
database?: OpenClawAgentDatabase;
database?: AuthProfileDatabase;
};
type CredentialRejectReason = "non_object" | "invalid_type" | "missing_provider";
@@ -806,4 +810,22 @@ export function loadPersistedAuthProfileStore(
return merged;
}
/** Load the shared auth store from an explicit state root. */
export function loadPersistedSharedAuthProfileStore(
env: NodeJS.ProcessEnv,
): AuthProfileStore | null {
const raw = readPersistedSharedAuthProfileStoreRaw(env);
const store = coercePersistedAuthProfileStore(raw);
if (!store) {
return null;
}
return {
...store,
...mergeAuthProfileState(
coerceAuthProfileState(raw),
coerceAuthProfileState(readPersistedSharedAuthProfileStateRaw(env)),
),
};
}
/* oxlint-disable max-lines -- TODO: split this grandfathered oversized file. */
+302 -62
View File
@@ -22,31 +22,66 @@ import { readSqliteUserVersion } from "../../infra/sqlite-user-version.js";
import { registerSqliteCacheExitClose } from "../../infra/sqlite-wal.js";
import type { DB as OpenClawAgentKyselyDatabase } from "../../state/openclaw-agent-db.generated.js";
import {
deferOpenClawAgentPostCommitPublication,
OPENCLAW_AGENT_SCHEMA_VERSION,
runOpenClawAgentWriteTransaction,
type OpenClawAgentDatabase,
} from "../../state/openclaw-agent-db.js";
import { OPENCLAW_SQLITE_BUSY_TIMEOUT_MS } from "../../state/openclaw-state-db.js";
import { withExistingOpenClawStateDatabaseReadOnly } from "../../state/openclaw-state-db-readonly.js";
import type { DB as OpenClawStateKyselyDatabase } from "../../state/openclaw-state-db.generated.js";
import {
openOpenClawStateDatabase,
OPENCLAW_SQLITE_BUSY_TIMEOUT_MS,
runOpenClawStateWriteTransaction,
type OpenClawStateDatabase,
} from "../../state/openclaw-state-db.js";
import { resolveUserPath } from "../../utils.js";
import { resolveRegisteredAgentIdForDir } from "../agent-dir-registry.js";
import { resolveSharedAuthStorePath } from "./path-resolve.js";
import { resolveSharedAuthStoreOwnership, resolveSharedAuthStorePath } from "./path-resolve.js";
type AuthProfileDatabase = Pick<
type AgentAuthProfileDatabase = Pick<
OpenClawAgentKyselyDatabase,
"auth_profile_store" | "auth_profile_state"
>;
type SharedAuthProfileDatabase = Pick<
OpenClawStateKyselyDatabase,
"auth_profile_stores" | "auth_profile_state"
>;
export type AuthProfileDatabase = OpenClawAgentDatabase | OpenClawStateDatabase;
type AuthProfileDatabaseTarget =
| { kind: "agent"; agentId: string; path: string; env: NodeJS.ProcessEnv }
| { kind: "shared-state"; path: string; env: NodeJS.ProcessEnv };
// Auth profiles store one JSON blob for secrets and one JSON blob for runtime
// state. SQLite owns durability/transactions; JSON shape owns compatibility.
const PRIMARY_ROW_KEY = "primary";
const SHARED_ROW_KEY = "shared";
const AUTH_PROFILE_READ_HANDLE_CAP = 8;
const authProfileReadDatabases = new Map<string, DatabaseSync>();
const sharedAuthPostCommitPublications = new WeakMap<OpenClawStateDatabase, Array<() => void>>();
let unregisterReadHandleExitClose: (() => void) | null = null;
type AuthProfileReadPoolCloseScope =
| { kind: "database"; databasePath: string }
| { kind: "root"; rootPath: string };
/** Queue runtime publication on the transaction edge owned by this database. */
export function deferAuthProfilePostCommitPublication(
database: AuthProfileDatabase,
publish: () => void,
): boolean {
if ("agentId" in database) {
return deferOpenClawAgentPostCommitPublication(database, publish);
}
const publications = sharedAuthPostCommitPublications.get(database);
if (!publications) {
return false;
}
publications.push(publish);
return true;
}
function inferAgentIdFromDir(agentDir: string): string {
const normalized = path.normalize(agentDir);
if (path.basename(normalized) === "agent") {
@@ -60,19 +95,29 @@ function inferAgentIdFromDir(agentDir: string): string {
// The auth database lives in the agent dir and shares the openclaw-agent schema
// so auth store/state can move with the rest of agent-local durable state.
function resolveAuthProfileDatabaseOptions(agentDir?: string) {
function resolveAuthProfileDatabaseOptions(
agentDir?: string,
env: NodeJS.ProcessEnv = process.env,
): AuthProfileDatabaseTarget {
if (!agentDir) {
const pathname = resolveSharedAuthStorePath();
const pathname = resolveSharedAuthStorePath(env);
if (resolveSharedAuthStoreOwnership(env).location === "state-db") {
return { kind: "shared-state", path: pathname, env };
}
const dir = path.dirname(pathname);
return {
kind: "agent",
agentId: resolveRegisteredAgentIdForDir(dir) ?? inferAgentIdFromDir(dir),
path: pathname,
env,
};
}
const dir = resolveUserPath(agentDir);
return {
kind: "agent",
agentId: resolveRegisteredAgentIdForDir(dir) ?? inferAgentIdFromDir(dir),
path: path.join(dir, "openclaw-agent.sqlite"),
env,
};
}
@@ -83,7 +128,11 @@ export function resolveAuthProfileDatabasePath(agentDir: string): string {
/** Resolves the durable agent owner expected for an auth-profile database. */
export function resolveAuthProfileDatabaseOwnerId(agentDir: string): string {
return resolveAuthProfileDatabaseOptions(agentDir).agentId;
const target = resolveAuthProfileDatabaseOptions(agentDir);
if (target.kind !== "agent") {
throw new Error("agent auth database unexpectedly resolved to shared state");
}
return target.agentId;
}
/** Resolves the SQLite database and sidecar paths used by auth profiles. */
@@ -105,15 +154,36 @@ type PersistedAuthProfileStoreInspection =
| { status: "readable"; raw: unknown }
| { status: "unreadable" };
function getAuthProfileKysely(db: DatabaseSync) {
return getNodeSqliteKysely<AuthProfileDatabase>(db);
function getAgentAuthProfileKysely(db: DatabaseSync) {
return getNodeSqliteKysely<AgentAuthProfileDatabase>(db);
}
function getSharedAuthProfileKysely(db: DatabaseSync) {
return getNodeSqliteKysely<SharedAuthProfileDatabase>(db);
}
function resolveAuthProfileDatabaseKind(
agentDir: string | undefined,
database?: Pick<AuthProfileDatabase, "db">,
): AuthProfileDatabaseTarget["kind"] {
return agentDir !== undefined
? "agent"
: database && !("agentId" in database)
? "shared-state"
: resolveAuthProfileDatabaseOptions(agentDir).kind;
}
function inspectAuthProfileTable(
db: DatabaseSync,
target: "store" | "state",
databaseKind: AuthProfileDatabaseTarget["kind"],
): PersistedAuthProfileStoreInspection | null {
const tableName = target === "store" ? "auth_profile_store" : "auth_profile_state";
const tableName =
target === "store" && databaseKind === "shared-state"
? "auth_profile_stores"
: target === "store"
? "auth_profile_store"
: "auth_profile_state";
const schemaObject = db
.prepare("SELECT type FROM sqlite_master WHERE name = ?")
.get(tableName) as { type?: unknown } | undefined;
@@ -128,17 +198,41 @@ function inspectAuthProfileTable(
function inspectAuthProfileJsonCell(
db: DatabaseSync,
target: "store" | "state",
databaseKind: AuthProfileDatabaseTarget["kind"],
): PersistedAuthProfileStoreInspection {
const tableInspection = inspectAuthProfileTable(db, target);
const tableInspection = inspectAuthProfileTable(db, target, databaseKind);
if (tableInspection) {
return tableInspection;
}
const kysely = getAuthProfileKysely(db);
let raw: string;
if (target === "store") {
if (databaseKind === "shared-state" && target === "store") {
const row = executeSqliteQueryTakeFirstSync(
db,
kysely
getSharedAuthProfileKysely(db)
.selectFrom("auth_profile_stores")
.select("store_json")
.where("store_key", "=", SHARED_ROW_KEY),
);
if (!row) {
return { status: "missing", reason: "row" };
}
raw = row.store_json;
} else if (databaseKind === "shared-state") {
const row = executeSqliteQueryTakeFirstSync(
db,
getSharedAuthProfileKysely(db)
.selectFrom("auth_profile_state")
.select("state_json")
.where("store_key", "=", SHARED_ROW_KEY),
);
if (!row) {
return { status: "missing", reason: "row" };
}
raw = row.state_json;
} else if (target === "store") {
const row = executeSqliteQueryTakeFirstSync(
db,
getAgentAuthProfileKysely(db)
.selectFrom("auth_profile_store")
.select("store_json")
.where("store_key", "=", PRIMARY_ROW_KEY),
@@ -150,7 +244,7 @@ function inspectAuthProfileJsonCell(
} else {
const row = executeSqliteQueryTakeFirstSync(
db,
kysely
getAgentAuthProfileKysely(db)
.selectFrom("auth_profile_state")
.select("state_json")
.where("state_key", "=", PRIMARY_ROW_KEY),
@@ -261,10 +355,24 @@ function acquireAuthProfileReadDatabase(
}
function inspectAuthProfileJsonCellReadOnly(
pathname: string,
databaseTarget: AuthProfileDatabaseTarget,
target: "store" | "state",
): PersistedAuthProfileStoreInspection {
const acquired = acquireAuthProfileReadDatabase(pathname);
if (databaseTarget.kind === "shared-state") {
try {
return (
withExistingOpenClawStateDatabaseReadOnly(
({ db }) => inspectAuthProfileJsonCell(db, target, "shared-state"),
{ env: databaseTarget.env, path: databaseTarget.path },
) ?? { status: "missing", reason: "database" }
);
} catch {
return isMissingDatabasePath(databaseTarget.path)
? { status: "missing", reason: "database" }
: { status: "unreadable" };
}
}
const acquired = acquireAuthProfileReadDatabase(databaseTarget.path);
if (acquired.status === "missing") {
return { status: "missing", reason: "database" };
}
@@ -272,9 +380,9 @@ function inspectAuthProfileJsonCellReadOnly(
return { status: "unreadable" };
}
try {
return inspectAuthProfileJsonCell(acquired.db, target);
return inspectAuthProfileJsonCell(acquired.db, target, "agent");
} catch {
closeAuthProfileReadDatabase(pathname);
closeAuthProfileReadDatabase(databaseTarget.path);
return { status: "unreadable" };
}
}
@@ -282,27 +390,51 @@ function inspectAuthProfileJsonCellReadOnly(
/** Distinguishes an absent auth row from a present store that could not be read. */
export function inspectPersistedAuthProfileStoreRaw(
agentDir?: string,
database?: Pick<OpenClawAgentDatabase, "db">,
database?: Pick<AuthProfileDatabase, "db">,
): PersistedAuthProfileStoreInspection {
const databaseTarget = resolveAuthProfileDatabaseOptions(agentDir);
if (database) {
return inspectAuthProfileJsonCell(database.db, "store");
return inspectAuthProfileJsonCell(
database.db,
"store",
resolveAuthProfileDatabaseKind(agentDir, database),
);
}
return inspectAuthProfileJsonCellReadOnly(
resolveAuthProfileDatabaseOptions(agentDir).path,
"store",
);
return inspectAuthProfileJsonCellReadOnly(databaseTarget, "store");
}
/** Distinguishes an absent auth-state row from state that could not be read. */
export function inspectPersistedAuthProfileStateRaw(
agentDir?: string,
database?: Pick<OpenClawAgentDatabase, "db">,
database?: Pick<AuthProfileDatabase, "db">,
): PersistedAuthProfileStoreInspection {
const databaseTarget = resolveAuthProfileDatabaseOptions(agentDir);
if (database) {
return inspectAuthProfileJsonCell(database.db, "state");
return inspectAuthProfileJsonCell(
database.db,
"state",
resolveAuthProfileDatabaseKind(agentDir, database),
);
}
return inspectAuthProfileJsonCellReadOnly(databaseTarget, "state");
}
/** Inspect the shared store for an explicit state root without projecting it to an agent dir. */
export function inspectPersistedSharedAuthProfileStoreRaw(
env: NodeJS.ProcessEnv,
): PersistedAuthProfileStoreInspection {
return inspectAuthProfileJsonCellReadOnly(
resolveAuthProfileDatabaseOptions(agentDir).path,
resolveAuthProfileDatabaseOptions(undefined, env),
"store",
);
}
/** Inspect shared runtime state for an explicit state root. */
export function inspectPersistedSharedAuthProfileStateRaw(
env: NodeJS.ProcessEnv,
): PersistedAuthProfileStoreInspection {
return inspectAuthProfileJsonCellReadOnly(
resolveAuthProfileDatabaseOptions(undefined, env),
"state",
);
}
@@ -310,46 +442,72 @@ export function inspectPersistedAuthProfileStateRaw(
/** Reads the raw persisted secrets-store payload without coercing the schema. */
export function readPersistedAuthProfileStoreRaw(
agentDir?: string,
database?: OpenClawAgentDatabase,
database?: AuthProfileDatabase,
): unknown {
const databaseTarget = resolveAuthProfileDatabaseOptions(agentDir);
if (database) {
const db = getAuthProfileKysely(database.db);
if (resolveAuthProfileDatabaseKind(agentDir, database) === "shared-state") {
const row = executeSqliteQueryTakeFirstSync(
database.db,
getSharedAuthProfileKysely(database.db)
.selectFrom("auth_profile_stores")
.select("store_json")
.where("store_key", "=", SHARED_ROW_KEY),
);
return parseJsonCell(row?.store_json);
}
const row = executeSqliteQueryTakeFirstSync(
database.db,
db
getAgentAuthProfileKysely(database.db)
.selectFrom("auth_profile_store")
.select("store_json")
.where("store_key", "=", PRIMARY_ROW_KEY),
);
return parseJsonCell(row?.store_json);
}
const result = inspectAuthProfileJsonCellReadOnly(
resolveAuthProfileDatabaseOptions(agentDir).path,
"store",
);
const result = inspectAuthProfileJsonCellReadOnly(databaseTarget, "store");
return result.status === "readable" ? result.raw : null;
}
/** Reads the raw persisted runtime-state payload without coercing the schema. */
export function readPersistedAuthProfileStateRaw(
agentDir?: string,
database?: OpenClawAgentDatabase,
database?: AuthProfileDatabase,
): unknown {
const databaseTarget = resolveAuthProfileDatabaseOptions(agentDir);
if (database) {
const db = getAuthProfileKysely(database.db);
if (resolveAuthProfileDatabaseKind(agentDir, database) === "shared-state") {
const row = executeSqliteQueryTakeFirstSync(
database.db,
getSharedAuthProfileKysely(database.db)
.selectFrom("auth_profile_state")
.select("state_json")
.where("store_key", "=", SHARED_ROW_KEY),
);
return parseJsonCell(row?.state_json);
}
const row = executeSqliteQueryTakeFirstSync(
database.db,
db
getAgentAuthProfileKysely(database.db)
.selectFrom("auth_profile_state")
.select("state_json")
.where("state_key", "=", PRIMARY_ROW_KEY),
);
return parseJsonCell(row?.state_json);
}
const result = inspectAuthProfileJsonCellReadOnly(
resolveAuthProfileDatabaseOptions(agentDir).path,
"state",
);
const result = inspectAuthProfileJsonCellReadOnly(databaseTarget, "state");
return result.status === "readable" ? result.raw : null;
}
/** Read the shared credential row for an explicit state root. */
export function readPersistedSharedAuthProfileStoreRaw(env: NodeJS.ProcessEnv): unknown {
const result = inspectPersistedSharedAuthProfileStoreRaw(env);
return result.status === "readable" ? result.raw : null;
}
/** Read the shared runtime-state row for an explicit state root. */
export function readPersistedSharedAuthProfileStateRaw(env: NodeJS.ProcessEnv): unknown {
const result = inspectPersistedSharedAuthProfileStateRaw(env);
return result.status === "readable" ? result.raw : null;
}
@@ -357,13 +515,32 @@ export function readPersistedAuthProfileStateRaw(
export function writePersistedAuthProfileStoreRaw(
payload: unknown,
agentDir?: string,
database?: OpenClawAgentDatabase,
database?: AuthProfileDatabase,
): void {
const write = (target: OpenClawAgentDatabase) => {
const db = getAuthProfileKysely(target.db);
const databaseKind = resolveAuthProfileDatabaseKind(agentDir, database);
const write = (target: AuthProfileDatabase) => {
if (databaseKind === "shared-state") {
executeSqliteQuerySync(
target.db,
getSharedAuthProfileKysely(target.db)
.insertInto("auth_profile_stores")
.values({
store_key: SHARED_ROW_KEY,
store_json: JSON.stringify(payload),
updated_at: Date.now(),
})
.onConflict((conflict) =>
conflict.column("store_key").doUpdateSet({
store_json: JSON.stringify(payload),
updated_at: Date.now(),
}),
),
);
return;
}
executeSqliteQuerySync(
target.db,
db
getAgentAuthProfileKysely(target.db)
.insertInto("auth_profile_store")
.values({
store_key: PRIMARY_ROW_KEY,
@@ -382,36 +559,70 @@ export function writePersistedAuthProfileStoreRaw(
write(database);
return;
}
runOpenClawAgentWriteTransaction(write, resolveAuthProfileDatabaseOptions(agentDir));
runAuthProfileWriteTransaction(agentDir, write);
}
/** Deletes the persisted secrets-store row while leaving runtime state intact. */
export function deletePersistedAuthProfileStoreRaw(
agentDir?: string,
database?: OpenClawAgentDatabase,
database?: AuthProfileDatabase,
): void {
const remove = (target: OpenClawAgentDatabase) => {
const db = getAuthProfileKysely(target.db);
const databaseKind = resolveAuthProfileDatabaseKind(agentDir, database);
const remove = (target: AuthProfileDatabase) => {
executeSqliteQuerySync(
target.db,
db.deleteFrom("auth_profile_store").where("store_key", "=", PRIMARY_ROW_KEY),
databaseKind === "shared-state"
? getSharedAuthProfileKysely(target.db)
.deleteFrom("auth_profile_stores")
.where("store_key", "=", SHARED_ROW_KEY)
: getAgentAuthProfileKysely(target.db)
.deleteFrom("auth_profile_store")
.where("store_key", "=", PRIMARY_ROW_KEY),
);
};
if (database) {
remove(database);
return;
}
runOpenClawAgentWriteTransaction(remove, resolveAuthProfileDatabaseOptions(agentDir));
runAuthProfileWriteTransaction(agentDir, remove);
}
/** Writes or deletes the persisted runtime-state payload. */
export function writePersistedAuthProfileStateRaw(
payload: unknown,
agentDir?: string,
database?: OpenClawAgentDatabase,
database?: AuthProfileDatabase,
): void {
const write = (target: OpenClawAgentDatabase) => {
const db = getAuthProfileKysely(target.db);
const databaseKind = resolveAuthProfileDatabaseKind(agentDir, database);
const write = (target: AuthProfileDatabase) => {
if (databaseKind === "shared-state") {
const db = getSharedAuthProfileKysely(target.db);
if (!payload) {
executeSqliteQuerySync(
target.db,
db.deleteFrom("auth_profile_state").where("store_key", "=", SHARED_ROW_KEY),
);
return;
}
executeSqliteQuerySync(
target.db,
db
.insertInto("auth_profile_state")
.values({
store_key: SHARED_ROW_KEY,
state_json: JSON.stringify(payload),
updated_at: Date.now(),
})
.onConflict((conflict) =>
conflict.column("store_key").doUpdateSet({
state_json: JSON.stringify(payload),
updated_at: Date.now(),
}),
),
);
return;
}
const db = getAgentAuthProfileKysely(target.db);
if (!payload) {
executeSqliteQuerySync(
target.db,
@@ -440,18 +651,47 @@ export function writePersistedAuthProfileStateRaw(
write(database);
return;
}
runOpenClawAgentWriteTransaction(write, resolveAuthProfileDatabaseOptions(agentDir));
runAuthProfileWriteTransaction(agentDir, write);
}
/** Runs an auth-profile database write transaction for store/state updates. */
export function runAuthProfileWriteTransaction<T>(
agentDir: string | undefined,
operation: (database: OpenClawAgentDatabase) => T,
options: { stateDir?: string } = {},
operation: (database: AuthProfileDatabase) => T,
options: { env?: NodeJS.ProcessEnv; stateDir?: string } = {},
): T {
const databaseOptions = resolveAuthProfileDatabaseOptions(agentDir);
return runOpenClawAgentWriteTransaction(operation, {
...databaseOptions,
...(options.stateDir ? { env: { ...process.env, OPENCLAW_STATE_DIR: options.stateDir } } : {}),
});
const env =
options.env ??
(options.stateDir ? { ...process.env, OPENCLAW_STATE_DIR: options.stateDir } : process.env);
const databaseTarget = resolveAuthProfileDatabaseOptions(agentDir, env);
if (databaseTarget.kind === "agent") {
return runOpenClawAgentWriteTransaction(operation, databaseTarget);
}
const database = openOpenClawStateDatabase({ env, path: databaseTarget.path });
const enteredNestedTransaction = database.db.isTransaction;
const publications: Array<() => void> | undefined = enteredNestedTransaction
? sharedAuthPostCommitPublications.get(database)
: [];
const publicationStart = publications?.length ?? 0;
if (!enteredNestedTransaction && publications) {
sharedAuthPostCommitPublications.set(database, publications);
}
let result: T;
try {
result = runOpenClawStateWriteTransaction(operation, { env, database });
} catch (error) {
publications?.splice(publicationStart);
throw error;
} finally {
if (!enteredNestedTransaction && publications) {
sharedAuthPostCommitPublications.delete(database);
}
}
if (!enteredNestedTransaction) {
for (const publish of publications ?? []) {
publish();
}
}
return result;
}
+2 -3
View File
@@ -8,9 +8,8 @@ import { asFiniteNumber } from "@openclaw/normalization-core/number-coercion";
import { isRecord } from "@openclaw/normalization-core/record-coerce";
import { normalizeOptionalString } from "@openclaw/normalization-core/string-coerce";
import { normalizeTrimmedStringList } from "@openclaw/normalization-core/string-normalization";
import type { OpenClawAgentDatabase } from "../../state/openclaw-agent-db.js";
import { AUTH_STORE_VERSION } from "./constants.js";
import { readPersistedAuthProfileStateRaw } from "./sqlite.js";
import { readPersistedAuthProfileStateRaw, type AuthProfileDatabase } from "./sqlite.js";
import type {
AuthProfileBlockedReason,
AuthProfileBlockedSource,
@@ -187,7 +186,7 @@ export function mergeAuthProfileState(
/** Loads persisted auth profile runtime state from SQLite. */
export function loadPersistedAuthProfileState(
agentDir?: string,
database?: OpenClawAgentDatabase,
database?: AuthProfileDatabase,
): AuthProfileState {
return coerceAuthProfileState(readPersistedAuthProfileStateRaw(agentDir, database));
}
+153 -124
View File
@@ -7,10 +7,6 @@ import { AsyncLocalStorage } from "node:async_hooks";
import { isDeepStrictEqual } from "node:util";
import type { OpenClawConfig } from "../../config/types.openclaw.js";
import { isSecretRef } from "../../config/types.secrets.js";
import {
deferOpenClawAgentPostCommitPublication,
type OpenClawAgentDatabase,
} from "../../state/openclaw-agent-db.js";
import { isRecord } from "../../utils.js";
import { cloneAuthProfileStore } from "./clone.js";
import { AUTH_STORE_VERSION, authProfilesLog } from "./constants.js";
@@ -63,6 +59,7 @@ import {
setRuntimeAuthProfileStoreSnapshotAtDatabasePath,
} from "./runtime-snapshots.js";
import {
deferAuthProfilePostCommitPublication,
deletePersistedAuthProfileStoreRaw,
inspectPersistedAuthProfileStoreRaw,
readPersistedAuthProfileStoreRaw,
@@ -71,6 +68,7 @@ import {
runAuthProfileWriteTransaction,
writePersistedAuthProfileStateRaw,
writePersistedAuthProfileStoreRaw,
type AuthProfileDatabase,
} from "./sqlite.js";
import { buildPersistedAuthProfileState, loadPersistedAuthProfileState } from "./state.js";
import type { AuthProfileStore, RuntimeAuthProfileStore } from "./types.js";
@@ -78,7 +76,7 @@ import type { AuthProfileStore, RuntimeAuthProfileStore } from "./types.js";
type LoadAuthProfileStoreOptions = {
allowKeychainPrompt?: boolean;
config?: OpenClawConfig;
database?: OpenClawAgentDatabase;
database?: AuthProfileDatabase;
externalCli?: ExternalCliAuthDiscovery;
inheritedAuthDir?: string;
readOnly?: boolean;
@@ -238,7 +236,7 @@ if (process.env.VITEST || process.env.NODE_ENV === "test") {
function resolvePersistedLoadOptions(
options: Pick<LoadAuthProfileStoreOptions, "allowKeychainPrompt" | "database"> | undefined,
): { allowKeychainPrompt?: boolean; database?: OpenClawAgentDatabase } {
): { allowKeychainPrompt?: boolean; database?: AuthProfileDatabase } {
return {
...(options?.allowKeychainPrompt !== undefined
? { allowKeychainPrompt: options.allowKeychainPrompt }
@@ -249,7 +247,7 @@ function resolvePersistedLoadOptions(
function loadPersistedAuthProfileStores(
agentDir?: string,
database?: OpenClawAgentDatabase,
database?: AuthProfileDatabase,
): PersistedAuthProfileStores {
const localStore = loadPersistedAuthProfileStore(agentDir, database ? { database } : undefined);
const localAuthPath = agentDir ? resolveAgentAuthPath(agentDir) : resolveSharedAuthPath();
@@ -1311,11 +1309,11 @@ function saveAuthProfileStoreInTransaction(
store: AuthProfileStore,
agentDir: string | undefined,
options: SaveAuthProfileStoreOptions | undefined,
database: OpenClawAgentDatabase,
database: AuthProfileDatabase,
publishFromSuppliedStore = false,
): () => void {
const savedAuthPath = agentDir ? resolveAgentAuthPath(agentDir) : resolveSharedAuthPath();
const mainAuthPath = resolveSharedAuthPath();
const savedAuthPath = agentDir ? resolveAgentAuthPath(agentDir) : database.path;
const mainAuthPath = agentDir ? resolveSharedAuthPath() : database.path;
const savesMainStore = savedAuthPath === mainAuthPath;
const loadedPersistedStores = loadPersistedAuthProfileStores(agentDir, database);
const persistedStores: PersistedAuthProfileStores = {
@@ -1432,7 +1430,7 @@ export function saveAuthProfileStore(
store: AuthProfileStore,
agentDir?: string,
options?: SaveAuthProfileStoreOptions,
database?: OpenClawAgentDatabase,
database?: AuthProfileDatabase,
): void {
const effectiveAgentDir = resolveRuntimeAuthProfileAgentDir(agentDir);
if (database) {
@@ -1446,7 +1444,7 @@ export function saveAuthProfileStore(
const publishAfterCommit = () => {
publishRuntimeSnapshotsAfterCommit(publishRuntimeSnapshots);
};
if (!deferOpenClawAgentPostCommitPublication(database, publishAfterCommit)) {
if (!deferAuthProfilePostCommitPublication(database, publishAfterCommit)) {
// A supplied connection outside the transaction wrapper autocommits each write.
publishAfterCommit();
}
@@ -1497,12 +1495,17 @@ type CommittedAuthProfileStoreSave = {
function captureRuntimeAuthProfileStorePersistenceSnapshot(
agentDir?: string,
canonicalDatabasePath?: string,
): Pick<
AuthProfileStorePersistenceSnapshot,
"runtimeCaptured" | "runtimeRevision" | "runtimeStore" | "derivedRuntimeStores"
> {
const capturedAuthPath = agentDir ? resolveAgentAuthPath(agentDir) : resolveSharedAuthPath();
const mainAuthPath = resolveSharedAuthPath();
const capturedAuthPath =
canonicalDatabasePath ?? (agentDir ? resolveAgentAuthPath(agentDir) : resolveSharedAuthPath());
const mainAuthPath =
agentDir === undefined && canonicalDatabasePath
? canonicalDatabasePath
: resolveSharedAuthPath();
return {
runtimeCaptured: true,
runtimeRevision: getRuntimeAuthProfileStoreSnapshotRevisionAtDatabasePath(capturedAuthPath),
@@ -1613,15 +1616,20 @@ function rebuildRuntimeAuthProfileStoreSnapshot(
/** Capture both persisted auth rows under one database lock. */
export function captureAuthProfileStorePersistenceSnapshot(
agentDir?: string,
options: { stateDir?: string } = {},
): AuthProfileStorePersistenceSnapshot {
const effectiveAgentDir = resolveRuntimeAuthProfileAgentDir(agentDir);
return runAuthProfileWriteTransaction(effectiveAgentDir, (database) => {
return {
credentialsRaw: readPersistedAuthProfileStoreRaw(effectiveAgentDir, database),
stateRaw: readPersistedAuthProfileStateRaw(effectiveAgentDir, database),
...captureRuntimeAuthProfileStorePersistenceSnapshot(effectiveAgentDir),
};
});
return runAuthProfileWriteTransaction(
effectiveAgentDir,
(database) => {
return {
credentialsRaw: readPersistedAuthProfileStoreRaw(effectiveAgentDir, database),
stateRaw: readPersistedAuthProfileStateRaw(effectiveAgentDir, database),
...captureRuntimeAuthProfileStorePersistenceSnapshot(effectiveAgentDir, database.path),
};
},
options,
);
}
/**
@@ -1633,6 +1641,7 @@ export function saveAuthProfileStoreIfPersistenceSnapshotMatches(params: {
snapshot: AuthProfileStorePersistenceSnapshot;
agentDir?: string;
options?: SaveAuthProfileStoreOptions;
stateDir?: string;
}): CommittedAuthProfileStoreSave {
const agentDir = resolveRuntimeAuthProfileAgentDir(params.agentDir);
let publishRuntimeSnapshots: (() => void) | undefined;
@@ -1641,50 +1650,67 @@ export function saveAuthProfileStoreIfPersistenceSnapshotMatches(params: {
stateRaw: null,
runtimeCaptured: false,
};
runAuthProfileWriteTransaction(agentDir, (database) => {
const currentCredentials = readPersistedAuthProfileStoreRaw(agentDir, database);
const currentState = readPersistedAuthProfileStateRaw(agentDir, database);
if (
!isDeepStrictEqual(currentCredentials, params.snapshot.credentialsRaw) ||
!isDeepStrictEqual(currentState, params.snapshot.stateRaw)
) {
throw new Error("auth profile store changed after secrets apply captured it");
}
const runtimeAtSaveEdge = captureRuntimeAuthProfileStorePersistenceSnapshot(agentDir);
owned.runtimeRevisionAtSaveEdge = runtimeAtSaveEdge.runtimeRevision;
owned.derivedRuntimeRevisionsAtSaveEdge = runtimeAtSaveEdge.derivedRuntimeStores?.flatMap(
(entry) =>
typeof entry.runtimeRevision === "number"
? [
{
databasePath: entry.databasePath,
agentDir: entry.agentDir,
runtimeRevision: entry.runtimeRevision,
},
]
: [],
);
publishRuntimeSnapshots = saveAuthProfileStoreInTransaction(
params.store,
agentDir,
params.options,
database,
);
owned.credentialsRaw = readPersistedAuthProfileStoreRaw(agentDir, database);
owned.stateRaw = readPersistedAuthProfileStateRaw(agentDir, database);
});
runAuthProfileWriteTransaction(
agentDir,
(database) => {
const currentCredentials = readPersistedAuthProfileStoreRaw(agentDir, database);
const currentState = readPersistedAuthProfileStateRaw(agentDir, database);
if (
!isDeepStrictEqual(currentCredentials, params.snapshot.credentialsRaw) ||
!isDeepStrictEqual(currentState, params.snapshot.stateRaw)
) {
throw new Error("auth profile store changed after secrets apply captured it");
}
const runtimeAtSaveEdge = captureRuntimeAuthProfileStorePersistenceSnapshot(
agentDir,
database.path,
);
owned.runtimeRevisionAtSaveEdge = runtimeAtSaveEdge.runtimeRevision;
owned.derivedRuntimeRevisionsAtSaveEdge = runtimeAtSaveEdge.derivedRuntimeStores?.flatMap(
(entry) =>
typeof entry.runtimeRevision === "number"
? [
{
databasePath: entry.databasePath,
agentDir: entry.agentDir,
runtimeRevision: entry.runtimeRevision,
},
]
: [],
);
publishRuntimeSnapshots = saveAuthProfileStoreInTransaction(
params.store,
agentDir,
params.options,
database,
);
owned.credentialsRaw = readPersistedAuthProfileStoreRaw(agentDir, database);
owned.stateRaw = readPersistedAuthProfileStateRaw(agentDir, database);
},
params.stateDir ? { stateDir: params.stateDir } : {},
);
return {
owned,
publishRuntimeSnapshots: () =>
publishRuntimeSnapshotsAfterCommit(() => {
recordRuntimeAuthProfileStorePublicationEdge(
owned,
captureRuntimeAuthProfileStorePersistenceSnapshot(agentDir),
captureRuntimeAuthProfileStorePersistenceSnapshot(
agentDir,
params.stateDir && agentDir === undefined
? resolveSharedAuthPath({ ...process.env, OPENCLAW_STATE_DIR: params.stateDir })
: undefined,
),
);
publishRuntimeSnapshots?.();
recordRuntimeAuthProfileStoreOwnership(
owned,
captureRuntimeAuthProfileStorePersistenceSnapshot(params.agentDir),
captureRuntimeAuthProfileStorePersistenceSnapshot(
params.agentDir,
params.stateDir && params.agentDir === undefined
? resolveSharedAuthPath({ ...process.env, OPENCLAW_STATE_DIR: params.stateDir })
: undefined,
),
);
}),
};
@@ -1806,83 +1832,86 @@ export function restoreAuthProfileStorePersistenceSnapshot(
snapshot: AuthProfileStorePersistenceSnapshot,
owned: AuthProfileStorePersistenceSnapshot,
agentDir?: string,
options: { stateDir?: string } = {},
): void {
let credentialsOwned = false;
let stateOwned = false;
let credentialsRestored = false;
let stateRestored = false;
let publishRuntimeSnapshots: (() => void) | undefined;
runAuthProfileWriteTransaction(agentDir, (database) => {
const existingRaw = readPersistedAuthProfileStoreRaw(agentDir, database);
const existingState = readPersistedAuthProfileStateRaw(agentDir, database);
credentialsOwned = isDeepStrictEqual(existingRaw, owned.credentialsRaw);
stateOwned = isDeepStrictEqual(existingState, owned.stateRaw);
const beforeProfiles =
isRecord(existingRaw) && isRecord(existingRaw.profiles) ? existingRaw.profiles : {};
const restoredProfiles =
isRecord(snapshot.credentialsRaw) && isRecord(snapshot.credentialsRaw.profiles)
? snapshot.credentialsRaw.profiles
: {};
const changedProfileIds = [
...new Set([...Object.keys(beforeProfiles), ...Object.keys(restoredProfiles)]),
].filter(
(profileId) => !isDeepStrictEqual(beforeProfiles[profileId], restoredProfiles[profileId]),
);
const profileSetChanged = changedProfileIds.some(
(profileId) =>
Object.hasOwn(beforeProfiles, profileId) !== Object.hasOwn(restoredProfiles, profileId),
);
credentialsRestored =
credentialsOwned && !isDeepStrictEqual(existingRaw, snapshot.credentialsRaw);
stateRestored = stateOwned && !isDeepStrictEqual(existingState, snapshot.stateRaw);
if (credentialsRestored) {
if (snapshot.credentialsRaw === null) {
deletePersistedAuthProfileStoreRaw(agentDir, database);
} else {
writePersistedAuthProfileStoreRaw(snapshot.credentialsRaw, agentDir, database);
}
}
if (stateRestored) {
writePersistedAuthProfileStateRaw(snapshot.stateRaw, agentDir, database);
}
publishRuntimeSnapshots = () => {
// Main credential mutation lineage invalidates derived snapshots. Capture
// them first so exact-owned entries can restore and newer entries rebuild.
const currentRuntimeStores = listRuntimeAuthProfileStoreSnapshots().map(
({ agentDir: runtimeAgentDir, databasePath, store }) => ({
databasePath,
agentDir: runtimeAgentDir,
store,
runtimeRevision: getRuntimeAuthProfileStoreSnapshotRevisionAtDatabasePath(databasePath),
}),
runAuthProfileWriteTransaction(
agentDir,
(database) => {
const existingRaw = readPersistedAuthProfileStoreRaw(agentDir, database);
const existingState = readPersistedAuthProfileStateRaw(agentDir, database);
credentialsOwned = isDeepStrictEqual(existingRaw, owned.credentialsRaw);
stateOwned = isDeepStrictEqual(existingState, owned.stateRaw);
const beforeProfiles =
isRecord(existingRaw) && isRecord(existingRaw.profiles) ? existingRaw.profiles : {};
const restoredProfiles =
isRecord(snapshot.credentialsRaw) && isRecord(snapshot.credentialsRaw.profiles)
? snapshot.credentialsRaw.profiles
: {};
const changedProfileIds = [
...new Set([...Object.keys(beforeProfiles), ...Object.keys(restoredProfiles)]),
].filter(
(profileId) => !isDeepStrictEqual(beforeProfiles[profileId], restoredProfiles[profileId]),
);
const currentRuntimePath = agentDir
? resolveAgentAuthPath(agentDir)
: resolveSharedAuthPath();
const currentRuntimeRevision =
getRuntimeAuthProfileStoreSnapshotRevisionAtDatabasePath(currentRuntimePath);
if (credentialsRestored || stateRestored) {
noteRuntimeAuthProfileStorePersistedMutation(agentDir, {
credentialsChanged: credentialsRestored,
profileSetChanged: credentialsRestored && profileSetChanged,
stateChanged: stateRestored,
profileIds: credentialsRestored ? changedProfileIds : [],
});
const profileSetChanged = changedProfileIds.some(
(profileId) =>
Object.hasOwn(beforeProfiles, profileId) !== Object.hasOwn(restoredProfiles, profileId),
);
credentialsRestored =
credentialsOwned && !isDeepStrictEqual(existingRaw, snapshot.credentialsRaw);
stateRestored = stateOwned && !isDeepStrictEqual(existingState, snapshot.stateRaw);
if (credentialsRestored) {
if (snapshot.credentialsRaw === null) {
deletePersistedAuthProfileStoreRaw(agentDir, database);
} else {
writePersistedAuthProfileStoreRaw(snapshot.credentialsRaw, agentDir, database);
}
}
reconcileRuntimeAuthProfileStorePersistenceSnapshot({
snapshot,
owned,
agentDir,
credentialsOwned,
stateOwned,
credentialsRestored,
stateRestored,
currentRuntimeStores,
currentRuntimeRevision,
});
};
});
if (stateRestored) {
writePersistedAuthProfileStateRaw(snapshot.stateRaw, agentDir, database);
}
publishRuntimeSnapshots = () => {
// Main credential mutation lineage invalidates derived snapshots. Capture
// them first so exact-owned entries can restore and newer entries rebuild.
const currentRuntimeStores = listRuntimeAuthProfileStoreSnapshots().map(
({ agentDir: runtimeAgentDir, databasePath, store }) => ({
databasePath,
agentDir: runtimeAgentDir,
store,
runtimeRevision: getRuntimeAuthProfileStoreSnapshotRevisionAtDatabasePath(databasePath),
}),
);
const currentRuntimePath = agentDir ? resolveAgentAuthPath(agentDir) : database.path;
const currentRuntimeRevision =
getRuntimeAuthProfileStoreSnapshotRevisionAtDatabasePath(currentRuntimePath);
if (credentialsRestored || stateRestored) {
noteRuntimeAuthProfileStorePersistedMutation(agentDir, {
credentialsChanged: credentialsRestored,
profileSetChanged: credentialsRestored && profileSetChanged,
stateChanged: stateRestored,
profileIds: credentialsRestored ? changedProfileIds : [],
});
}
reconcileRuntimeAuthProfileStorePersistenceSnapshot({
snapshot,
owned,
agentDir,
credentialsOwned,
stateOwned,
credentialsRestored,
stateRestored,
currentRuntimeStores,
currentRuntimeRevision,
});
};
},
options,
);
publishRuntimeSnapshotsAfterCommit(publishRuntimeSnapshots);
}
/* oxlint-disable max-lines -- TODO: split this grandfathered oversized file. */
+2 -2
View File
@@ -17,7 +17,6 @@ import type {
OAuthProviderId,
} from "../../llm/utils/oauth/types.js";
import { OAuthProviderConfiguredUnavailableError } from "../../plugins/provider-runtime.errors.js";
import type { OpenClawAgentDatabase } from "../../state/openclaw-agent-db.js";
import { AUTH_STORE_VERSION, OAUTH_REFRESH_LOCK_OPTIONS } from "../auth-profiles/constants.js";
import {
assertAuthProfileMigrationReady,
@@ -32,6 +31,7 @@ import {
inspectPersistedAuthProfileStoreRaw,
resolveAuthProfileDatabasePath,
runAuthProfileWriteTransaction,
type AuthProfileDatabase,
} from "../auth-profiles/sqlite.js";
import { loadPersistedAuthProfileState } from "../auth-profiles/state.js";
import {
@@ -269,7 +269,7 @@ function collectStateOnlyAuthProfileIds(store: AuthProfileStore): string[] {
function loadSqliteAuthStorageStore(
agentDir: string,
database?: OpenClawAgentDatabase,
database?: AuthProfileDatabase,
): AuthProfileStore {
const inspection = inspectPersistedAuthProfileStoreRaw(agentDir, database);
if (inspection.status === "missing") {
+43 -35
View File
@@ -7,7 +7,9 @@ import { AUTH_STORE_VERSION } from "../agents/auth-profiles/constants.js";
import { loadPersistedAuthProfileStore } from "../agents/auth-profiles/persisted.js";
import { resolveAuthProfileDatabasePath } from "../agents/auth-profiles/sqlite.js";
import { saveAuthProfileStore } from "../agents/auth-profiles/store.js";
import type { AuthProfileStore } from "../agents/auth-profiles/types.js";
import { formatCliCommand } from "../cli/command-format.js";
import { writeConfigMachineState } from "../state/config-machine-state.js";
import { closeOpenClawAgentDatabasesForTest } from "../state/openclaw-agent-db.js";
import { closeOpenClawStateDatabaseForTest } from "../state/openclaw-state-db.js";
import { createSuiteTempRootTracker } from "../test-helpers/temp-dir.js";
@@ -305,14 +307,15 @@ describe("agents add command", () => {
);
});
it("reports only auth profiles persisted to the new agent store", async () => {
await withAgentsAddStateRoot("openclaw-agents-add-auth-copy-", async (root) => {
const sourceAgentDir = path.join(root, "agents", "main", "agent");
const destAgentDir = path.join(root, "agents", "work", "agent");
const workspaceDir = path.join(root, "workspace-work");
await fs.mkdir(sourceAgentDir, { recursive: true });
saveAuthProfileStore(
{
it.each(["legacy-main", "state-db"] as const)(
"reports only auth profiles persisted to the new agent store with %s shared auth",
async (location) => {
await withAgentsAddStateRoot("openclaw-agents-add-auth-copy-", async (root) => {
const sourceAgentDir = path.join(root, "agents", "main", "agent");
const destAgentDir = path.join(root, "agents", "work", "agent");
const workspaceDir = path.join(root, "workspace-work");
await fs.mkdir(sourceAgentDir, { recursive: true });
const sourceStore: AuthProfileStore = {
version: AUTH_STORE_VERSION,
profiles: {
"openai:api-key": {
@@ -329,34 +332,39 @@ describe("agents add command", () => {
copyToAgents: true,
},
},
},
sourceAgentDir,
);
readConfigFileSnapshotMock.mockResolvedValue({
...baseConfigSnapshot,
config: { agents: { list: [{ id: "main", default: true }] } },
sourceConfig: { agents: { list: [{ id: "main", default: true }] } },
};
if (location === "state-db") {
writeConfigMachineState("auth.sharedStore", { location: "state-db" });
saveAuthProfileStore(sourceStore);
} else {
saveAuthProfileStore(sourceStore, sourceAgentDir);
}
readConfigFileSnapshotMock.mockResolvedValue({
...baseConfigSnapshot,
config: { agents: { list: [{ id: "main", default: true }] } },
sourceConfig: { agents: { list: [{ id: "main", default: true }] } },
});
const prompter = {
intro: vi.fn(),
text: vi.fn().mockResolvedValueOnce("work").mockResolvedValueOnce(workspaceDir),
confirm: vi.fn().mockResolvedValueOnce(true).mockResolvedValueOnce(false),
note: vi.fn(),
outro: vi.fn(),
};
wizardMocks.createClackPrompter.mockReturnValue(prompter);
await agentsAddCommand({}, runtime);
expect(Object.keys(loadPersistedAuthProfileStore(destAgentDir)?.profiles ?? {})).toEqual([
"openai:api-key",
]);
expect(prompter.note).toHaveBeenCalledWith(
'Copied 1 portable auth profile from "main". OAuth profiles stay shared from "main" unless this agent signs in separately.',
"Auth profiles",
);
});
const prompter = {
intro: vi.fn(),
text: vi.fn().mockResolvedValueOnce("work").mockResolvedValueOnce(workspaceDir),
confirm: vi.fn().mockResolvedValueOnce(true).mockResolvedValueOnce(false),
note: vi.fn(),
outro: vi.fn(),
};
wizardMocks.createClackPrompter.mockReturnValue(prompter);
await agentsAddCommand({}, runtime);
expect(Object.keys(loadPersistedAuthProfileStore(destAgentDir)?.profiles ?? {})).toEqual([
"openai:api-key",
]);
expect(prompter.note).toHaveBeenCalledWith(
'Copied 1 portable auth profile from "main". OAuth profiles stay shared from "main" unless this agent signs in separately.',
"Auth profiles",
);
});
});
},
);
it("fails before config mutation when the source auth store is unreadable", async () => {
await withAgentsAddStateRoot("openclaw-agents-add-auth-unreadable-", async (root) => {
+10 -5
View File
@@ -17,13 +17,16 @@ import {
type AuthProfileStore,
} from "../agents/auth-profiles.js";
import { AuthProfileStoreUnreadableError } from "../agents/auth-profiles/legacy-source-diagnostic.js";
import { resolveSharedAuthStorePath } from "../agents/auth-profiles/path-resolve.js";
import { loadPersistedAuthProfileStore } from "../agents/auth-profiles/persisted.js";
import { resolveSharedMainAuthAgentDir } from "../agents/auth-profiles/shared-main-dir.js";
import {
inspectPersistedAuthProfileStoreRaw,
resolveAuthProfileDatabasePath,
} from "../agents/auth-profiles/sqlite.js";
import { saveAuthProfileStore } from "../agents/auth-profiles/store.js";
import {
loadAuthProfileStoreWithoutExternalProfiles,
saveAuthProfileStore,
} from "../agents/auth-profiles/store.js";
import { formatCliCommand } from "../cli/command-format.js";
import { logConfigUpdated } from "../config/logging.js";
import {
@@ -257,15 +260,17 @@ export async function agentsAddCommand(
const sourceAgentDir = resolveAgentDir(cfg, defaultAgentId);
const sourceAuthPath = resolveAuthProfileDatabasePath(sourceAgentDir);
const destAuthPath = resolveAuthProfileDatabasePath(agentDir);
const mainAuthPath = resolveSharedAuthStorePath();
const sharedMainAgentPath = resolveAuthProfileDatabasePath(resolveSharedMainAuthAgentDir());
const sameAuthPath =
normalizeLowercaseStringOrEmpty(path.resolve(sourceAuthPath)) ===
normalizeLowercaseStringOrEmpty(path.resolve(destAuthPath));
const sourceIsInheritedMain =
normalizeLowercaseStringOrEmpty(path.resolve(sourceAuthPath)) ===
normalizeLowercaseStringOrEmpty(path.resolve(mainAuthPath));
normalizeLowercaseStringOrEmpty(path.resolve(sharedMainAgentPath));
if (!sameAuthPath) {
const sourceStore = loadReadablePersistedAuthProfileStore(sourceAgentDir);
const sourceStore = sourceIsInheritedMain
? loadAuthProfileStoreWithoutExternalProfiles(sourceAgentDir)
: loadReadablePersistedAuthProfileStore(sourceAgentDir);
const destStore = loadReadablePersistedAuthProfileStore(agentDir);
const portable = sourceStore
? buildPortableAuthProfileStoreForAgentCopy(sourceStore)
+7 -3
View File
@@ -114,9 +114,10 @@ export async function agentsDeleteCommand(
runtime.log(`Normalized agent id to "${agentId}".`);
}
const agentDir = resolveAgentDir(cfg, agentId);
const sharedAuthOwnership = resolveSharedAuthStoreOwnership();
if (
isSharedAuthStoreOwner({
ownership: resolveSharedAuthStoreOwnership(),
ownership: sharedAuthOwnership,
agentAuthDbPath: resolveAuthProfileDatabasePath(agentDir),
sharedAuthDbPath: resolveSharedAuthStorePath(),
})
@@ -137,8 +138,11 @@ export async function agentsDeleteCommand(
runtime.exit(1);
return;
}
if (agentId === normalizeAgentId(resolveLegacyInheritedAuthAgentId(cfg))) {
// H2-2 owns credential relocation; deleting this directory first destroys the shared store.
const explicitInheritedAuthAgentId = cfg.agents?.defaults?.authInheritance?.agentId?.trim();
const inheritedAuthAgentId =
explicitInheritedAuthAgentId ||
(sharedAuthOwnership.location === "legacy-main" ? resolveLegacyInheritedAuthAgentId(cfg) : "");
if (inheritedAuthAgentId && agentId === normalizeAgentId(inheritedAuthAgentId)) {
runtime.error(
`Agent "${agentId}" owns inherited credentials through agents.defaults.authInheritance.agentId and cannot be deleted. Relocate those credentials, then re-point or remove that binding before retrying.`,
);
+30
View File
@@ -19,6 +19,7 @@ import {
} from "../config/sessions/session-accessor.js";
import type { OpenClawConfig } from "../config/types.openclaw.js";
import { parseAgentSessionKey } from "../routing/session-key.js";
import { writeConfigMachineState } from "../state/config-machine-state.js";
import { withStateDirEnv } from "../test-helpers/state-dir-env.js";
import { baseConfigSnapshot, createTestRuntime } from "./test-runtime-config-helpers.js";
@@ -239,6 +240,35 @@ describe("agents delete command", () => {
});
});
it("deletes main normally after shared auth ownership moves to state SQLite", async () => {
await withStateDirEnv("openclaw-agents-delete-relocated-auth-", async ({ stateDir }) => {
const cfg: OpenClawConfig = {
agents: {
list: [
{ id: "main", workspace: path.join(stateDir, "workspace-main") },
{ id: "ops", default: true, workspace: path.join(stateDir, "workspace-ops") },
],
},
};
writeConfigMachineState("auth.sharedStore", { location: "state-db" });
await arrangeAgentsDeleteTest({
stateDir,
cfg,
deletedAgentId: "main",
sessions: {
"agent:main:main": { sessionId: "sess-main", updatedAt: Date.now() },
},
});
await agentsDeleteCommand({ id: "main", force: true, json: true }, runtime);
expect(runtime.error).not.toHaveBeenCalled();
expect(runtime.exit).not.toHaveBeenCalledWith(1);
expect(configMocks.replaceConfigFile).toHaveBeenCalledOnce();
expectSessionStore(cfg, {}, "main");
});
});
it("refuses deleting the auth-inheritance owner until credentials are relocated", async () => {
await withStateDirEnv("openclaw-agents-delete-auth-owner-", async ({ stateDir }) => {
const cfg: OpenClawConfig = {
+73 -48
View File
@@ -5,7 +5,10 @@ import { DatabaseSync } from "node:sqlite";
import { afterEach, describe, expect, it, vi } from "vitest";
import { listAuthProfileStoresRequiringMigration } from "../agents/auth-profiles/legacy-source-diagnostic.js";
import { resolveAuthProfileEligibility } from "../agents/auth-profiles/order.js";
import { loadPersistedAuthProfileStore } from "../agents/auth-profiles/persisted.js";
import {
loadPersistedAuthProfileStore,
loadPersistedSharedAuthProfileStore,
} from "../agents/auth-profiles/persisted.js";
import { clearRuntimeAuthProfileStoreSnapshots } from "../agents/auth-profiles/runtime-snapshots.js";
import {
readPersistedAuthProfileStoreRaw,
@@ -21,6 +24,11 @@ import {
replaceSessionEntry,
} from "../config/sessions/session-accessor.js";
import type { OpenClawConfig } from "../config/types.openclaw.js";
import {
detectSharedAuthStoreMigration,
migrateSharedAuthStore,
} from "../infra/state-migrations.shared-auth-store.js";
import { writeConfigMachineState } from "../state/config-machine-state.js";
import { closeOpenClawAgentDatabasesForTest } from "../state/openclaw-agent-db.js";
import {
closeOpenClawStateDatabaseForTest,
@@ -91,9 +99,9 @@ async function expectSelectedCodexAccountStatus(params: {
fetchUsageSnapshot: async (context) => {
usageProfileIds.push(context.authProfileId);
const selectedProfile = context.authProfileId ?? "openai:peter";
const credential = loadPersistedAuthProfileStore(params.state.agentDir())?.profiles[
selectedProfile
];
const credential =
loadPersistedAuthProfileStore(params.state.agentDir())?.profiles[selectedProfile] ??
loadPersistedSharedAuthProfileStore(params.state.env)?.profiles[selectedProfile];
return {
provider: "openai",
displayName: "OpenAI",
@@ -495,53 +503,64 @@ describe("maybeMigrateAuthProfileJsonStoresToSqlite", () => {
expect(readPersistedAuthProfileStoreRaw(state.agentDir())).toEqual(unreadableStore);
});
it("imports legacy JSON auth profiles and state into the agent sqlite database", async () => {
const state = await makeTestState();
const authPath = await writeLegacyAuthProfilesJson(state, {
version: 1,
profiles: {
"openai:default": {
type: "api_key",
provider: "openai",
key: "sk-migrated",
it.each(["legacy-main", "state-db"] as const)(
"imports legacy JSON auth profiles and state into the %s shared database",
async (location) => {
const state = await makeTestState();
if (location === "state-db") {
writeConfigMachineState("auth.sharedStore", { location: "state-db" }, { env: state.env });
}
const authPath = await writeLegacyAuthProfilesJson(state, {
version: 1,
profiles: {
"openai:default": {
type: "api_key",
provider: "openai",
key: "sk-migrated",
},
},
},
order: { openai: ["openai:default"] },
});
const statePath = await state.writeText(
"agents/main/agent/auth-state.json",
`${JSON.stringify({ version: 1, lastGood: { openai: "openai:default" } })}\n`,
);
order: { openai: ["openai:default"] },
});
const statePath = await state.writeText(
"agents/main/agent/auth-state.json",
`${JSON.stringify({ version: 1, lastGood: { openai: "openai:default" } })}\n`,
);
const result = await maybeMigrateAuthProfileJsonStoresToSqlite({
cfg: {},
prompter: makePrompter(true),
now: () => 456,
});
const result = await maybeMigrateAuthProfileJsonStoresToSqlite({
cfg: {},
prompter: makePrompter(true),
now: () => 456,
env: state.env,
});
expect(result.detected.toSorted()).toEqual([authPath, statePath].toSorted());
expect(result.warnings).toStrictEqual([]);
expect(loadPersistedAuthProfileStore(state.agentDir())).toMatchObject({
profiles: {
"openai:default": {
type: "api_key",
provider: "openai",
key: "sk-migrated",
expect(result.detected.toSorted()).toEqual([authPath, statePath].toSorted());
expect(result.warnings).toStrictEqual([]);
const loaded =
location === "state-db"
? loadPersistedSharedAuthProfileStore(state.env)
: loadPersistedAuthProfileStore(state.agentDir());
expect(loaded).toMatchObject({
profiles: {
"openai:default": {
type: "api_key",
provider: "openai",
key: "sk-migrated",
},
},
},
lastGood: { openai: "openai:default" },
});
expect(fs.existsSync(authPath)).toBe(false);
expect(fs.existsSync(statePath)).toBe(false);
expectMigratedArchive(authPath);
expectMigratedArchive(statePath);
const combinedReceipt = openOpenClawStateDatabase({ env: state.env })
.db.prepare("SELECT report_json FROM migration_sources WHERE source_path = ?")
.get(authPath) as { report_json?: string } | undefined;
expect(JSON.parse(combinedReceipt?.report_json ?? "null")?.expectedStateSha256).toEqual(
expect.any(String),
);
});
lastGood: { openai: "openai:default" },
});
expect(fs.existsSync(authPath)).toBe(false);
expect(fs.existsSync(statePath)).toBe(false);
expectMigratedArchive(authPath);
expectMigratedArchive(statePath);
const combinedReceipt = openOpenClawStateDatabase({ env: state.env })
.db.prepare("SELECT report_json FROM migration_sources WHERE source_path = ?")
.get(authPath) as { report_json?: string } | undefined;
expect(JSON.parse(combinedReceipt?.report_json ?? "null")?.expectedStateSha256).toEqual(
expect.any(String),
);
},
);
it("imports a valid legacy auth sibling when auth-profiles.json is malformed", async () => {
const state = await makeTestState();
@@ -2206,6 +2225,12 @@ describe("legacy OpenAI auth profiles through the canonical migration owner", ()
authProfileOverride: "openai-codex:default",
});
const detected = detectSharedAuthStoreMigration({
stateDir: state.stateDir,
doctorOnlyStateMigrations: true,
});
await migrateSharedAuthStore({ detected, stateDir: state.stateDir, env: state.env });
const recoveredMap = collectOpenAICodexAuthProfileStoreIdMap({ cfg: {}, env: state.env });
expect(recoveredMap.get("openai-codex:default")).toBe("openai:chatgpt-default");
const repair = await maybeRepairCodexSessionRoutes({
@@ -2220,7 +2245,7 @@ describe("legacy OpenAI auth profiles through the canonical migration owner", ()
authProfileOverride: "openai:chatgpt-default",
authProfileOverrideSource: "user",
});
expect(loadPersistedAuthProfileStore(state.agentDir())?.profiles).toMatchObject({
expect(loadPersistedSharedAuthProfileStore(state.env)?.profiles).toMatchObject({
"openai:default": { type: "api_key" },
"openai:chatgpt-default": { accountId: "kate-account" },
"openai:peter": { accountId: "peter-account" },
+206 -136
View File
@@ -19,21 +19,30 @@ import {
hasMatchingOAuthIdentity,
} from "../agents/auth-profiles/oauth-shared.js";
import { isInheritedMainOAuthCredentialFromStores } from "../agents/auth-profiles/ownership.js";
import { resolveSharedAuthStorePath } from "../agents/auth-profiles/path-resolve.js";
import {
resolveSharedAuthStoreOwnership,
resolveSharedAuthStorePath,
} from "../agents/auth-profiles/path-resolve.js";
import {
applyLegacyAuthStore,
coerceLegacyAuthStore,
coercePersistedAuthProfileStore,
loadPersistedAuthProfileStore,
loadPersistedSharedAuthProfileStore,
parseLegacyCredentialEntry,
} from "../agents/auth-profiles/persisted.js";
import { clearRuntimeAuthProfileStoreSnapshots } from "../agents/auth-profiles/runtime-snapshots.js";
import { resolveSharedMainAuthAgentDir } from "../agents/auth-profiles/shared-main-dir.js";
import {
inspectPersistedAuthProfileStateRaw,
inspectPersistedAuthProfileStoreRaw,
inspectPersistedSharedAuthProfileStateRaw,
inspectPersistedSharedAuthProfileStoreRaw,
readPersistedAuthProfileStateRaw,
readPersistedSharedAuthProfileStateRaw,
resolveAuthProfileDatabasePath,
runAuthProfileWriteTransaction,
type AuthProfileDatabase,
} from "../agents/auth-profiles/sqlite.js";
import { coerceAuthProfileState } from "../agents/auth-profiles/state.js";
import { saveAuthProfileStore } from "../agents/auth-profiles/store.js";
@@ -51,7 +60,6 @@ import type { OpenClawConfig } from "../config/types.openclaw.js";
import { coerceSecretRef } from "../config/types.secrets.js";
import { loadJsonFileThroughSymlink } from "../infra/json-file.js";
import { readLegacyMigrationReceipt } from "../infra/state-migrations.receipts.js";
import type { OpenClawAgentDatabase } from "../state/openclaw-agent-db.js";
import { shortenHomePath } from "../utils.js";
import {
resolveLegacyAuthProfilesPath as resolveAuthStorePath,
@@ -80,8 +88,11 @@ type AuthProfileSqliteMigrationCandidate = AuthProfileRepairCandidate & {
legacyPath: string;
};
function resolveMigrationTargetDatabasePath(agentDir?: string): string {
return agentDir ? resolveAuthProfileDatabasePath(agentDir) : resolveSharedAuthStorePath();
function resolveMigrationTargetDatabasePath(
agentDir?: string,
env: NodeJS.ProcessEnv = process.env,
): string {
return agentDir ? resolveAuthProfileDatabasePath(agentDir) : resolveSharedAuthStorePath(env);
}
type AwsSdkProfileMarker = {
@@ -287,7 +298,11 @@ function addCandidate(
agentDir: string | undefined,
): void {
const authPath = resolveAuthStorePath(agentDir);
candidates.set(path.resolve(authPath), { agentDir, authPath });
const key = path.resolve(authPath);
const existing = candidates.get(key);
if (!existing || agentDir === undefined) {
candidates.set(key, { agentDir, authPath });
}
}
function listExistingAgentDirsFromState(env: NodeJS.ProcessEnv): string[] {
@@ -315,6 +330,7 @@ function listAuthProfileRepairCandidates(
env: NodeJS.ProcessEnv,
): AuthProfileRepairCandidate[] {
const candidates = new Map<string, AuthProfileRepairCandidate>();
addCandidate(candidates, undefined);
addCandidate(candidates, resolveLegacyInheritedAuthDir(cfg, env));
const envAgentDir =
readNonEmptyString(env.OPENCLAW_AGENT_DIR) ?? readNonEmptyString(env.PI_CODING_AGENT_DIR);
@@ -491,7 +507,8 @@ function isDefaultAgentCandidate(
env: NodeJS.ProcessEnv,
): boolean {
return (
path.resolve(candidate.agentDir ?? "") === path.resolve(resolveLegacyInheritedAuthDir(cfg, env))
candidate.agentDir === undefined ||
path.resolve(candidate.agentDir) === path.resolve(resolveLegacyInheritedAuthDir(cfg, env))
);
}
@@ -658,6 +675,7 @@ function prepareAuthProfileSourceReceipt(params: {
pathname: string;
targetDatabasePath: string;
targetTable: AuthProfileMigrationSourceReceipt["targetTable"];
targetStoreKey?: AuthProfileMigrationSourceReceipt["targetStoreKey"];
now: () => number;
env?: NodeJS.ProcessEnv;
}): AuthProfileMigrationSourceReceipt {
@@ -675,6 +693,7 @@ function prepareAuthProfileSourceReceipt(params: {
sourceRecordCount,
targetDatabasePath: params.targetDatabasePath,
targetTable: params.targetTable,
...(params.targetStoreKey ? { targetStoreKey: params.targetStoreKey } : {}),
now: new Date(params.now()),
...(params.env ? { env: params.env } : {}),
});
@@ -765,24 +784,37 @@ function coerceLegacyOAuthFile(raw: unknown): {
function loadAuthProfileMigrationTargetStore(
agentDir: string | undefined,
loadStore: typeof loadPersistedAuthProfileStore = loadPersistedAuthProfileStore,
database?: OpenClawAgentDatabase,
database?: AuthProfileDatabase,
env: NodeJS.ProcessEnv = process.env,
): AuthProfileStore {
const inspection = inspectPersistedAuthProfileStoreRaw(agentDir, database);
const store = loadStore(agentDir, database ? { database } : undefined);
const explicitSharedRead = agentDir === undefined && database === undefined;
const inspection = explicitSharedRead
? inspectPersistedSharedAuthProfileStoreRaw(env)
: inspectPersistedAuthProfileStoreRaw(agentDir, database);
const store =
explicitSharedRead && loadStore === loadPersistedAuthProfileStore
? loadPersistedSharedAuthProfileStore(env)
: loadStore(agentDir, database ? { database } : undefined);
if (store) {
return store;
}
if (inspection.status !== "missing") {
throw new Error("canonical auth profile store is unreadable; legacy source left in place");
}
const stateInspection = inspectPersistedAuthProfileStateRaw(agentDir, database);
const stateInspection = explicitSharedRead
? inspectPersistedSharedAuthProfileStateRaw(env)
: inspectPersistedAuthProfileStateRaw(agentDir, database);
if (stateInspection.status === "unreadable") {
throw new Error("canonical auth profile state is unreadable; legacy source left in place");
}
return {
version: AUTH_STORE_VERSION,
profiles: {},
...coerceAuthProfileState(readPersistedAuthProfileStateRaw(agentDir, database)),
...coerceAuthProfileState(
explicitSharedRead
? readPersistedSharedAuthProfileStateRaw(env)
: readPersistedAuthProfileStateRaw(agentDir, database),
),
};
}
@@ -809,12 +841,13 @@ function migrateLockedLegacyOAuthFile(params: {
now: () => number;
result: LegacyFlatAuthProfileRepairResult;
}): void {
const mainAgentDir = path.dirname(resolveSharedAuthStorePath(params.env));
const targetDatabasePath = resolveAuthProfileDatabasePath(mainAgentDir);
const targetDatabasePath = resolveSharedAuthStorePath(params.env);
const sharedStateTarget = resolveSharedAuthStoreOwnership(params.env).location === "state-db";
const receipt = prepareAuthProfileSourceReceipt({
pathname: params.oauthPath,
targetDatabasePath,
targetTable: "auth_profile_store",
targetTable: sharedStateTarget ? "auth_profile_stores" : "auth_profile_store",
targetStoreKey: sharedStateTarget ? "shared" : "primary",
now: params.now,
env: params.env,
});
@@ -831,51 +864,60 @@ function migrateLockedLegacyOAuthFile(params: {
);
return;
}
const existing = loadAuthProfileMigrationTargetStore(mainAgentDir);
const existing = loadAuthProfileMigrationTargetStore(
undefined,
loadPersistedAuthProfileStore,
undefined,
params.env,
);
const importedProfileIds = new Set(Object.keys(imported.profiles));
const next = mergeImportedAuthProfiles({
store: existing,
profiles: imported.profiles,
existingProfileIds: new Set(Object.keys(existing.profiles)),
});
const loaded = runAuthProfileWriteTransaction(mainAgentDir, (database) => {
const authoritative = loadAuthProfileMigrationTargetStore(
mainAgentDir,
loadPersistedAuthProfileStore,
database,
);
if (!isDeepStrictEqual(authoritative, existing)) {
throw new Error("canonical auth profile store changed during legacy OAuth migration");
}
saveAuthProfileStore(
next,
mainAgentDir,
{
filterExternalAuthProfiles: false,
preserveStateProfileIds: collectAuthProfileStateProfileIds(
coerceAuthProfileState(existing),
),
syncExternalCli: false,
},
database,
);
const verified = loadPersistedAuthProfileStore(mainAgentDir, { database });
const verificationFailure = formatMissingAuthProfileSqliteVerification({
expected: next,
importedProfileIds,
loaded: verified,
});
const mismatched = [...importedProfileIds].filter((profileId) => {
if (existing.profiles[profileId]) {
return false;
const loaded = runAuthProfileWriteTransaction(
undefined,
(database) => {
const authoritative = loadAuthProfileMigrationTargetStore(
undefined,
loadPersistedAuthProfileStore,
database,
);
if (!isDeepStrictEqual(authoritative, existing)) {
throw new Error("canonical auth profile store changed during legacy OAuth migration");
}
return !isDeepStrictEqual(verified?.profiles[profileId], imported.profiles[profileId]);
});
if (verificationFailure || mismatched.length > 0 || !verified) {
throw new Error("legacy OAuth import verification failed");
}
return verified;
});
saveAuthProfileStore(
next,
undefined,
{
filterExternalAuthProfiles: false,
preserveStateProfileIds: collectAuthProfileStateProfileIds(
coerceAuthProfileState(existing),
),
syncExternalCli: false,
},
database,
);
const verified = loadPersistedAuthProfileStore(undefined, { database });
const verificationFailure = formatMissingAuthProfileSqliteVerification({
expected: next,
importedProfileIds,
loaded: verified,
});
const mismatched = [...importedProfileIds].filter((profileId) => {
if (existing.profiles[profileId]) {
return false;
}
return !isDeepStrictEqual(verified?.profiles[profileId], imported.profiles[profileId]);
});
if (verificationFailure || mismatched.length > 0 || !verified) {
throw new Error("legacy OAuth import verification failed");
}
return verified;
},
{ env: params.env },
);
receipt.expectedProfileSha256 = Object.fromEntries(
[...importedProfileIds].map((profileId) => [
profileId,
@@ -1001,13 +1043,21 @@ export async function maybeMigrateAuthProfileJsonStoresToSqlite(params: {
fs.mkdirSync(path.dirname(pathname), { recursive: true });
}
releaseSources = acquireAuthProfileMigrationSourceLocks(candidateSourcePaths);
const targetDatabasePath = resolveMigrationTargetDatabasePath(candidate.agentDir);
const targetDatabasePath = resolveMigrationTargetDatabasePath(candidate.agentDir, env);
const sharedStateTarget =
candidate.agentDir === undefined &&
resolveSharedAuthStoreOwnership(env).location === "state-db";
let sourceReceipts = candidateSourcePaths.filter(fs.existsSync).map((pathname) =>
prepareAuthProfileSourceReceipt({
pathname,
targetDatabasePath,
targetTable:
pathname === candidate.statePath ? "auth_profile_state" : "auth_profile_store",
pathname === candidate.statePath
? "auth_profile_state"
: sharedStateTarget
? "auth_profile_stores"
: "auth_profile_store",
targetStoreKey: sharedStateTarget ? "shared" : "primary",
now,
env,
}),
@@ -1096,7 +1146,12 @@ export async function maybeMigrateAuthProfileJsonStoresToSqlite(params: {
continue;
}
const existing = loadAuthProfileMigrationTargetStore(candidate.agentDir, loadMigratedStore);
const existing = loadAuthProfileMigrationTargetStore(
candidate.agentDir,
loadMigratedStore,
undefined,
env,
);
const existingProfileIds = new Set(Object.keys(existing.profiles));
const existingState = coerceAuthProfileState(existing);
let next: AuthProfileStore = { ...existing };
@@ -1162,80 +1217,83 @@ export async function maybeMigrateAuthProfileJsonStoresToSqlite(params: {
];
try {
assertAuthProfileMigrationSourcesUnchanged(candidate, sourceReceipts);
verifiedStore = runAuthProfileWriteTransaction(candidate.agentDir, (database) => {
const authoritative = loadAuthProfileMigrationTargetStore(
candidate.agentDir,
loadMigratedStore,
database,
);
// This store includes the separately persisted auth_profile_state row,
// so state-only concurrent changes abort before either table is written.
if (!isDeepStrictEqual(authoritative, existing)) {
throw new Error("canonical auth profile store changed during legacy migration");
}
saveAuthProfileStore(
next,
candidate.agentDir,
{
filterExternalAuthProfiles: false,
// Imported state may reference external profiles absent from this store.
preserveStateProfileIds: stateProfileIds,
syncExternalCli: false,
},
database,
);
const loaded = loadMigratedStore(candidate.agentDir, { database });
const mainAgentDir = path.dirname(resolveSharedAuthStorePath(params.env));
const persistedStores = {
isMainStore:
resolveMigrationTargetDatabasePath(candidate.agentDir) ===
resolveAuthProfileDatabasePath(mainAgentDir),
localStore: loaded,
mainStore:
resolveMigrationTargetDatabasePath(candidate.agentDir) ===
resolveAuthProfileDatabasePath(mainAgentDir)
? loaded
: loadPersistedAuthProfileStore(mainAgentDir),
};
// A non-main store drops an OAuth credential the main store already
// owns at the same or newer expiry. That dedup is intentional, so
// verifying it as missing would abort a migration that lost nothing
// and leave the legacy JSON in place, which blocks gateway startup.
const dedupedToMainProfileIds = new Set(
[...importedProfileIds].filter((profileId) => {
const credential = next.profiles[profileId];
return (
credential !== undefined &&
!loaded?.profiles[profileId] &&
isInheritedMainOAuthCredentialFromStores({
profileId,
credential,
persistedStores,
})
);
}),
);
const verifiableProfileIds = new Set(
[...importedProfileIds].filter(
(profileId) => !dedupedToMainProfileIds.has(profileId),
),
);
const verificationFailure = formatMissingAuthProfileSqliteVerification({
expected: next,
importedProfileIds: verifiableProfileIds,
loaded,
});
const mismatchedCredential = [...verifiableProfileIds].some((profileId) => {
if (existingProfileIds.has(profileId)) {
return false;
verifiedStore = runAuthProfileWriteTransaction(
candidate.agentDir,
(database) => {
const authoritative = loadAuthProfileMigrationTargetStore(
candidate.agentDir,
loadMigratedStore,
database,
);
// This store includes the separately persisted auth_profile_state row,
// so state-only concurrent changes abort before either table is written.
if (!isDeepStrictEqual(authoritative, existing)) {
throw new Error("canonical auth profile store changed during legacy migration");
}
return !isDeepStrictEqual(loaded?.profiles[profileId], next.profiles[profileId]);
});
if (verificationFailure || mismatchedCredential || !loaded) {
throw new AuthProfileMigrationVerificationError(verificationFailure);
}
return loaded;
});
saveAuthProfileStore(
next,
candidate.agentDir,
{
filterExternalAuthProfiles: false,
// Imported state may reference external profiles absent from this store.
preserveStateProfileIds: stateProfileIds,
syncExternalCli: false,
},
database,
);
const loaded = loadMigratedStore(candidate.agentDir, { database });
const persistedStores = {
isMainStore:
resolveMigrationTargetDatabasePath(candidate.agentDir, env) ===
resolveSharedAuthStorePath(env),
localStore: loaded,
mainStore:
resolveMigrationTargetDatabasePath(candidate.agentDir, env) ===
resolveSharedAuthStorePath(env)
? loaded
: loadPersistedSharedAuthProfileStore(env),
};
// A non-main store drops an OAuth credential the main store already
// owns at the same or newer expiry. That dedup is intentional, so
// verifying it as missing would abort a migration that lost nothing
// and leave the legacy JSON in place, which blocks gateway startup.
const dedupedToMainProfileIds = new Set(
[...importedProfileIds].filter((profileId) => {
const credential = next.profiles[profileId];
return (
credential !== undefined &&
!loaded?.profiles[profileId] &&
isInheritedMainOAuthCredentialFromStores({
profileId,
credential,
persistedStores,
})
);
}),
);
const verifiableProfileIds = new Set(
[...importedProfileIds].filter(
(profileId) => !dedupedToMainProfileIds.has(profileId),
),
);
const verificationFailure = formatMissingAuthProfileSqliteVerification({
expected: next,
importedProfileIds: verifiableProfileIds,
loaded,
});
const mismatchedCredential = [...verifiableProfileIds].some((profileId) => {
if (existingProfileIds.has(profileId)) {
return false;
}
return !isDeepStrictEqual(loaded?.profiles[profileId], next.profiles[profileId]);
});
if (verificationFailure || mismatchedCredential || !loaded) {
throw new AuthProfileMigrationVerificationError(verificationFailure);
}
return loaded;
},
{ env },
);
} catch (error) {
if (!(error instanceof AuthProfileMigrationVerificationError)) {
throw error;
@@ -1310,7 +1368,7 @@ export async function maybeMigrateAuthProfileJsonStoresToSqlite(params: {
releaseSources?.();
}
}
const sharedMainAgentDir = path.dirname(resolveSharedAuthStorePath(env));
const sharedMainAgentDir = resolveSharedMainAuthAgentDir(env);
const sharedMainCredentialSourceRemains = [
resolveAuthStorePath(sharedMainAgentDir),
resolveLegacyAuthStorePath(sharedMainAgentDir),
@@ -1712,14 +1770,19 @@ function recoverArchivedOpenAICodexAuthProfileIdMap(params: {
}): Map<string, string> {
const recovered = new Map<string, string>();
const ambiguous = new Set<string>();
const agentDirs = params.candidates.flatMap((candidate) =>
candidate.agentDir ? [candidate.agentDir] : [],
);
const agentDirs = [
resolveSharedMainAuthAgentDir(params.env),
...params.candidates.flatMap((candidate) => (candidate.agentDir ? [candidate.agentDir] : [])),
];
const archives = listLegacyAuthProfileArchives({ agentDirs, env: params.env }).filter(
(archive) => archive.kind === "auth-profiles",
);
for (const candidate of params.candidates) {
const canonicalProfiles = loadPersistedAuthProfileStore(candidate.agentDir)?.profiles;
const canonicalProfiles = (
candidate.agentDir
? loadPersistedAuthProfileStore(candidate.agentDir)
: loadPersistedSharedAuthProfileStore(params.env)
)?.profiles;
if (!canonicalProfiles) {
continue;
}
@@ -1737,16 +1800,20 @@ function recoverArchivedOpenAICodexAuthProfileIdMap(params: {
continue;
}
const report = JSON.parse(receipt.reportJson) as unknown;
const sharedStateTarget =
candidate.agentDir === undefined &&
resolveSharedAuthStoreOwnership(params.env).location === "state-db";
if (
!isRecord(report) ||
report.format !== "auth-profile-json-to-sqlite-v2" ||
report.completionStatus !== "completed" ||
report.targetTable !== "auth_profile_store" ||
report.targetTable !==
(sharedStateTarget ? "auth_profile_stores" : "auth_profile_store") ||
typeof report.archivePath !== "string" ||
path.resolve(report.archivePath) !== path.resolve(archive.path) ||
typeof report.targetDatabasePath !== "string" ||
path.resolve(report.targetDatabasePath) !==
path.resolve(resolveMigrationTargetDatabasePath(candidate.agentDir)) ||
path.resolve(resolveMigrationTargetDatabasePath(candidate.agentDir, params.env)) ||
!isRecord(report.expectedProfileSha256)
) {
continue;
@@ -1820,7 +1887,10 @@ export function collectOpenAICodexAuthProfileStoreIdMap(params: {
};
addProfileIds(Object.keys(params.cfg.auth?.profiles ?? {}));
for (const candidate of candidates) {
addProfileIds(Object.keys(loadPersistedAuthProfileStore(candidate.agentDir)?.profiles ?? {}));
const persistedStore = candidate.agentDir
? loadPersistedAuthProfileStore(candidate.agentDir)
: loadPersistedSharedAuthProfileStore(env);
addProfileIds(Object.keys(persistedStore?.profiles ?? {}));
if (!fs.existsSync(candidate.authPath)) {
continue;
}
+2 -2
View File
@@ -1,9 +1,9 @@
import path from "node:path";
import { resolveSharedAuthStoreDir } from "../agents/auth-profiles/path-resolve.js";
import { resolveSharedMainAuthAgentDir } from "../agents/auth-profiles/shared-main-dir.js";
import { resolveUserPath } from "../utils.js";
function resolveLegacyAuthAgentDir(agentDir?: string): string {
return agentDir ? resolveUserPath(agentDir) : resolveSharedAuthStoreDir();
return agentDir ? resolveUserPath(agentDir) : resolveSharedMainAuthAgentDir();
}
export function resolveLegacyAuthProfilesPath(agentDir?: string): string {
+46 -17
View File
@@ -26,6 +26,10 @@ type AuthProfileTargetDatabase = Pick<
OpenClawAgentKyselyDatabase,
"auth_profile_store" | "auth_profile_state"
>;
type SharedAuthProfileTargetDatabase = Pick<
OpenClawStateDatabase,
"auth_profile_stores" | "auth_profile_state"
>;
export type AuthProfileMigrationSourceReceipt = {
sourceKey: string;
@@ -37,7 +41,8 @@ export type AuthProfileMigrationSourceReceipt = {
/** In-memory migration snapshot; never serialized into the receipt ledger or diagnostics. */
sourceBytes?: Buffer;
targetDatabasePath: string;
targetTable: "auth_profile_store" | "auth_profile_state";
targetTable: "auth_profile_store" | "auth_profile_stores" | "auth_profile_state";
targetStoreKey?: "primary" | "shared";
archivePath: string;
expectedProfileSha256?: Record<string, string>;
expectedStateSha256?: string;
@@ -55,6 +60,7 @@ export function createAuthProfileMigrationSourceReceipt(params: {
sourceRecordCount: number;
targetDatabasePath: string;
targetTable: AuthProfileMigrationSourceReceipt["targetTable"];
targetStoreKey?: AuthProfileMigrationSourceReceipt["targetStoreKey"];
now?: Date;
env?: NodeJS.ProcessEnv;
}): AuthProfileMigrationSourceReceipt {
@@ -72,6 +78,7 @@ export function createAuthProfileMigrationSourceReceipt(params: {
sourceBytes: Buffer.from(params.sourceBytes),
targetDatabasePath: path.resolve(params.targetDatabasePath),
targetTable: params.targetTable,
...(params.targetStoreKey ? { targetStoreKey: params.targetStoreKey } : {}),
archivePath: `${sourcePath}.migrated-${stamp}-${randomUUID()}`,
...(params.env ? { env: params.env } : {}),
};
@@ -83,6 +90,7 @@ function reportJson(receipt: AuthProfileMigrationSourceReceipt): string {
archivePath: receipt.archivePath,
targetDatabasePath: receipt.targetDatabasePath,
targetTable: receipt.targetTable,
targetStoreKey: receipt.targetStoreKey ?? "primary",
expectedProfileSha256: receipt.expectedProfileSha256,
expectedStateSha256: receipt.expectedStateSha256,
completionStatus: receipt.completionStatus ?? "completed",
@@ -262,15 +270,24 @@ function verifyAuthProfileMigrationTarget(receipt: AuthProfileMigrationSourceRec
}
const db = openNodeSqliteDatabase(receipt.targetDatabasePath, { readOnly: true });
try {
const kysely = getNodeSqliteKysely<AuthProfileTargetDatabase>(db);
const targetStoreKey = receipt.targetStoreKey ?? "primary";
if (hasExpectedProfiles && receipt.expectedProfileSha256) {
const row = executeSqliteQueryTakeFirstSync(
db,
kysely
.selectFrom("auth_profile_store")
.select("store_json")
.where("store_key", "=", "primary"),
);
const row =
targetStoreKey === "shared"
? executeSqliteQueryTakeFirstSync(
db,
getNodeSqliteKysely<SharedAuthProfileTargetDatabase>(db)
.selectFrom("auth_profile_stores")
.select("store_json")
.where("store_key", "=", "shared"),
)
: executeSqliteQueryTakeFirstSync(
db,
getNodeSqliteKysely<AuthProfileTargetDatabase>(db)
.selectFrom("auth_profile_store")
.select("store_json")
.where("store_key", "=", "primary"),
);
const store = typeof row?.store_json === "string" ? JSON.parse(row.store_json) : null;
for (const [profileId, expectedSha256] of Object.entries(receipt.expectedProfileSha256)) {
if (digestAuthProfileMigrationValue(store?.profiles?.[profileId]) !== expectedSha256) {
@@ -279,13 +296,22 @@ function verifyAuthProfileMigrationTarget(receipt: AuthProfileMigrationSourceRec
}
}
if (receipt.expectedStateSha256) {
const row = executeSqliteQueryTakeFirstSync(
db,
kysely
.selectFrom("auth_profile_state")
.select("state_json")
.where("state_key", "=", "primary"),
);
const row =
targetStoreKey === "shared"
? executeSqliteQueryTakeFirstSync(
db,
getNodeSqliteKysely<SharedAuthProfileTargetDatabase>(db)
.selectFrom("auth_profile_state")
.select("state_json")
.where("store_key", "=", "shared"),
)
: executeSqliteQueryTakeFirstSync(
db,
getNodeSqliteKysely<AuthProfileTargetDatabase>(db)
.selectFrom("auth_profile_state")
.select("state_json")
.where("state_key", "=", "primary"),
);
const state = typeof row?.state_json === "string" ? JSON.parse(row.state_json) : null;
if (digestAuthProfileMigrationValue(state) !== receipt.expectedStateSha256) {
throw new Error("auth profile migration target verification failed");
@@ -346,7 +372,9 @@ export function resumePendingAuthProfileMigrationArchives(env?: NodeJS.ProcessEn
typeof row.source_record_count !== "number" ||
typeof report.archivePath !== "string" ||
typeof report.targetDatabasePath !== "string" ||
(row.target_table !== "auth_profile_store" && row.target_table !== "auth_profile_state")
(row.target_table !== "auth_profile_store" &&
row.target_table !== "auth_profile_stores" &&
row.target_table !== "auth_profile_state")
) {
throw new Error("invalid pending auth profile migration receipt");
}
@@ -359,6 +387,7 @@ export function resumePendingAuthProfileMigrationArchives(env?: NodeJS.ProcessEn
sourceRecordCount: row.source_record_count,
targetDatabasePath: report.targetDatabasePath,
targetTable: row.target_table,
targetStoreKey: report.targetStoreKey === "shared" ? "shared" : "primary",
archivePath: report.archivePath,
...(isRecordOfStrings(report.expectedProfileSha256)
? { expectedProfileSha256: report.expectedProfileSha256 }
+27 -1
View File
@@ -1,11 +1,16 @@
// Doctor auth hint tests cover OAuth refresh failure formatting and auth repair guidance.
import { beforeEach, describe, expect, it, vi } from "vitest";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { useAutoCleanupTempDirTracker } from "../../test/helpers/temp-dir.js";
import type { OpenClawConfig } from "../config/types.openclaw.js";
import { writeConfigMachineState } from "../state/config-machine-state.js";
import {
collectAuthProfileHealthFindings,
noteLegacyCodexProviderOverride,
noteSharedAuthStoreStatus,
} from "./doctor-auth.js";
const tempDirs = useAutoCleanupTempDirTracker(afterEach);
const mocks = vi.hoisted(() => ({
ensureAuthProfileStore: vi.fn(),
note: vi.fn(),
@@ -63,6 +68,27 @@ describe("doctor auth hints", () => {
);
});
it("reports the legacy shared auth owner with the migration command", () => {
noteSharedAuthStoreStatus({
...process.env,
OPENCLAW_STATE_DIR: tempDirs.make("openclaw-doctor-shared-auth-"),
});
expect(mocks.note).toHaveBeenCalledWith(
expect.stringContaining("openclaw doctor --fix"),
"Shared auth store",
);
mocks.note.mockClear();
const relocatedEnv = {
...process.env,
OPENCLAW_STATE_DIR: tempDirs.make("openclaw-doctor-relocated-auth-"),
};
writeConfigMachineState("auth.sharedStore", { location: "state-db" }, { env: relocatedEnv });
noteSharedAuthStoreStatus(relocatedEnv);
expect(mocks.note).not.toHaveBeenCalled();
});
it("collects legacy Codex override structured findings", async () => {
const findings = await collectAuthProfileHealthFindings({
cfg: doctorFixtureConfig({
+15 -1
View File
@@ -30,7 +30,10 @@ import {
formatOAuthRefreshFailureLoginCommandMarkdown,
type OAuthRefreshFailureReason,
} from "../agents/auth-profiles/oauth-refresh-failure.js";
import { resolveAuthStorePathForDisplay } from "../agents/auth-profiles/path-resolve.js";
import {
resolveAuthStorePathForDisplay,
resolveSharedAuthStoreOwnership,
} from "../agents/auth-profiles/path-resolve.js";
import { buildProviderAuthRecoveryHint } from "../agents/provider-auth-recovery-hint.js";
import type { OpenClawConfig } from "../config/types.openclaw.js";
import type { HealthFinding } from "../flows/health-checks.js";
@@ -49,6 +52,17 @@ const DOCTOR_REAUTH_PROVIDER_ALIASES: Readonly<Record<string, string>> = {
[LEGACY_CODEX_PROVIDER_ID]: OPENAI_PROVIDER_ID,
};
/** Surface the one-time relocation while the legacy shared owner is still active. */
export function noteSharedAuthStoreStatus(env: NodeJS.ProcessEnv = process.env): void {
if (resolveSharedAuthStoreOwnership(env).location !== "legacy-main") {
return;
}
note(
"Shared auth profiles still live in the main agent database. Run `openclaw doctor --fix` to move them into shared SQLite state and make the main agent deletable.",
"Shared auth store",
);
}
function hasConfiguredCodexOAuthProfile(cfg: OpenClawConfig): boolean {
return Object.values(cfg.auth?.profiles ?? {}).some(
(profile) =>
@@ -6,9 +6,12 @@ import { isRecord } from "@openclaw/normalization-core/record-coerce";
import { note } from "../../packages/terminal-core/src/note.js";
import { resolveDefaultAgentDir } from "../agents/agent-scope.js";
import { AUTH_STORE_VERSION } from "../agents/auth-profiles/constants.js";
import { resolveSharedAuthStorePath } from "../agents/auth-profiles/path-resolve.js";
import { mergeAuthProfileStores } from "../agents/auth-profiles/persisted.js";
import { loadPersistedAuthProfileStore } from "../agents/auth-profiles/persisted.js";
import {
loadPersistedAuthProfileStore,
loadPersistedSharedAuthProfileStore,
mergeAuthProfileStores,
} from "../agents/auth-profiles/persisted.js";
import { resolveSharedMainAuthAgentDir } from "../agents/auth-profiles/shared-main-dir.js";
import { updateAuthProfileStoreWithLock } from "../agents/auth-profiles/store.js";
import type { AuthProfileCredential, AuthProfileStore } from "../agents/auth-profiles/types.js";
import { isNonSecretApiKeyMarker } from "../agents/model-auth-markers.js";
@@ -126,7 +129,7 @@ function allocateProfileId(
}
async function persistCredentials(params: {
agentDir: string;
agentDir?: string;
blockedStores?: readonly AuthProfileStore[];
credentials: readonly PlaintextCredential[];
inheritedStore?: AuthProfileStore;
@@ -169,7 +172,12 @@ async function persistCredentials(params: {
if (!updated) {
throw new Error("auth profile store could not be updated");
}
const persisted = loadPersistedAuthProfileStore(params.agentDir);
const persisted = params.agentDir
? loadPersistedAuthProfileStore(params.agentDir)
: loadPersistedSharedAuthProfileStore({
...process.env,
OPENCLAW_STATE_DIR: params.stateDir,
});
const effectivePersisted = params.inheritedStore
? mergeAuthProfileStores(params.inheritedStore, persisted ?? emptyStore())
: persisted;
@@ -230,17 +238,17 @@ export async function maybeMigrateModelCatalogCredentials(params: {
const warnings: string[] = [];
const env = params.env ?? process.env;
const stateDir = resolveStateDir(env);
const mainAgentDir = path.dirname(resolveSharedAuthStorePath(env));
const mainAgentDir = resolveSharedMainAuthAgentDir(env);
const discoveredAgentDirs = listAgentModelsJsonPaths(params.cfg, stateDir, env).map(
(modelsPath) => path.dirname(modelsPath),
);
const agentDirs = [
...new Set([mainAgentDir, resolveDefaultAgentDir(params.cfg, env), ...discoveredAgentDirs]),
];
const mainStore = loadPersistedAuthProfileStore(mainAgentDir) ?? emptyStore();
const mainStore = loadPersistedSharedAuthProfileStore(env) ?? emptyStore();
const catalogs = agentDirs.map((agentDir) => collectAgentCatalogs(agentDir, warnings));
const effectiveStores = catalogs.map(({ agentDir, localStore }) =>
agentDir === mainAgentDir ? mainStore : mergeAuthProfileStores(mainStore, localStore),
const effectiveStores = catalogs.map(({ localStore }) =>
mergeAuthProfileStores(mainStore, localStore),
);
const childStores = catalogs
.filter((catalog) => catalog.agentDir !== mainAgentDir)
@@ -284,7 +292,6 @@ export async function maybeMigrateModelCatalogCredentials(params: {
let migrated = 0;
try {
migrated += await persistCredentials({
agentDir: mainAgentDir,
blockedStores: childStores,
credentials: configCredentials,
stateDir,
@@ -295,11 +302,11 @@ export async function maybeMigrateModelCatalogCredentials(params: {
params.runtime.error(warning);
}
const migratedMainStore = loadPersistedAuthProfileStore(mainAgentDir) ?? mainStore;
const migratedMainStore = loadPersistedSharedAuthProfileStore(env) ?? mainStore;
for (const [index, catalog] of catalogs.entries()) {
try {
migrated += await persistCredentials({
agentDir: catalog.agentDir,
...(catalog.agentDir === mainAgentDir ? {} : { agentDir: catalog.agentDir }),
credentials: catalogCredentials[index] ?? [],
...(catalog.agentDir === mainAgentDir ? {} : { inheritedStore: migratedMainStore }),
stateDir,
@@ -6,6 +6,12 @@ import type { DatabaseSync } from "node:sqlite";
import { gunzipSync, gzipSync } from "node:zlib";
import { afterEach, beforeAll, describe, expect, it, vi } from "vitest";
import { useAutoCleanupTempDirTracker } from "../../test/helpers/temp-dir.js";
import { resolveSharedMainAuthAgentDir } from "../agents/auth-profiles/shared-main-dir.js";
import {
readPersistedAuthProfileStoreRaw,
readPersistedSharedAuthProfileStoreRaw,
writePersistedAuthProfileStoreRaw,
} from "../agents/auth-profiles/sqlite.js";
import type { OpenClawConfig } from "../config/config.js";
import type { SessionEntry } from "../config/sessions/types.js";
import { requireNodeSqlite } from "../infra/node-sqlite.js";
@@ -24,6 +30,7 @@ import {
} from "../plugins/installed-plugin-index-store.js";
import type { InstalledPluginInstallRecordInfo } from "../plugins/installed-plugin-index.js";
import { EMPTY_LEGACY_SESSION_SURFACES } from "../plugins/legacy-session-surfaces.types.js";
import { closeOpenClawAgentDatabasesForTest } from "../state/openclaw-agent-db.js";
import {
closeOpenClawStateDatabaseForTest,
openOpenClawStateDatabase,
@@ -250,6 +257,7 @@ afterEach(() => {
resetAutoMigrateLegacyStateDirForTest();
resetAutoMigrateLegacyTaskStateSidecarsForTest();
closeOpenClawStateDatabaseForTest();
closeOpenClawAgentDatabasesForTest();
setMaxPluginStateEntriesPerPluginForTests();
resetPluginStateStoreForTests();
mockedChannelMigrationPlans.plans = [];
@@ -915,6 +923,39 @@ describe("doctor legacy state migrations", () => {
expect(store["agent:main:subagent:xyz"]?.sessionId).toBe("e");
});
it("routes shared auth relocation through the doctor-only migration plan", async () => {
const stateDir = makeDoctorStateDir();
const env = { ...process.env, OPENCLAW_STATE_DIR: stateDir };
const mainAgentDir = resolveSharedMainAuthAgentDir(env);
const store = {
version: 1,
profiles: {
"openai:default": { type: "api_key" as const, provider: "openai", key: "secret" },
},
};
writePersistedAuthProfileStoreRaw(store, mainAgentDir);
const detected = await detectLegacyStateMigrations({
cfg: {},
env,
doctorOnlyStateMigrations: true,
});
expect(detected.sharedAuthStore.hasLegacy).toBe(true);
expect(detected.preview).toContain(
"- Shared auth store: legacy main-agent rows → shared SQLite state",
);
const result = await autoMigrateLegacyState({
cfg: {},
env,
doctorOnlyStateMigrations: true,
});
expect(result.warnings).toEqual([]);
expect(result.changes).toContain("Relocated shared auth profiles into shared SQLite state.");
expect(readPersistedSharedAuthProfileStoreRaw(env)).toEqual(store);
expect(readPersistedAuthProfileStoreRaw(mainAgentDir)).toBeNull();
});
it("removes stale transcript paths left by a shipped legacy migration", async () => {
const root = makeDoctorStateDir();
const legacyDir = path.join(root, "sessions");
+4 -1
View File
@@ -206,7 +206,6 @@ function createLegacyStateMigrationDetectionResult(params?: {
return {
targetAgentId: "main",
targetMainKey: "main",
targetScope: undefined,
stateDir: "/tmp/state",
oauthDir: "/tmp/oauth",
deviceAuth: {
@@ -267,6 +266,10 @@ function createLegacyStateMigrationDetectionResult(params?: {
hasLegacy: false,
preview: [],
},
sharedAuthStore: {
sourcePath: "/tmp/state/agents/main/agent/openclaw-agent.sqlite",
hasLegacy: false,
},
worktrees: { hasLegacy: false, pathRewrites: [] },
taskStateSidecars: {
taskRunsPath: "/tmp/state/tasks/runs.sqlite",
@@ -1,5 +1,4 @@
import fs from "node:fs";
import path from "node:path";
import { isRecord } from "@openclaw/normalization-core/record-coerce";
import { normalizeOptionalLowercaseString as normalizeString } from "@openclaw/normalization-core/string-coerce";
import { resolveAgentDir } from "../../../agents/agent-scope.js";
@@ -7,9 +6,9 @@ import {
areOAuthCredentialsEquivalent,
hasMatchingOAuthIdentity,
} from "../../../agents/auth-profiles/oauth-shared.js";
import { resolveSharedAuthStorePath } from "../../../agents/auth-profiles/path-resolve.js";
import {
loadPersistedAuthProfileStore,
loadPersistedSharedAuthProfileStore,
parseLegacyCredentialEntry,
} from "../../../agents/auth-profiles/persisted.js";
import {
@@ -332,9 +331,7 @@ function resolveVerifiedSessionAuthProfileIdMap(params: {
}
const agentDir = resolveAgentDir(params.cfg, params.agentId, params.env);
const localProfiles = loadPersistedAuthProfileStore(agentDir)?.profiles ?? {};
const mainProfiles =
loadPersistedAuthProfileStore(path.dirname(resolveSharedAuthStorePath(params.env)))?.profiles ??
{};
const mainProfiles = loadPersistedSharedAuthProfileStore(params.env)?.profiles ?? {};
const localLegacyAuthPath = resolveLegacyAuthProfilesPath(agentDir);
const localLegacySourceExists = fs.existsSync(localLegacyAuthPath);
const localLegacySource = localLegacySourceExists
@@ -0,0 +1,162 @@
import fs from "node:fs";
import path from "node:path";
import { coercePersistedAuthProfileStore } from "../../../agents/auth-profiles/persisted.js";
import {
inspectPersistedAuthProfileStateRaw,
inspectPersistedAuthProfileStoreRaw,
inspectPersistedSharedAuthProfileStateRaw,
inspectPersistedSharedAuthProfileStoreRaw,
resolveAuthProfileDatabaseFilePaths,
} from "../../../agents/auth-profiles/sqlite.js";
import {
coerceAuthProfileState,
mergeAuthProfileState,
} from "../../../agents/auth-profiles/state.js";
import type { AuthProfileStore } from "../../../agents/auth-profiles/types.js";
import { isRecord } from "../../../utils.js";
import {
resolveLegacyAuthProfilesPath,
resolveLegacyAuthStatePath,
resolveLegacyFlatAuthPath,
} from "../../doctor-auth-legacy-paths.js";
function inspectAuthPath(pathname: string): "present" | "missing" | "unreadable" {
try {
fs.statSync(pathname);
return "present";
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== "ENOENT") {
return "unreadable";
}
}
try {
fs.lstatSync(pathname);
return "unreadable";
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== "ENOENT") {
return "unreadable";
}
}
// Accept ENOENT only when no broken symlink or non-directory ancestor masks the source.
let ancestor = path.dirname(pathname);
while (true) {
try {
const stat = fs.lstatSync(ancestor);
if (!stat.isSymbolicLink()) {
return stat.isDirectory() ? "missing" : "unreadable";
}
try {
return fs.statSync(ancestor).isDirectory() ? "missing" : "unreadable";
} catch {
return "unreadable";
}
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== "ENOENT") {
return "unreadable";
}
}
const parent = path.dirname(ancestor);
if (parent === ancestor) {
return "missing";
}
ancestor = parent;
}
}
export function inspectUnmigratedAuthStoreSources(
agentDir: string,
): "present" | "missing" | "unreadable" {
const results = new Set(
[
resolveLegacyAuthProfilesPath(agentDir),
resolveLegacyAuthStatePath(agentDir),
resolveLegacyFlatAuthPath(agentDir),
].map((pathname) => inspectAuthPath(pathname)),
);
if (results.has("unreadable")) {
return "unreadable";
}
return results.has("present") ? "present" : "missing";
}
export function inspectAuthDatabaseFiles(agentDir: string): "present" | "missing" | "unreadable" {
const [databasePath, ...sidecarPaths] = resolveAuthProfileDatabaseFilePaths(agentDir);
if (!databasePath) {
return "unreadable";
}
const availability = inspectAuthPath(databasePath);
const sidecarAvailability = sidecarPaths.map((pathname) => inspectAuthPath(pathname));
if (
availability === "unreadable" ||
sidecarAvailability.some((status) => status === "unreadable")
) {
return "unreadable";
}
if (availability === "present") {
return "present";
}
return sidecarAvailability.every((sidecar) => sidecar === "missing") ? "missing" : "unreadable";
}
export function loadCompletePersistedStore(
agentDir?: string,
env: NodeJS.ProcessEnv = process.env,
):
| { status: "ok"; store: AuthProfileStore | null; hasAuthTables: boolean }
| { status: "invalid" } {
const inspection = agentDir
? inspectPersistedAuthProfileStoreRaw(agentDir)
: inspectPersistedSharedAuthProfileStoreRaw(env);
const stateInspection = agentDir
? inspectPersistedAuthProfileStateRaw(agentDir)
: inspectPersistedSharedAuthProfileStateRaw(env);
if (inspection.status === "unreadable" || stateInspection.status === "unreadable") {
return { status: "invalid" };
}
const storeMissingReason = inspection.status === "missing" ? inspection.reason : undefined;
const stateMissingReason =
stateInspection.status === "missing" ? stateInspection.reason : undefined;
if (storeMissingReason === "database" || stateMissingReason === "database") {
return storeMissingReason === "database" && stateMissingReason === "database"
? { status: "ok", store: null, hasAuthTables: false }
: { status: "invalid" };
}
if ((storeMissingReason === "table") !== (stateMissingReason === "table")) {
return { status: "invalid" };
}
if (storeMissingReason === "table") {
return { status: "ok", store: null, hasAuthTables: false };
}
const persistedState =
stateInspection.status === "readable" ? coerceAuthProfileState(stateInspection.raw) : {};
if (inspection.status === "missing") {
return stateInspection.status === "missing"
? { status: "ok", store: null, hasAuthTables: true }
: {
status: "ok",
store: { version: 1, profiles: {}, ...persistedState },
hasAuthTables: true,
};
}
if (!isRecord(inspection.raw) || !isRecord(inspection.raw.profiles)) {
return { status: "invalid" };
}
const store = coercePersistedAuthProfileStore(inspection.raw);
const rawProfileIds = Object.keys(inspection.raw.profiles);
if (
!store ||
rawProfileIds.length !== Object.keys(store.profiles).length ||
rawProfileIds.some((profileId) => !Object.hasOwn(store.profiles, profileId))
) {
return { status: "invalid" };
}
return {
status: "ok",
store: {
...store,
...mergeAuthProfileState(coerceAuthProfileState(inspection.raw), persistedState),
},
hasAuthTables: true,
};
}
+61 -164
View File
@@ -7,22 +7,16 @@ import {
resolveAuthProfileEligibility,
resolveAuthProfileOrder,
} from "../../../agents/auth-profiles/order.js";
import { resolveSharedAuthStorePath } from "../../../agents/auth-profiles/path-resolve.js";
import {
coercePersistedAuthProfileStore,
mergeAuthProfileStores,
} from "../../../agents/auth-profiles/persisted.js";
resolveSharedAuthStoreOwnership,
resolveSharedAuthStorePath,
} from "../../../agents/auth-profiles/path-resolve.js";
import { mergeAuthProfileStores } from "../../../agents/auth-profiles/persisted.js";
import { resolveSharedMainAuthAgentDir } from "../../../agents/auth-profiles/shared-main-dir.js";
import {
inspectPersistedAuthProfileStateRaw,
inspectPersistedAuthProfileStoreRaw,
resolveAuthProfileDatabaseOwnerId,
resolveAuthProfileDatabasePath,
resolveAuthProfileDatabaseFilePaths,
} from "../../../agents/auth-profiles/sqlite.js";
import {
coerceAuthProfileState,
mergeAuthProfileState,
} from "../../../agents/auth-profiles/state.js";
import type { AuthProfileStore } from "../../../agents/auth-profiles/types.js";
import { resolveProviderIdForAuth } from "../../../agents/provider-auth-aliases.js";
import { resolveStateDir } from "../../../config/paths.js";
@@ -34,10 +28,10 @@ import {
} from "../../../state/openclaw-agent-db.js";
import { isRecord, resolveUserPath } from "../../../utils.js";
import {
resolveLegacyAuthProfilesPath as resolveAuthStorePath,
resolveLegacyAuthStatePath as resolveAuthStatePath,
resolveLegacyFlatAuthPath as resolveLegacyAuthStorePath,
} from "../../doctor-auth-legacy-paths.js";
inspectAuthDatabaseFiles,
inspectUnmigratedAuthStoreSources,
loadCompletePersistedStore,
} from "./stale-auth-order-store.js";
type StaleConfiguredAuthOrder = {
provider: string;
@@ -104,146 +98,6 @@ function hasNonemptyConfiguredAuthOrder(cfg: OpenClawConfig): boolean {
return Boolean(order && Object.values(order).some((profileIds) => profileIds.length > 0));
}
function inspectAuthPath(pathname: string): "present" | "missing" | "unreadable" {
try {
fs.statSync(pathname);
return "present";
} catch (error) {
const code = (error as NodeJS.ErrnoException).code;
if (code !== "ENOENT") {
return "unreadable";
}
}
try {
// A dangling final symlink is unavailable state, not a stale registry row.
fs.lstatSync(pathname);
return "unreadable";
} catch (error) {
const code = (error as NodeJS.ErrnoException).code;
if (code !== "ENOENT") {
return "unreadable";
}
}
// Accept ENOENT only when the missing suffix has no broken symlink or
// non-directory ancestor masking an unavailable auth source.
let ancestor = path.dirname(pathname);
while (true) {
try {
const stat = fs.lstatSync(ancestor);
if (!stat.isSymbolicLink()) {
return stat.isDirectory() ? "missing" : "unreadable";
}
try {
return fs.statSync(ancestor).isDirectory() ? "missing" : "unreadable";
} catch {
return "unreadable";
}
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== "ENOENT") {
return "unreadable";
}
}
const parent = path.dirname(ancestor);
if (parent === ancestor) {
return "missing";
}
ancestor = parent;
}
}
function inspectUnmigratedAuthStoreSources(agentDir: string): "present" | "missing" | "unreadable" {
const results = new Set(
[
resolveAuthStorePath(agentDir),
resolveAuthStatePath(agentDir),
resolveLegacyAuthStorePath(agentDir),
].map((pathname) => inspectAuthPath(pathname)),
);
if (results.has("unreadable")) {
return "unreadable";
}
return results.has("present") ? "present" : "missing";
}
function inspectAuthDatabaseFiles(agentDir: string): "present" | "missing" | "unreadable" {
const [databasePath, ...sidecarPaths] = resolveAuthProfileDatabaseFilePaths(agentDir);
if (!databasePath) {
return "unreadable";
}
const availability = inspectAuthPath(databasePath);
const sidecarAvailability = sidecarPaths.map((pathname) => inspectAuthPath(pathname));
if (
availability === "unreadable" ||
sidecarAvailability.some((status) => status === "unreadable")
) {
return "unreadable";
}
if (availability === "present") {
return "present";
}
return sidecarAvailability.every((sidecar) => sidecar === "missing") ? "missing" : "unreadable";
}
function loadCompletePersistedStore(
agentDir: string,
):
| { status: "ok"; store: AuthProfileStore | null; hasAuthTables: boolean }
| { status: "invalid" } {
const inspection = inspectPersistedAuthProfileStoreRaw(agentDir);
const stateInspection = inspectPersistedAuthProfileStateRaw(agentDir);
if (inspection.status === "unreadable" || stateInspection.status === "unreadable") {
return { status: "invalid" };
}
const storeMissingReason = inspection.status === "missing" ? inspection.reason : undefined;
const stateMissingReason =
stateInspection.status === "missing" ? stateInspection.reason : undefined;
if (storeMissingReason === "database" || stateMissingReason === "database") {
return storeMissingReason === "database" && stateMissingReason === "database"
? { status: "ok", store: null, hasAuthTables: false }
: { status: "invalid" };
}
if ((storeMissingReason === "table") !== (stateMissingReason === "table")) {
return { status: "invalid" };
}
if (storeMissingReason === "table") {
return { status: "ok", store: null, hasAuthTables: false };
}
const persistedState =
stateInspection.status === "readable" ? coerceAuthProfileState(stateInspection.raw) : {};
if (inspection.status === "missing") {
return stateInspection.status === "missing"
? { status: "ok", store: null, hasAuthTables: true }
: {
status: "ok",
store: { version: 1, profiles: {}, ...persistedState },
hasAuthTables: true,
};
}
if (!isRecord(inspection.raw) || !isRecord(inspection.raw.profiles)) {
return { status: "invalid" };
}
const store = coercePersistedAuthProfileStore(inspection.raw);
const rawProfileIds = Object.keys(inspection.raw.profiles);
if (
!store ||
rawProfileIds.length !== Object.keys(store.profiles).length ||
rawProfileIds.some((profileId) => !Object.hasOwn(store.profiles, profileId))
) {
// Coercion deliberately drops malformed credentials. A dropped id may be
// the user's explicit selection, so doctor must not infer that it vanished.
return { status: "invalid" };
}
return {
status: "ok",
store: {
...store,
...mergeAuthProfileState(coerceAuthProfileState(inspection.raw), persistedState),
},
hasAuthTables: true,
};
}
function listRetainedStateAgentDirs(env: NodeJS.ProcessEnv): string[] | null {
const agentsRoot = path.join(resolveStateDir(env), "agents");
let entries: fs.Dirent[];
@@ -295,9 +149,11 @@ function loadConfiguredAgentAuthStores(
if (!order || !hasValidConfiguredAuthProfiles(cfg)) {
return undefined;
}
// Every secondary agent inherits the legacy main store at runtime, even when
// `agents.list` names a different default agent.
const mainAgentDir = path.dirname(resolveSharedAuthStorePath(env));
// Every agent inherits the shared store at runtime, even when `agents.list`
// names a different default agent.
const mainAgentDir = resolveSharedMainAuthAgentDir(env);
const sharedDatabasePath = path.resolve(resolveSharedAuthStorePath(env));
const sharedOwnership = resolveSharedAuthStoreOwnership(env);
const activeAgentDirs = new Set<string>();
const expectedAgentIdsByDir = new Map<string, Set<string>>();
const addExpectedAgentDir = (agentDir: string, agentId: string) => {
@@ -305,7 +161,9 @@ function loadConfiguredAgentAuthStores(
owners.add(normalizeAgentId(agentId));
expectedAgentIdsByDir.set(agentDir, owners);
};
addExpectedAgentDir(mainAgentDir, resolveAuthProfileDatabaseOwnerId(mainAgentDir));
if (sharedOwnership.location === "legacy-main") {
addExpectedAgentDir(mainAgentDir, resolveAuthProfileDatabaseOwnerId(mainAgentDir));
}
for (const agentId of listAgentIds(cfg)) {
const agentDir = path.resolve(resolveAgentDir(cfg, agentId, env));
activeAgentDirs.add(agentDir);
@@ -325,10 +183,43 @@ function loadConfiguredAgentAuthStores(
const agentDirs = new Set([mainAgentDir, ...activeAgentDirs, ...retainedAgentDirs]);
const entries: Array<{
agentDir: string;
agentDir?: string;
databasePath: string;
store: AuthProfileStore | null;
isShared: boolean;
}> = [];
const sharedLegacyAvailability = inspectUnmigratedAuthStoreSources(mainAgentDir);
if (sharedLegacyAvailability === "unreadable") {
return { status: "blocked", warnings: [INVALID_SQLITE_STORE_WARNING] };
}
if (sharedLegacyAvailability === "present") {
return undefined;
}
const sharedLoaded = loadCompletePersistedStore(undefined, env);
if (sharedLoaded.status === "invalid") {
return { status: "blocked", warnings: [INVALID_SQLITE_STORE_WARNING] };
}
if (sharedOwnership.location === "legacy-main") {
const availability = inspectAuthDatabaseFiles(mainAgentDir);
const expectedAgentIds = expectedAgentIdsByDir.get(mainAgentDir);
const owner =
availability === "present"
? inspectOpenClawAgentDatabaseOwner(sharedDatabasePath)
: undefined;
if (
availability === "unreadable" ||
owner?.status === "unreadable" ||
(expectedAgentIds && owner?.status === "owned" && !expectedAgentIds.has(owner.agentId)) ||
(owner?.status === "unowned" && sharedLoaded.hasAuthTables)
) {
return { status: "blocked", warnings: [INVALID_SQLITE_STORE_WARNING] };
}
}
entries.push({
databasePath: sharedDatabasePath,
store: sharedLoaded.store,
isShared: true,
});
for (const agentDir of agentDirs) {
const expectedAgentIds = expectedAgentIdsByDir.get(agentDir);
if (expectedAgentIds && expectedAgentIds.size !== 1) {
@@ -342,6 +233,9 @@ function loadConfiguredAgentAuthStores(
return undefined;
}
const databasePath = path.resolve(resolveAuthProfileDatabasePath(agentDir));
if (databasePath === sharedDatabasePath) {
continue;
}
const availability = inspectAuthDatabaseFiles(agentDir);
if (availability === "unreadable") {
return { status: "blocked", warnings: [INVALID_SQLITE_STORE_WARNING] };
@@ -363,7 +257,7 @@ function loadConfiguredAgentAuthStores(
if (owner?.status === "unowned" && loaded.hasAuthTables) {
return { status: "blocked", warnings: [INVALID_SQLITE_STORE_WARNING] };
}
entries.push({ agentDir, databasePath, store: loaded.store });
entries.push({ agentDir, databasePath, store: loaded.store, isShared: false });
}
let registeredDatabases: Array<{ agentId: string; path: string }>;
@@ -429,17 +323,20 @@ function loadConfiguredAgentAuthStores(
}
const emptyStore: AuthProfileStore = { version: 1, profiles: {} };
const mainStore = entries.find((entry) => entry.agentDir === mainAgentDir)?.store ?? emptyStore;
const mainStore = entries.find((entry) => entry.isShared)?.store ?? emptyStore;
const agentStores = entries.map((entry) => {
const localStore = entry.store ?? emptyStore;
return entry.agentDir === mainAgentDir
return entry.isShared
? mainStore
: mergeAuthProfileStores(mainStore, localStore, {
preserveBaseRuntimeExternalProfiles: true,
});
});
const activeStores = entries.flatMap((entry, index) =>
activeAgentDirs.has(entry.agentDir) ? [agentStores[index] ?? emptyStore] : [],
(entry.isShared && activeAgentDirs.has(mainAgentDir)) ||
(entry.agentDir !== undefined && activeAgentDirs.has(entry.agentDir))
? [agentStores[index] ?? emptyStore]
: [],
);
const stores = [
...agentStores,
@@ -13,8 +13,11 @@ import {
areOAuthCredentialsEquivalent,
isSafeToAdoptMainStoreOAuthIdentity,
} from "../../../agents/auth-profiles/oauth-shared.js";
import { resolveSharedAuthStorePath } from "../../../agents/auth-profiles/path-resolve.js";
import { loadPersistedAuthProfileStore } from "../../../agents/auth-profiles/persisted.js";
import {
loadPersistedAuthProfileStore,
loadPersistedSharedAuthProfileStore,
} from "../../../agents/auth-profiles/persisted.js";
import { resolveSharedMainAuthAgentDir } from "../../../agents/auth-profiles/shared-main-dir.js";
import { updateAuthProfileStoreWithLock } from "../../../agents/auth-profiles/store.js";
import type { AuthProfileStore, OAuthCredential } from "../../../agents/auth-profiles/types.js";
import { resolveStateDir } from "../../../config/paths.js";
@@ -113,9 +116,8 @@ export async function scanStaleOAuthProfileShadows(params: {
}): Promise<StaleOAuthProfileShadow[]> {
const env = params.env ?? process.env;
const now = params.now ?? Date.now();
const mainAgentDir = path.dirname(resolveSharedAuthStorePath(env));
const mainAuthPath = path.resolve(resolveAuthStorePath(mainAgentDir));
const mainStore = loadPersistedAuthProfileStore(mainAgentDir);
const mainAuthPath = path.resolve(resolveAuthStorePath(resolveSharedMainAuthAgentDir(env)));
const mainStore = loadPersistedSharedAuthProfileStore(env);
if (!mainStore) {
return [];
}
@@ -294,7 +296,7 @@ export async function repairStaleOAuthProfileShadows(params: {
byAgentDir.set(hit.agentDir, existing);
}
for (const [agentDir, agentHits] of byAgentDir) {
const mainStore = loadPersistedAuthProfileStore(path.dirname(resolveSharedAuthStorePath(env)));
const mainStore = loadPersistedSharedAuthProfileStore(env);
if (!mainStore) {
continue;
}
@@ -50,6 +50,7 @@ const mocks = vi.hoisted(() => ({
collectAuthProfileHealthFindings: vi.fn(async () => []),
noteAuthProfileHealth: vi.fn().mockResolvedValue(undefined),
noteLegacyCodexProviderOverride: vi.fn(),
noteSharedAuthStoreStatus: vi.fn(),
noteMemorySearchHealth: vi.fn().mockResolvedValue(undefined),
noteWebFetchProxyDiagnostic: vi.fn().mockResolvedValue(undefined),
buildGatewayConnectionDetails: vi.fn(() => ({ message: "gateway details" })),
@@ -306,6 +307,7 @@ vi.mock("../commands/doctor-auth.js", () => ({
collectAuthProfileHealthFindings: mocks.collectAuthProfileHealthFindings,
noteAuthProfileHealth: mocks.noteAuthProfileHealth,
noteLegacyCodexProviderOverride: mocks.noteLegacyCodexProviderOverride,
noteSharedAuthStoreStatus: mocks.noteSharedAuthStoreStatus,
}));
vi.mock("../commands/doctor-memory-search.js", () => ({
@@ -629,6 +631,7 @@ describe("doctor health contributions", () => {
mocks.noteAuthProfileHealth.mockClear();
mocks.noteAuthProfileHealth.mockResolvedValue(undefined);
mocks.noteLegacyCodexProviderOverride.mockClear();
mocks.noteSharedAuthStoreStatus.mockClear();
mocks.noteMemorySearchHealth.mockClear();
mocks.noteMemorySearchHealth.mockResolvedValue(undefined);
mocks.noteWebFetchProxyDiagnostic.mockClear();
+2 -1
View File
@@ -64,7 +64,7 @@ async function runAuthProfileHealth(ctx: DoctorHealthFlowContext): Promise<void>
await import("../commands/doctor-auth-oauth-sidecar.js");
const { maybeMigrateLegacyPluginModelCatalogs } =
await import("../commands/doctor-plugin-model-catalog.js");
const { noteAuthProfileHealth, noteLegacyCodexProviderOverride } =
const { noteAuthProfileHealth, noteLegacyCodexProviderOverride, noteSharedAuthStoreStatus } =
await import("../commands/doctor-auth.js");
const { buildGatewayConnectionDetails } = await import("../gateway/call.js");
const { note } = await loadNoteModule();
@@ -98,6 +98,7 @@ async function runAuthProfileHealth(ctx: DoctorHealthFlowContext): Promise<void>
allowKeychainPrompt: ctx.options.nonInteractive !== true && process.stdin.isTTY,
});
noteLegacyCodexProviderOverride(ctx.cfg);
noteSharedAuthStoreStatus(ctx.env);
ctx.gatewayDetails = buildGatewayConnectionDetails({ config: ctx.cfg });
if (ctx.gatewayDetails.remoteFallbackNote) {
note(ctx.gatewayDetails.remoteFallbackNote, "Gateway");
@@ -13,6 +13,9 @@ import { FsSafeError } from "../../infra/fs-safe.js";
/* ------------------------------------------------------------------ */
const mocks = vi.hoisted(() => ({
sharedAuthStoreOwnership: { location: "legacy-main" } as {
location: "legacy-main" | "state-db";
},
loadConfigReturn: {} as Record<string, unknown>,
listAgentEntries: vi.fn((_cfg?: unknown) => [] as Array<Record<string, unknown>>),
findAgentEntryIndex: vi.fn((_list?: unknown, _agentId?: string) => -1),
@@ -178,7 +181,7 @@ vi.mock("../../agents/auth-profiles/path-resolve.js", async () => ({
...(await vi.importActual<typeof import("../../agents/auth-profiles/path-resolve.js")>(
"../../agents/auth-profiles/path-resolve.js",
)),
resolveSharedAuthStoreOwnership: () => ({ location: "legacy-main" }),
resolveSharedAuthStoreOwnership: () => mocks.sharedAuthStoreOwnership,
resolveSharedAuthStorePath: () => "/resolved/agents/main/agent/openclaw-agent.sqlite",
}));
@@ -360,6 +363,7 @@ const { testing: agentsTesting, agentsHandlers } = await import("./agents.js");
beforeEach(() => {
agentsTesting.resetDepsForTests();
mocks.omitConfigMutationResult = false;
mocks.sharedAuthStoreOwnership = { location: "legacy-main" };
mocks.withAgentExecApprovalsRemoved
.mockReset()
.mockImplementation(async (_agentId: string, commit: () => Promise<unknown>) => await commit());
@@ -2942,6 +2946,23 @@ describe("agents.delete", () => {
expect(mocks.writeConfigFile).not.toHaveBeenCalled();
});
it("deletes main through the normal journal path after shared auth relocation", async () => {
mocks.sharedAuthStoreOwnership = { location: "state-db" };
mocks.loadConfigReturn = {
agents: { list: [{ id: "main" }, { id: "ops", default: true }] },
};
const { respond, promise } = makeCall("agents.delete", {
agentId: "main",
});
await promise;
expectRespondOk(respond, { ok: true, agentId: "main" });
expect(mocks.beginAgentDeletionCommit).toHaveBeenCalledOnce();
expect(mocks.beginAgentDeletionFinish).toHaveBeenCalledOnce();
expect(mocks.writeConfigFile).toHaveBeenCalledOnce();
});
it("returns not found when a concurrent delete wins the delete race", async () => {
let findCallCount = 0;
mocks.findAgentEntryIndex.mockImplementation(() => {
+19
View File
@@ -1,6 +1,7 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { resolveSharedMainAuthAgentDir } from "../agents/auth-profiles/shared-main-dir.js";
import {
discardLegacyRegistryWorktrees,
hasLegacyRegistryWorktrees,
@@ -136,6 +137,10 @@ import {
resolveSessionStoreOwnership,
type SessionStoreOwnership,
} from "./state-migrations.session-store.js";
import {
detectSharedAuthStoreMigration,
migrateSharedAuthStore,
} from "./state-migrations.shared-auth-store.js";
import {
autoMigrateLegacyStateDir,
migrateLegacyProfileWorkspace,
@@ -524,6 +529,13 @@ export async function detectLegacyStateMigrations(params: {
const managedOutgoingImages = detectDoctorOwnedState(detectLegacyManagedOutgoingImages);
const apns = detectDoctorOwnedState(detectLegacyApnsRegistrations);
const deviceAuth = detectDoctorOwnedState(detectLegacyDeviceAuth);
const sharedAuthStore =
stateSchemaMigrations.length === 0
? detectDoctorOwnedState(detectSharedAuthStoreMigration)
: {
sourcePath: path.join(resolveSharedMainAuthAgentDir(env), "openclaw-agent.sqlite"),
hasLegacy: false,
};
const deviceIdentity = detectLegacyDeviceIdentity({
stateDir,
env,
@@ -686,6 +698,10 @@ export async function detectLegacyStateMigrations(params: {
preview.push(`- Task flow sidecar: finish archive cleanup for ${flowRunsSidecarPath}`);
}
const stateMigrationPreviews: Array<readonly [hasLegacy: boolean, message: string]> = [
[
sharedAuthStore.hasLegacy,
"- Shared auth store: legacy main-agent rows → shared SQLite state",
],
[hasDeliveryQueues, "- Delivery queues: legacy JSON queue files → shared SQLite state"],
[hasVoiceWake, "- Voice Wake settings: legacy JSON files → shared SQLite state"],
[hasUpdateCheck, "- Update-check state: legacy JSON file → shared SQLite state"],
@@ -790,6 +806,7 @@ export async function detectLegacyStateMigrations(params: {
hasLegacy: stateSchemaMigrations.length > 0,
preview: stateSchemaMigrations.map((migration) => migration.path),
},
sharedAuthStore,
worktrees,
taskStateSidecars: {
taskRunsPath: taskRunsSidecarPath,
@@ -1097,6 +1114,7 @@ function buildLegacyStateMigrationSteps(
];
const sharedSteps: LegacyStateMigrationStep[] = [
ownerStep(detected.sharedAuthStore, migrateSharedAuthStore, "shared"),
sharedStep(() => migrateLegacyPluginStateSidecar({ stateDir })),
sharedStep(() => migrateLegacyInstalledPluginIndex({ stateDir }), true),
ownerStep(
@@ -1496,6 +1514,7 @@ export async function autoMigrateLegacyState(params: {
!detected.pluginInstallIndex.hasLegacy &&
!detected.debugProxyCaptureSidecar.hasLegacy &&
!detected.stateSchema.hasLegacy &&
!detected.sharedAuthStore.hasLegacy &&
!detected.worktrees.hasLegacy &&
detected.worktrees.pathRewrites.length === 0 &&
!detected.taskStateSidecars.hasLegacy &&
@@ -0,0 +1,317 @@
import { createHash } from "node:crypto";
import fs from "node:fs";
import path from "node:path";
import { DatabaseSync } from "node:sqlite";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { useAutoCleanupTempDirTracker } from "../../test/helpers/temp-dir.js";
const tempDirs = useAutoCleanupTempDirTracker(afterEach);
function makeStore(profileId: string, key: string) {
return {
version: 1,
profiles: {
[profileId]: { type: "api_key", provider: "openai", key },
},
};
}
describe("shared auth store relocation", () => {
beforeEach(() => {
vi.resetModules();
});
afterEach(async () => {
const [{ closeOpenClawAgentDatabasesForTest }, { closeOpenClawStateDatabaseForTest }] =
await Promise.all([
import("../state/openclaw-agent-db.js"),
import("../state/openclaw-state-db.js"),
]);
closeOpenClawAgentDatabasesForTest();
closeOpenClawStateDatabaseForTest();
vi.unstubAllEnvs();
});
async function createFixture() {
const stateDir = tempDirs.make("openclaw-shared-auth-relocate-");
vi.stubEnv("OPENCLAW_STATE_DIR", stateDir);
vi.stubEnv("OPENCLAW_AGENT_DIR", "");
const env = { ...process.env, OPENCLAW_STATE_DIR: stateDir, OPENCLAW_AGENT_DIR: undefined };
const [paths, ownership, sqlite, storeModule, persisted, authState, migration, stateDb] =
await Promise.all([
import("../agents/auth-profiles/shared-main-dir.js"),
import("../agents/auth-profiles/path-resolve.js"),
import("../agents/auth-profiles/sqlite.js"),
import("../agents/auth-profiles/store.js"),
import("../agents/auth-profiles/persisted.js"),
import("../agents/auth-profiles/state.js"),
import("./state-migrations.shared-auth-store.js"),
import("../state/openclaw-state-db.js"),
]);
const mainAgentDir = paths.resolveSharedMainAuthAgentDir(env);
const opsAgentDir = path.join(stateDir, "agents", "ops", "agent");
const sharedStore = makeStore("openai:shared", "shared-key");
const sharedState = {
version: 1,
order: { openai: ["openai:shared"] },
lastGood: { openai: "openai:shared" },
};
const opsStore = makeStore("openai:ops", "ops-key");
sqlite.writePersistedAuthProfileStoreRaw(sharedStore, mainAgentDir);
sqlite.writePersistedAuthProfileStateRaw(sharedState, mainAgentDir);
sqlite.writePersistedAuthProfileStoreRaw(opsStore, opsAgentDir);
return {
env,
stateDir,
mainAgentDir,
opsAgentDir,
sharedStore,
sharedState,
ownership,
sqlite,
storeModule,
persisted,
authState,
migration,
stateDb,
};
}
it("moves exact rows, preserves every effective agent store, and records receipts", async () => {
const fixture = await createFixture();
const effectiveBytes = (agentDir: string) => {
const effective = fixture.storeModule.loadAuthProfileStoreWithoutExternalProfiles(agentDir);
return JSON.stringify({
credentials: fixture.persisted.buildPersistedAuthProfileSecretsStore(effective),
state: fixture.authState.buildPersistedAuthProfileState(effective),
});
};
const before = {
main: effectiveBytes(fixture.mainAgentDir),
ops: effectiveBytes(fixture.opsAgentDir),
};
const detected = fixture.migration.detectSharedAuthStoreMigration({
stateDir: fixture.stateDir,
doctorOnlyStateMigrations: true,
});
expect(
await fixture.migration.migrateSharedAuthStore({ detected, stateDir: fixture.stateDir }),
).toMatchObject({ warnings: [], changes: [expect.stringContaining("Relocated shared auth")] });
expect(fixture.sqlite.readPersistedAuthProfileStoreRaw()).toEqual(fixture.sharedStore);
expect(fixture.sqlite.readPersistedAuthProfileStateRaw()).toEqual(fixture.sharedState);
expect(fixture.sqlite.readPersistedAuthProfileStoreRaw(fixture.mainAgentDir)).toBeNull();
expect(fixture.sqlite.readPersistedAuthProfileStateRaw(fixture.mainAgentDir)).toBeNull();
expect({
main: effectiveBytes(fixture.mainAgentDir),
ops: effectiveBytes(fixture.opsAgentDir),
}).toEqual(before);
const database = fixture.stateDb.openOpenClawStateDatabase({ env: fixture.env }).db;
expect(
database
.prepare("SELECT store_key, store_json FROM auth_profile_stores WHERE store_key = 'shared'")
.get(),
).toEqual({ store_key: "shared", store_json: JSON.stringify(fixture.sharedStore) });
expect(
database
.prepare("SELECT store_key, state_json FROM auth_profile_state WHERE store_key = 'shared'")
.get(),
).toEqual({ store_key: "shared", state_json: JSON.stringify(fixture.sharedState) });
expect(
database
.prepare("SELECT COUNT(*) AS count FROM migration_sources WHERE migration_kind = ?")
.get("shared-auth-store-state-db"),
).toEqual({ count: 2 });
expect(
database
.prepare("SELECT value_json FROM config_machine_state WHERE state_key = 'auth.sharedStore'")
.get(),
).toEqual({ value_json: JSON.stringify({ location: "state-db" }) });
});
for (const crashState of [
"copied-not-flipped",
"copied-source-empty-not-flipped",
"flipped-not-cleaned",
"flipped-cleaned-not-finalized",
] as const) {
it(`converges after the ${crashState} stage boundary`, async () => {
const fixture = await createFixture();
const sourcePath = fixture.sqlite.resolveAuthProfileDatabasePath(fixture.mainAgentDir);
const source = new DatabaseSync(sourcePath);
const sourceStore = source
.prepare(
"SELECT store_json, updated_at FROM auth_profile_store WHERE store_key = 'primary'",
)
.get() as { store_json: string; updated_at: number };
const sourceState = source
.prepare(
"SELECT state_json, updated_at FROM auth_profile_state WHERE state_key = 'primary'",
)
.get() as { state_json: string; updated_at: number };
const target = fixture.stateDb.openOpenClawStateDatabase({ env: fixture.env }).db;
target
.prepare("INSERT INTO auth_profile_stores VALUES ('shared', ?, ?)")
.run(sourceStore.store_json, sourceStore.updated_at);
target
.prepare("INSERT INTO auth_profile_state VALUES ('shared', ?, ?)")
.run(sourceState.state_json, sourceState.updated_at);
if (
crashState === "copied-source-empty-not-flipped" ||
crashState === "flipped-cleaned-not-finalized"
) {
source.prepare("DELETE FROM auth_profile_store WHERE store_key = 'primary'").run();
source.prepare("DELETE FROM auth_profile_state WHERE state_key = 'primary'").run();
}
source.close();
if (crashState === "flipped-not-cleaned" || crashState === "flipped-cleaned-not-finalized") {
target
.prepare(
`INSERT INTO config_machine_state (state_key, value_json, updated_at_ms)
VALUES ('auth.sharedStore', ?, 1)`,
)
.run(JSON.stringify({ location: "state-db" }));
fixture.ownership.noteCommittedSharedAuthStoreOwnership(
{ location: "state-db" },
fixture.env,
);
const runId = "test-shared-auth-pending-cleanup";
const sourceKey = `shared-auth-store:${createHash("sha256")
.update(path.resolve(sourcePath))
.update("\0")
.update("auth_profile_store")
.digest("hex")}`;
target
.prepare(
`INSERT INTO migration_runs (id, started_at, finished_at, status, report_json)
VALUES (?, 1, NULL, 'ownership-flipped', '{}')`,
)
.run(runId);
target
.prepare(
`INSERT INTO migration_sources
(source_key, migration_kind, source_path, target_table, source_sha256,
source_size_bytes, source_record_count, last_run_id, status, imported_at,
removed_source, report_json)
VALUES (?, 'shared-auth-store-state-db', ?, 'auth_profile_stores', NULL,
NULL, NULL, ?, 'ownership-flipped', 1, 0, '{}')`,
)
.run(sourceKey, sourcePath, runId);
}
const detected = fixture.migration.detectSharedAuthStoreMigration({
stateDir: fixture.stateDir,
doctorOnlyStateMigrations: true,
});
const first = await fixture.migration.migrateSharedAuthStore({
detected,
stateDir: fixture.stateDir,
});
const retryDetected = fixture.migration.detectSharedAuthStoreMigration({
stateDir: fixture.stateDir,
doctorOnlyStateMigrations: true,
});
const retry = await fixture.migration.migrateSharedAuthStore({
detected: retryDetected,
stateDir: fixture.stateDir,
});
expect(first.warnings).toEqual([]);
expect(retryDetected).toMatchObject({ hasLegacy: false });
expect(fixture.ownership.resolveSharedAuthStoreOwnership(fixture.env)).toEqual({
location: "state-db",
});
expect(retry).toEqual({ changes: [], warnings: [] });
expect(target.prepare("SELECT COUNT(*) AS count FROM auth_profile_stores").get()).toEqual({
count: 1,
});
expect(target.prepare("SELECT COUNT(*) AS count FROM auth_profile_state").get()).toEqual({
count: 1,
});
const cleanedSource = new DatabaseSync(sourcePath, { readOnly: true });
expect(
cleanedSource
.prepare("SELECT COUNT(*) AS count FROM auth_profile_store WHERE store_key = 'primary'")
.get(),
).toEqual({ count: 0 });
expect(
cleanedSource
.prepare("SELECT COUNT(*) AS count FROM auth_profile_state WHERE state_key = 'primary'")
.get(),
).toEqual({ count: 0 });
cleanedSource.close();
});
}
it("fails closed when the legacy source is a dangling symlink", async () => {
const fixture = await createFixture();
const sourcePath = fixture.sqlite.resolveAuthProfileDatabasePath(fixture.mainAgentDir);
const { closeOpenClawAgentDatabasesForTest } = await import("../state/openclaw-agent-db.js");
closeOpenClawAgentDatabasesForTest();
fs.unlinkSync(sourcePath);
fs.symlinkSync(`${sourcePath}.missing`, sourcePath);
expect(() =>
fixture.migration.detectSharedAuthStoreMigration({
stateDir: fixture.stateDir,
doctorOnlyStateMigrations: true,
}),
).toThrowError(
expect.objectContaining({
name: "SharedAuthStoreSourceInspectionError",
code: "SHARED_AUTH_STORE_SOURCE_UNREADABLE",
action: "openclaw doctor --fix",
sourcePath,
}),
);
expect(
fixture.stateDb
.openOpenClawStateDatabase({ env: fixture.env })
.db.prepare(
"SELECT value_json FROM config_machine_state WHERE state_key = 'auth.sharedStore'",
)
.get(),
).toBeUndefined();
});
it("inspects an unreadable legacy source only in the explicit Doctor path", async () => {
const fixture = await createFixture();
const sourcePath = fixture.sqlite.resolveAuthProfileDatabasePath(fixture.mainAgentDir);
const realLstat = fs.lstatSync;
vi.spyOn(fs, "lstatSync").mockImplementation((pathname, options) => {
if (path.resolve(String(pathname)) === path.resolve(sourcePath)) {
throw Object.assign(new Error("permission denied"), { code: "EACCES" });
}
return realLstat(pathname, options as never);
});
expect(
fixture.migration.detectSharedAuthStoreMigration({
stateDir: fixture.stateDir,
doctorOnlyStateMigrations: false,
}),
).toEqual({ sourcePath, hasLegacy: false });
expect(() =>
fixture.migration.detectSharedAuthStoreMigration({
stateDir: fixture.stateDir,
doctorOnlyStateMigrations: true,
}),
).toThrowError(
expect.objectContaining({
name: "SharedAuthStoreSourceInspectionError",
code: "SHARED_AUTH_STORE_SOURCE_UNREADABLE",
sourcePath,
}),
);
expect(
fixture.stateDb
.openOpenClawStateDatabase({ env: fixture.env })
.db.prepare(
"SELECT value_json FROM config_machine_state WHERE state_key = 'auth.sharedStore'",
)
.get(),
).toBeUndefined();
});
});
@@ -0,0 +1,625 @@
// Doctor-owned staged relocation of legacy shared auth rows into shared SQLite state.
import { createHash } from "node:crypto";
import fs from "node:fs";
import path from "node:path";
import type { DatabaseSync } from "node:sqlite";
import {
noteCommittedSharedAuthStoreOwnership,
resolveSharedAuthStoreOwnership,
SHARED_AUTH_STORE_STATE_KEY,
} from "../agents/auth-profiles/path-resolve.js";
import { resolveSharedMainAuthAgentDir } from "../agents/auth-profiles/shared-main-dir.js";
import {
closeAuthProfileReadPool,
resolveAuthProfileDatabaseOwnerId,
} from "../agents/auth-profiles/sqlite.js";
import type { DB as OpenClawAgentKyselyDatabase } from "../state/openclaw-agent-db.generated.js";
import {
closeOpenClawAgentDatabaseByPath,
runOpenClawAgentWriteTransaction,
} from "../state/openclaw-agent-db.js";
import { withExistingOpenClawStateDatabaseReadOnly } from "../state/openclaw-state-db-readonly.js";
import { tableExists as sqliteTableExists } from "../state/openclaw-state-db-schema-helpers.js";
import type { DB as OpenClawStateKyselyDatabase } from "../state/openclaw-state-db.generated.js";
import { runOpenClawStateWriteTransaction } from "../state/openclaw-state-db.js";
import {
executeSqliteQuerySync,
executeSqliteQueryTakeFirstSync,
getNodeSqliteKysely,
} from "./kysely-sync.js";
import { openNodeSqliteDatabase } from "./node-sqlite.js";
import { withLegacyMigrationStateLock } from "./state-migrations.lock.js";
import type { SharedAuthStoreMigrationDetection } from "./state-migrations.shared-auth-store.types.js";
import type { MigrationMessages } from "./state-migrations.types.js";
const MIGRATION_KIND = "shared-auth-store-state-db";
const AUTH_JSON_MIGRATION_KIND = "auth-profile-json-to-sqlite-v2";
const SOURCE_STORE_KEY = "primary";
const TARGET_STORE_KEY = "shared";
type SourceAuthDatabase = Pick<
OpenClawAgentKyselyDatabase,
"auth_profile_store" | "auth_profile_state"
>;
type SharedAuthMigrationDatabase = Pick<
OpenClawStateKyselyDatabase,
| "auth_profile_stores"
| "auth_profile_state"
| "config_machine_state"
| "migration_runs"
| "migration_sources"
>;
type StoreRow = { store_json: string; updated_at: number };
type StateRow = { state_json: string; updated_at: number };
type AuthRows = { store: StoreRow | null; state: StateRow | null };
type MigrationStage = "copied" | "ownership-flipped" | "completed";
class SharedAuthStoreSourceInspectionError extends Error {
readonly code = "SHARED_AUTH_STORE_SOURCE_UNREADABLE" as const;
readonly action = "openclaw doctor --fix" as const;
readonly sourcePath: string;
constructor(sourcePath: string, operation: string, cause: unknown) {
const detail = cause instanceof Error ? cause.message : String(cause);
super(`Cannot ${operation} legacy shared auth database ${sourcePath}: ${detail}`, { cause });
this.name = "SharedAuthStoreSourceInspectionError";
this.sourcePath = sourcePath;
}
}
function sourceMigrationKey(sourcePath: string, sourceTable: string): string {
return `shared-auth-store:${createHash("sha256")
.update(path.resolve(sourcePath))
.update("\0")
.update(sourceTable)
.digest("hex")}`;
}
function inspectSourceFile(
sourcePath: string,
): { status: "missing" } | { status: "present"; size: number } {
let entry: fs.Stats;
try {
entry = fs.lstatSync(sourcePath);
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") {
return { status: "missing" };
}
throw new SharedAuthStoreSourceInspectionError(sourcePath, "inspect", error);
}
let target = entry;
if (entry.isSymbolicLink()) {
try {
target = fs.statSync(sourcePath);
} catch (error) {
throw new SharedAuthStoreSourceInspectionError(sourcePath, "resolve", error);
}
}
if (!target.isFile()) {
throw new SharedAuthStoreSourceInspectionError(
sourcePath,
"open",
new Error("path is not a regular file"),
);
}
return { status: "present", size: target.size };
}
function readSourceRowsFromDatabase(database: DatabaseSync): AuthRows {
const db = getNodeSqliteKysely<SourceAuthDatabase>(database);
const store = sqliteTableExists(database, "auth_profile_store")
? (executeSqliteQueryTakeFirstSync(
database,
db
.selectFrom("auth_profile_store")
.select(["store_json", "updated_at"])
.where("store_key", "=", SOURCE_STORE_KEY),
) ?? null)
: null;
const state = sqliteTableExists(database, "auth_profile_state")
? (executeSqliteQueryTakeFirstSync(
database,
db
.selectFrom("auth_profile_state")
.select(["state_json", "updated_at"])
.where("state_key", "=", SOURCE_STORE_KEY),
) ?? null)
: null;
return { store, state };
}
function inspectSourceRowsReadOnly(sourcePath: string): AuthRows {
const source = inspectSourceFile(sourcePath);
if (source.status === "missing") {
return { store: null, state: null };
}
let database: DatabaseSync;
try {
database = openNodeSqliteDatabase(sourcePath, { readOnly: true });
} catch (error) {
throw new SharedAuthStoreSourceInspectionError(sourcePath, "open", error);
}
try {
return readSourceRowsFromDatabase(database);
} catch (error) {
throw new SharedAuthStoreSourceInspectionError(sourcePath, "read", error);
} finally {
database.close();
}
}
function readSourceSnapshot(params: { env: NodeJS.ProcessEnv; sourcePath: string }): {
rows: AuthRows;
size: number | null;
} {
const source = inspectSourceFile(params.sourcePath);
if (source.status === "missing") {
return { rows: { store: null, state: null }, size: null };
}
try {
const rows = runOpenClawAgentWriteTransaction(
({ db }) => readSourceRowsFromDatabase(db),
{
agentId: resolveAuthProfileDatabaseOwnerId(path.dirname(params.sourcePath)),
path: params.sourcePath,
env: params.env,
},
{ operationLabel: "state-migration.shared-auth-source-read" },
);
closeAuthProfileReadPool({ kind: "database", databasePath: params.sourcePath });
closeOpenClawAgentDatabaseByPath(params.sourcePath);
return { rows, size: fs.statSync(params.sourcePath).size };
} catch (error) {
throw new SharedAuthStoreSourceInspectionError(params.sourcePath, "read", error);
}
}
function readTargetRows(database: DatabaseSync): AuthRows {
const db = getNodeSqliteKysely<SharedAuthMigrationDatabase>(database);
return {
store:
executeSqliteQueryTakeFirstSync(
database,
db
.selectFrom("auth_profile_stores")
.select(["store_json", "updated_at"])
.where("store_key", "=", TARGET_STORE_KEY),
) ?? null,
state:
executeSqliteQueryTakeFirstSync(
database,
db
.selectFrom("auth_profile_state")
.select(["state_json", "updated_at"])
.where("store_key", "=", TARGET_STORE_KEY),
) ?? null,
};
}
function rowDigest(row: StoreRow | StateRow | null): string {
return createHash("sha256").update(JSON.stringify(row)).digest("hex");
}
function rowsMatch<T extends StoreRow | StateRow>(left: T, right: T): boolean {
return JSON.stringify(left) === JSON.stringify(right);
}
function assertRowsMatch(expected: AuthRows, actual: AuthRows, label: string): void {
if (
(expected.store !== null &&
(actual.store === null || !rowsMatch(expected.store, actual.store))) ||
(expected.state !== null && (actual.state === null || !rowsMatch(expected.state, actual.state)))
) {
throw new Error(`shared auth relocation ${label} verification failed`);
}
}
function migrationRunId(rows: AuthRows): string {
return `shared-auth-store:${createHash("sha256")
.update(rowDigest(rows.store))
.update(rowDigest(rows.state))
.digest("hex")
.slice(0, 24)}`;
}
function recordMigrationLedger(params: {
database: DatabaseSync;
sourcePath: string;
sourceSize: number | null;
rows: AuthRows;
stage: MigrationStage;
now: number;
}): void {
const db = getNodeSqliteKysely<SharedAuthMigrationDatabase>(params.database);
const runId = migrationRunId(params.rows);
const removedSource = params.stage === "completed" ? 1 : 0;
const runReport = JSON.stringify({
source: MIGRATION_KIND,
target: "auth_profile_stores,auth_profile_state",
stage: params.stage,
importedRecordCount: Number(params.rows.store !== null) + Number(params.rows.state !== null),
});
executeSqliteQuerySync(
params.database,
db
.insertInto("migration_runs")
.values({
id: runId,
started_at: params.now,
finished_at: params.stage === "completed" ? params.now : null,
status: params.stage,
report_json: runReport,
})
.onConflict((conflict) =>
conflict.column("id").doUpdateSet({
finished_at: params.stage === "completed" ? params.now : null,
status: params.stage,
report_json: runReport,
}),
),
);
for (const entry of [
{
sourceTable: "auth_profile_store",
targetTable: "auth_profile_stores",
row: params.rows.store,
},
{
sourceTable: "auth_profile_state",
targetTable: "auth_profile_state",
row: params.rows.state,
},
] as const) {
const reportJson = JSON.stringify({
source: entry.sourceTable,
target: entry.targetTable,
stage: params.stage,
sourceSha256: rowDigest(entry.row),
importedRecordCount: entry.row ? 1 : 0,
});
executeSqliteQuerySync(
params.database,
db
.insertInto("migration_sources")
.values({
source_key: sourceMigrationKey(params.sourcePath, entry.sourceTable),
migration_kind: MIGRATION_KIND,
source_path: params.sourcePath,
target_table: entry.targetTable,
source_sha256: rowDigest(entry.row),
source_size_bytes: params.sourceSize,
source_record_count: entry.row ? 1 : 0,
last_run_id: runId,
status: params.stage,
imported_at: params.now,
removed_source: removedSource,
report_json: reportJson,
})
.onConflict((conflict) =>
conflict.column("source_key").doUpdateSet({
source_sha256: rowDigest(entry.row),
source_size_bytes: params.sourceSize,
source_record_count: entry.row ? 1 : 0,
last_run_id: runId,
status: params.stage,
imported_at: params.now,
removed_source: removedSource,
report_json: reportJson,
}),
),
);
}
}
function rewriteAuthJsonMigrationReceipts(
database: DatabaseSync,
sourceDatabasePath: string,
targetDatabasePath: string,
): void {
const db = getNodeSqliteKysely<SharedAuthMigrationDatabase>(database);
const receipts = executeSqliteQuerySync(
database,
db
.selectFrom("migration_sources")
.select(["source_key", "last_run_id", "target_table", "report_json"])
.where("migration_kind", "=", AUTH_JSON_MIGRATION_KIND),
).rows;
for (const receipt of receipts) {
let report: Record<string, unknown>;
try {
report = JSON.parse(receipt.report_json) as Record<string, unknown>;
} catch {
continue;
}
if (
typeof report.targetDatabasePath !== "string" ||
path.resolve(report.targetDatabasePath) !== path.resolve(sourceDatabasePath) ||
(receipt.target_table !== "auth_profile_store" &&
receipt.target_table !== "auth_profile_state")
) {
continue;
}
const targetTable =
receipt.target_table === "auth_profile_store" ? "auth_profile_stores" : "auth_profile_state";
const reportJson = JSON.stringify({
...report,
relocatedFromDatabasePath: report.targetDatabasePath,
targetDatabasePath,
targetTable,
targetStoreKey: TARGET_STORE_KEY,
});
executeSqliteQuerySync(
database,
db
.updateTable("migration_sources")
.set({ target_table: targetTable, report_json: reportJson })
.where("source_key", "=", receipt.source_key),
);
executeSqliteQuerySync(
database,
db
.updateTable("migration_runs")
.set({ report_json: reportJson })
.where("id", "=", receipt.last_run_id),
);
}
}
function copyRowsToState(params: {
env: NodeJS.ProcessEnv;
sourcePath: string;
sourceSize: number | null;
sourceRows: AuthRows;
now: number;
}): AuthRows {
return runOpenClawStateWriteTransaction(
({ db: database, path: targetDatabasePath }) => {
const db = getNodeSqliteKysely<SharedAuthMigrationDatabase>(database);
const target = readTargetRows(database);
if (
params.sourceRows.store &&
target.store &&
!rowsMatch(params.sourceRows.store, target.store)
) {
throw new Error("shared auth credential rows conflict with the relocation target");
}
if (
params.sourceRows.state &&
target.state &&
!rowsMatch(params.sourceRows.state, target.state)
) {
throw new Error("shared auth state rows conflict with the relocation target");
}
if (params.sourceRows.store && !target.store) {
executeSqliteQuerySync(
database,
db.insertInto("auth_profile_stores").values({
store_key: TARGET_STORE_KEY,
store_json: params.sourceRows.store.store_json,
updated_at: params.sourceRows.store.updated_at,
}),
);
}
if (params.sourceRows.state && !target.state) {
executeSqliteQuerySync(
database,
db.insertInto("auth_profile_state").values({
store_key: TARGET_STORE_KEY,
state_json: params.sourceRows.state.state_json,
updated_at: params.sourceRows.state.updated_at,
}),
);
}
const canonicalRows = readTargetRows(database);
assertRowsMatch(params.sourceRows, canonicalRows, "copy");
rewriteAuthJsonMigrationReceipts(database, params.sourcePath, targetDatabasePath);
recordMigrationLedger({
database,
sourcePath: params.sourcePath,
sourceSize: params.sourceSize,
rows: canonicalRows,
stage: "copied",
now: params.now,
});
return canonicalRows;
},
{ env: params.env },
{ operationLabel: "state-migration.shared-auth-copy" },
);
}
function flipOwnership(params: {
env: NodeJS.ProcessEnv;
sourcePath: string;
sourceSize: number | null;
rows: AuthRows;
now: number;
}): boolean {
const flipped = resolveSharedAuthStoreOwnership(params.env).location !== "state-db";
runOpenClawStateWriteTransaction(
({ db: database }) => {
assertRowsMatch(params.rows, readTargetRows(database), "ownership");
const db = getNodeSqliteKysely<SharedAuthMigrationDatabase>(database);
executeSqliteQuerySync(
database,
db
.insertInto("config_machine_state")
.values({
state_key: SHARED_AUTH_STORE_STATE_KEY,
value_json: JSON.stringify({ location: "state-db" }),
updated_at_ms: params.now,
})
.onConflict((conflict) =>
conflict.column("state_key").doUpdateSet({
value_json: JSON.stringify({ location: "state-db" }),
updated_at_ms: params.now,
}),
),
);
recordMigrationLedger({ ...params, database, stage: "ownership-flipped" });
},
{ env: params.env },
{ operationLabel: "state-migration.shared-auth-ownership" },
);
noteCommittedSharedAuthStoreOwnership({ location: "state-db" }, params.env);
return flipped;
}
function cleanupSourceRows(params: { env: NodeJS.ProcessEnv; sourcePath: string }): boolean {
if (inspectSourceFile(params.sourcePath).status === "missing") {
return false;
}
try {
const removed = runOpenClawAgentWriteTransaction(
({ db: database }) => {
const db = getNodeSqliteKysely<SourceAuthDatabase>(database);
const before = readSourceRowsFromDatabase(database);
executeSqliteQuerySync(
database,
db.deleteFrom("auth_profile_store").where("store_key", "=", SOURCE_STORE_KEY),
);
executeSqliteQuerySync(
database,
db.deleteFrom("auth_profile_state").where("state_key", "=", SOURCE_STORE_KEY),
);
const after = readSourceRowsFromDatabase(database);
if (after.store || after.state) {
throw new Error("legacy shared auth rows remain after cleanup");
}
return before.store !== null || before.state !== null;
},
{
agentId: resolveAuthProfileDatabaseOwnerId(path.dirname(params.sourcePath)),
path: params.sourcePath,
env: params.env,
},
{ operationLabel: "state-migration.shared-auth-cleanup" },
);
closeAuthProfileReadPool({ kind: "database", databasePath: params.sourcePath });
closeOpenClawAgentDatabaseByPath(params.sourcePath);
return removed;
} catch (error) {
throw new SharedAuthStoreSourceInspectionError(params.sourcePath, "clean", error);
}
}
function finalizeMigration(params: {
env: NodeJS.ProcessEnv;
sourcePath: string;
sourceSize: number | null;
rows: AuthRows;
now: number;
}): void {
runOpenClawStateWriteTransaction(
({ db: database }) => {
assertRowsMatch(params.rows, readTargetRows(database), "cleanup");
recordMigrationLedger({ ...params, database, stage: "completed" });
},
{ env: params.env },
{ operationLabel: "state-migration.shared-auth-finalize" },
);
}
function hasPendingCleanup(env: NodeJS.ProcessEnv, sourcePath: string): boolean {
return (
withExistingOpenClawStateDatabaseReadOnly(
({ db: database }) => {
const db = getNodeSqliteKysely<SharedAuthMigrationDatabase>(database);
const row = executeSqliteQueryTakeFirstSync(
database,
db
.selectFrom("migration_sources")
.select("source_key")
.where("migration_kind", "=", MIGRATION_KIND)
.where("source_path", "=", sourcePath)
.where("removed_source", "=", 0)
.limit(1),
);
return Boolean(row);
},
{ env },
) ?? false
);
}
/** Detect relocation or unfinished cleanup only in the explicit Doctor repair path. */
export function detectSharedAuthStoreMigration(params: {
stateDir: string;
doctorOnlyStateMigrations?: boolean;
}): SharedAuthStoreMigrationDetection {
const env = { ...process.env, OPENCLAW_STATE_DIR: params.stateDir };
const sourcePath = path.join(resolveSharedMainAuthAgentDir(env), "openclaw-agent.sqlite");
if (params.doctorOnlyStateMigrations !== true) {
return { sourcePath, hasLegacy: false };
}
const ownership = resolveSharedAuthStoreOwnership(env);
const sourceRows = inspectSourceRowsReadOnly(sourcePath);
return {
sourcePath,
hasLegacy:
ownership.location === "legacy-main" ||
sourceRows.store !== null ||
sourceRows.state !== null ||
hasPendingCleanup(env, sourcePath),
};
}
/** Converge copy, ownership, and cleanup while excluding live Gateway writers. */
export async function migrateSharedAuthStore(params: {
detected: SharedAuthStoreMigrationDetection;
stateDir: string;
env?: NodeJS.ProcessEnv;
now?: () => number;
}): Promise<MigrationMessages> {
if (!params.detected.hasLegacy) {
return { changes: [], warnings: [] };
}
return await withLegacyMigrationStateLock({
stateDir: params.stateDir,
env: params.env,
label: "legacy shared auth store",
releaseLabel: "Shared auth store",
errorLabel: "Failed relocating the shared auth store",
run: async (env) => {
const now = params.now?.() ?? Date.now();
const source = readSourceSnapshot({ env, sourcePath: params.detected.sourcePath });
const rows = copyRowsToState({
env,
sourcePath: params.detected.sourcePath,
sourceSize: source.size,
sourceRows: source.rows,
now,
});
const ownershipFlipped = flipOwnership({
env,
sourcePath: params.detected.sourcePath,
sourceSize: source.size,
rows,
now,
});
const sourceCleaned = cleanupSourceRows({ env, sourcePath: params.detected.sourcePath });
finalizeMigration({
env,
sourcePath: params.detected.sourcePath,
sourceSize: source.size,
rows,
now,
});
return {
changes: [
...(ownershipFlipped ? ["Relocated shared auth profiles into shared SQLite state."] : []),
...(sourceCleaned && !ownershipFlipped
? ["Completed legacy shared auth row cleanup."]
: []),
],
warnings: [],
...(ownershipFlipped
? {
notices: ["The main agent no longer owns shared credentials and can now be deleted."],
}
: {}),
};
},
});
}
@@ -0,0 +1,4 @@
export type SharedAuthStoreMigrationDetection = {
sourcePath: string;
hasLegacy: boolean;
};
+2
View File
@@ -7,6 +7,7 @@ import type { LegacyExecApprovalsDetection } from "./state-migrations.exec-appro
import type { LegacyMcpOAuthDetection } from "./state-migrations.mcp-oauth.types.js";
import type { LegacyMeetingTranscriptsDetection } from "./state-migrations.meeting-transcripts.types.js";
import type { LegacyRestartSentinelDetection } from "./state-migrations.restart-sentinel.types.js";
import type { SharedAuthStoreMigrationDetection } from "./state-migrations.shared-auth-store.types.js";
import type { LegacyWorkspaceStateDetection } from "./state-migrations.workspace-setup.types.js";
export type LegacyRescuePendingDetection = {
@@ -64,6 +65,7 @@ export type LegacyStateDetection = {
hasLegacy: boolean;
preview: string[];
};
sharedAuthStore: SharedAuthStoreMigrationDetection;
worktrees: {
hasLegacy: boolean;
pathRewrites: Array<{ id: string; fromPath: string; toPath: string }>;
+100
View File
@@ -6,6 +6,7 @@ import { expectDefined } from "@openclaw/normalization-core";
import { afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
import { useAutoCleanupTempDirTracker } from "../../test/helpers/temp-dir.js";
import { registerResolvedAgentDir } from "../agents/agent-dir-registry.js";
import { noteCommittedSharedAuthStoreOwnership } from "../agents/auth-profiles/path-resolve.js";
import {
clearRuntimeAuthProfileStoreSnapshots,
getRuntimeAuthProfileStoreCredentialMutationToken,
@@ -14,6 +15,7 @@ import {
import {
readPersistedAuthProfileStateRaw,
readPersistedAuthProfileStoreRaw,
readPersistedSharedAuthProfileStoreRaw,
resolveAuthProfileDatabasePath,
writePersistedAuthProfileStateRaw,
} from "../agents/auth-profiles/sqlite.js";
@@ -27,6 +29,10 @@ import {
closeOpenClawAgentDatabasesForTest,
openOpenClawAgentDatabase,
} from "../state/openclaw-agent-db.js";
import {
closeOpenClawStateDatabaseForTest,
openOpenClawStateDatabase,
} from "../state/openclaw-state-db.js";
import {
buildTalkTestProviderConfig,
TALK_TEST_PROVIDER_API_KEY_PATH,
@@ -306,6 +312,8 @@ describe("secrets apply", () => {
storeTesting.resetRuntimeSnapshotPublisherForTest();
clearRuntimeAuthProfileStoreSnapshots();
closeOpenClawAgentDatabasesForTest();
closeOpenClawStateDatabaseForTest();
vi.unstubAllEnvs();
await fs.rm(fixture.rootDir, { recursive: true, force: true });
});
@@ -404,6 +412,98 @@ describe("secrets apply", () => {
expect(nextAuthStore.profiles["openai:default"].keyRef).toBeUndefined();
});
it("keeps shared and implicit agent writes inside the explicitly routed state root", async () => {
const ambientStateDir = path.join(fixture.rootDir, "ambient-state");
const ambientMainDir = path.join(ambientStateDir, "agents", "main", "agent");
const ambientOpsDir = path.join(ambientStateDir, "agents", "ops", "agent");
vi.stubEnv("OPENCLAW_STATE_DIR", ambientStateDir);
saveAuthProfileStore(
{
version: 1,
profiles: {
"openai:ambient-shared": {
type: "api_key",
provider: "openai",
key: "sk-ambient-shared",
},
},
},
ambientMainDir,
{ filterExternalAuthProfiles: false, syncExternalCli: false },
);
saveAuthProfileStore(
{
version: 1,
profiles: {
"openai:ambient-ops": {
type: "api_key",
provider: "openai",
key: "sk-ambient-ops",
},
},
},
ambientOpsDir,
{ filterExternalAuthProfiles: false, syncExternalCli: false },
);
await writeJsonFile(fixture.configPath, {
agents: { entries: { ops: {} } },
models: { providers: { openai: createOpenAiProviderConfig() } },
});
const stateDatabase = openOpenClawStateDatabase({ env: fixture.env }).db;
stateDatabase
.prepare(
`INSERT INTO config_machine_state (state_key, value_json, updated_at_ms)
VALUES ('auth.sharedStore', ?, 1)`,
)
.run(JSON.stringify({ location: "state-db" }));
stateDatabase
.prepare(
"INSERT INTO auth_profile_stores (store_key, store_json, updated_at) VALUES (?, ?, 1)",
)
.run(
"shared",
JSON.stringify({
version: 1,
profiles: {
"openai:target-shared": {
type: "api_key",
provider: "openai",
key: "sk-target-shared",
},
},
}),
);
noteCommittedSharedAuthStoreOwnership({ location: "state-db" }, fixture.env);
await runSecretsApply({
plan: createPlan({
targets: [createOpenAiProviderTarget()],
options: createOneWayScrubOptions(),
}),
env: fixture.env,
write: true,
});
expect(readPersistedSharedAuthProfileStoreRaw(fixture.env)).toEqual({
version: 1,
profiles: {
"openai:target-shared": {
type: "api_key",
provider: "openai",
},
},
});
expect(readPersistedAuthProfileStoreRaw(ambientMainDir)).toMatchObject({
profiles: { "openai:ambient-shared": { key: "sk-ambient-shared" } },
});
expect(readPersistedAuthProfileStoreRaw(ambientOpsDir)).toMatchObject({
profiles: { "openai:ambient-ops": { key: "sk-ambient-ops" } },
});
expect(
readPersistedAuthProfileStoreRaw(path.join(fixture.stateDir, "agents", "ops", "agent")),
).toBeNull();
});
it("skips exec SecretRef checks during dry-run unless explicitly allowed", async () => {
if (process.platform === "win32") {
return;
+73 -58
View File
@@ -1,16 +1,15 @@
/** Applies secrets migration plans across config files, auth stores, and env files. */
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { isDeepStrictEqual } from "node:util";
import { registerResolvedAgentDir } from "../agents/agent-dir-registry.js";
import { resolveAgentConfig } from "../agents/agent-scope.js";
import { resolveAgentDir } from "../agents/agent-scope.js";
import { loadAuthProfileStoreForSecretsRuntime } from "../agents/auth-profiles.js";
import { AUTH_STORE_VERSION } from "../agents/auth-profiles/constants.js";
import { resolveSharedAuthStorePath } from "../agents/auth-profiles/path-resolve.js";
import {
coercePersistedAuthProfileStore,
loadPersistedAuthProfileStore,
loadPersistedSharedAuthProfileStore,
} from "../agents/auth-profiles/persisted.js";
import { resolveAuthProfileDatabasePath } from "../agents/auth-profiles/sqlite.js";
import {
@@ -28,10 +27,12 @@ import {
import type { ConfigWriteOptions } from "../config/io.js";
import { coerceSecretRef, type SecretProviderConfig } from "../config/types.secrets.js";
import { normalizePluginConfigId } from "../plugins/plugin-config-trust.js";
import { normalizeAgentId } from "../routing/session-key.js";
import { resolveUserPath } from "../utils.js";
import { iterateAuthProfileCredentials } from "./auth-profiles-scan.js";
import { listAuthProfileStoreAgentDirs } from "./auth-store-paths.js";
import {
listAuthProfileStoreTargets as listDiscoveredAuthProfileStoreTargets,
type AuthProfileStoreTarget,
} from "./auth-store-paths.js";
import { createSecretsConfigIO } from "./config-io.js";
import { getSkippedExecRefStaticError } from "./exec-resolution-policy.js";
import { deletePathStrict, getPath, setPathCreateStrict } from "./path-utils.js";
@@ -61,7 +62,7 @@ type ApplyWrite = {
};
type AuthStoreSnapshot = {
agentDir: string;
target: AuthProfileStoreTarget;
persistence: ReturnType<typeof captureAuthProfileStorePersistenceSnapshot>;
owned?: ReturnType<typeof captureAuthProfileStorePersistenceSnapshot>;
};
@@ -72,7 +73,7 @@ type ProjectedState = {
configPath: string;
configWriteOptions: ConfigWriteOptions;
authStoreByPath: Map<string, Record<string, unknown>>;
authStoreAgentDirByPath: Map<string, string>;
authStoreTargetByPath: Map<string, AuthProfileStoreTarget>;
envRawByPath: Map<string, string>;
changedFiles: Set<string>;
warnings: string[];
@@ -92,7 +93,7 @@ type ConfigTargetMutationResult = {
providerTargets: Set<string>;
configChanged: boolean;
authStoreByPath: Map<string, Record<string, unknown>>;
authStoreAgentDirByPath: Map<string, string>;
authStoreTargetByPath: Map<string, AuthProfileStoreTarget>;
};
type MutableAuthProfileStore = Record<string, unknown> & {
@@ -309,8 +310,9 @@ async function projectPlanState(params: {
planTargets: params.plan.targets,
nextConfig,
stateDir,
env: params.env,
authStoreByPath: new Map<string, Record<string, unknown>>(),
authStoreAgentDirByPath: new Map<string, string>(),
authStoreTargetByPath: new Map<string, AuthProfileStoreTarget>(),
changedFiles,
});
if (targetMutations.configChanged) {
@@ -320,10 +322,11 @@ async function projectPlanState(params: {
const authStoreByPath = scrubAuthStoresForProviderTargets({
nextConfig,
stateDir,
env: params.env,
providerTargets: targetMutations.providerTargets,
scrubbedValues: targetMutations.scrubbedValues,
authStoreByPath: targetMutations.authStoreByPath,
authStoreAgentDirByPath: targetMutations.authStoreAgentDirByPath,
authStoreTargetByPath: targetMutations.authStoreTargetByPath,
changedFiles,
warnings,
enabled: options.scrubAuthProfilesForProviderTargets,
@@ -354,7 +357,7 @@ async function projectPlanState(params: {
configPath,
configWriteOptions: writeOptions,
authStoreByPath,
authStoreAgentDirByPath: targetMutations.authStoreAgentDirByPath,
authStoreTargetByPath: targetMutations.authStoreTargetByPath,
envRawByPath,
changedFiles,
warnings,
@@ -368,8 +371,9 @@ function applyConfigTargetMutations(params: {
planTargets: SecretsPlanTarget[];
nextConfig: OpenClawConfig;
stateDir: string;
env: NodeJS.ProcessEnv;
authStoreByPath: Map<string, Record<string, unknown>>;
authStoreAgentDirByPath: Map<string, string>;
authStoreTargetByPath: Map<string, AuthProfileStoreTarget>;
changedFiles: Set<string>;
}): ConfigTargetMutationResult {
const resolvedTargets = params.planTargets.map((target) => ({
@@ -387,8 +391,9 @@ function applyConfigTargetMutations(params: {
resolved,
nextConfig: params.nextConfig,
stateDir: params.stateDir,
env: params.env,
authStoreByPath: params.authStoreByPath,
authStoreAgentDirByPath: params.authStoreAgentDirByPath,
authStoreTargetByPath: params.authStoreTargetByPath,
scrubbedValues,
});
if (authStoreChanged) {
@@ -400,6 +405,7 @@ function applyConfigTargetMutations(params: {
resolveAuthStoreTargetForAgent({
nextConfig: params.nextConfig,
stateDir: params.stateDir,
env: params.env,
agentId,
}).path,
);
@@ -445,17 +451,18 @@ function applyConfigTargetMutations(params: {
providerTargets,
configChanged,
authStoreByPath: params.authStoreByPath,
authStoreAgentDirByPath: params.authStoreAgentDirByPath,
authStoreTargetByPath: params.authStoreTargetByPath,
};
}
function scrubAuthStoresForProviderTargets(params: {
nextConfig: OpenClawConfig;
stateDir: string;
env: NodeJS.ProcessEnv;
providerTargets: Set<string>;
scrubbedValues: Set<string>;
authStoreByPath: Map<string, Record<string, unknown>>;
authStoreAgentDirByPath: Map<string, string>;
authStoreTargetByPath: Map<string, AuthProfileStoreTarget>;
changedFiles: Set<string>;
warnings: string[];
enabled: boolean;
@@ -464,13 +471,21 @@ function scrubAuthStoresForProviderTargets(params: {
return params.authStoreByPath;
}
for (const target of listAuthProfileStoreTargets(params.nextConfig, params.stateDir)) {
const { agentDir, path: authStorePath } = target;
for (const target of listAuthProfileStoreTargets(
params.nextConfig,
params.stateDir,
params.env,
)) {
const authStorePath = target.path;
const existing = params.authStoreByPath.get(authStorePath);
if (!existing && !fs.existsSync(authStorePath)) {
continue;
}
const parsed = existing ?? loadPersistedAuthProfileStore(agentDir);
const parsed =
existing ??
(target.kind === "shared"
? loadPersistedSharedAuthProfileStore(target.env)
: loadPersistedAuthProfileStore(target.agentDir));
if (!parsed || !isRecord(parsed.profiles)) {
continue;
}
@@ -510,7 +525,7 @@ function scrubAuthStoresForProviderTargets(params: {
}
if (mutated) {
params.authStoreByPath.set(authStorePath, nextStore);
params.authStoreAgentDirByPath.set(authStorePath, agentDir);
params.authStoreTargetByPath.set(authStorePath, target);
params.changedFiles.add(authStorePath);
}
}
@@ -533,8 +548,9 @@ function resolveAuthStoreForTarget(params: {
target: SecretsPlanTarget;
nextConfig: OpenClawConfig;
stateDir: string;
env: NodeJS.ProcessEnv;
authStoreByPath: Map<string, Record<string, unknown>>;
authStoreAgentDirByPath: Map<string, string>;
authStoreTargetByPath: Map<string, AuthProfileStoreTarget>;
}): { path: string; store: MutableAuthProfileStore } {
const agentId = (params.target.agentId ?? "").trim();
if (!agentId) {
@@ -543,6 +559,7 @@ function resolveAuthStoreForTarget(params: {
const authStoreTarget = resolveAuthStoreTargetForAgent({
nextConfig: params.nextConfig,
stateDir: params.stateDir,
env: params.env,
agentId,
});
const authStorePath = authStoreTarget.path;
@@ -550,43 +567,31 @@ function resolveAuthStoreForTarget(params: {
const loaded = existing ?? loadPersistedAuthProfileStore(authStoreTarget.agentDir);
const store = ensureMutableAuthStore(isRecord(loaded) ? loaded : undefined);
params.authStoreByPath.set(authStorePath, store);
params.authStoreAgentDirByPath.set(authStorePath, authStoreTarget.agentDir);
params.authStoreTargetByPath.set(authStorePath, authStoreTarget);
return { path: authStorePath, store };
}
function resolveAuthStoreTargetForAgent(params: {
nextConfig: OpenClawConfig;
stateDir: string;
env: NodeJS.ProcessEnv;
agentId: string;
}): { agentDir: string; path: string } {
const normalizedAgentId = normalizeAgentId(params.agentId);
const configuredAgentDir = resolveAgentConfig(
params.nextConfig,
normalizedAgentId,
)?.agentDir?.trim();
if (configuredAgentDir) {
const agentDir = resolveUserPath(configuredAgentDir);
registerResolvedAgentDir({ agentId: normalizedAgentId, agentDir });
return { agentDir, path: resolveAuthProfileDatabasePath(agentDir) };
}
const agentDir = path.join(
resolveUserPath(params.stateDir),
"agents",
normalizedAgentId,
"agent",
);
registerResolvedAgentDir({ agentId: normalizedAgentId, agentDir });
return { agentDir, path: resolveAuthProfileDatabasePath(agentDir) };
}): Extract<AuthProfileStoreTarget, { kind: "agent" }> {
const scopedEnv = {
...params.env,
OPENCLAW_STATE_DIR: params.stateDir,
OPENCLAW_AGENT_DIR: undefined,
};
const agentDir = resolveAgentDir(params.nextConfig, params.agentId, scopedEnv);
return { kind: "agent", agentDir, path: resolveAuthProfileDatabasePath(agentDir) };
}
function listAuthProfileStoreTargets(
config: OpenClawConfig,
stateDir: string,
): Array<{ agentDir: string; path: string }> {
return listAuthProfileStoreAgentDirs(config, stateDir).map((agentDir) => ({
agentDir,
path: resolveAuthProfileDatabasePath(agentDir),
}));
env: NodeJS.ProcessEnv,
): AuthProfileStoreTarget[] {
return listDiscoveredAuthProfileStoreTargets(config, stateDir, env);
}
function ensureAuthProfileContainer(params: {
@@ -645,8 +650,9 @@ function applyAuthProfileTargetMutation(params: {
resolved: ResolvedPlanTargetEntry["resolved"];
nextConfig: OpenClawConfig;
stateDir: string;
env: NodeJS.ProcessEnv;
authStoreByPath: Map<string, Record<string, unknown>>;
authStoreAgentDirByPath: Map<string, string>;
authStoreTargetByPath: Map<string, AuthProfileStoreTarget>;
scrubbedValues: Set<string>;
}): boolean {
if (params.resolved.entry.configFile !== "auth-profiles.json") {
@@ -656,8 +662,9 @@ function applyAuthProfileTargetMutation(params: {
target: params.target,
nextConfig: params.nextConfig,
stateDir: params.stateDir,
env: params.env,
authStoreByPath: params.authStoreByPath,
authStoreAgentDirByPath: params.authStoreAgentDirByPath,
authStoreTargetByPath: params.authStoreTargetByPath,
});
let changed = ensureAuthProfileContainer({
target: params.target,
@@ -760,7 +767,7 @@ async function validateProjectedSecretsState(params: {
const authStoreLookup = new Map<string, Record<string, unknown>>();
for (const [authStorePath, store] of params.authStoreByPath.entries()) {
authStoreLookup.set(resolveUserPath(authStorePath), store);
authStoreLookup.set(resolveUserPath(authStorePath, params.env), store);
}
if (params.checkFullRuntime) {
await prepareSecretsRuntimeSnapshot({
@@ -772,7 +779,10 @@ async function validateProjectedSecretsState(params: {
includeAuthStoreRefs: params.write || params.authStoreByPath.size > 0,
loadAuthStore: (agentDir?: string) => {
const storePath = resolveUserPath(
agentDir ? resolveAuthProfileDatabasePath(agentDir) : resolveSharedAuthStorePath(),
agentDir
? resolveAuthProfileDatabasePath(agentDir)
: resolveSharedAuthStorePath(params.env),
params.env,
);
const override = authStoreLookup.get(storePath);
if (override) {
@@ -878,11 +888,14 @@ export async function runSecretsApply(params: {
snapshots.set(pathname, captureFileSnapshot(pathname));
}
};
const captureAuthStore = (pathname: string, agentDir: string) => {
const captureAuthStore = (pathname: string, target: AuthProfileStoreTarget) => {
if (!authStoreSnapshots.has(pathname)) {
authStoreSnapshots.set(pathname, {
agentDir,
persistence: captureAuthProfileStorePersistenceSnapshot(agentDir),
target,
persistence: captureAuthProfileStorePersistenceSnapshot(
target.kind === "agent" ? target.agentDir : undefined,
target.kind === "shared" ? { stateDir: target.stateDir } : {},
),
});
}
};
@@ -897,8 +910,8 @@ export async function runSecretsApply(params: {
mode: 0o600,
});
}
for (const [pathname, agentDir] of projected.authStoreAgentDirByPath.entries()) {
captureAuthStore(pathname, agentDir);
for (const [pathname, target] of projected.authStoreTargetByPath.entries()) {
captureAuthStore(pathname, target);
}
try {
@@ -913,9 +926,9 @@ export async function runSecretsApply(params: {
writeTextFileAtomic(writeLocal.path, writeLocal.content, writeLocal.mode);
}
for (const [pathname, value] of projected.authStoreByPath.entries()) {
const agentDir = projected.authStoreAgentDirByPath.get(pathname);
const target = projected.authStoreTargetByPath.get(pathname);
const store = coercePersistedAuthProfileStore(value);
if (agentDir && store) {
if (target && store) {
const snapshot = authStoreSnapshots.get(pathname);
if (!snapshot) {
throw new Error(`missing captured auth profile store for ${pathname}`);
@@ -923,7 +936,8 @@ export async function runSecretsApply(params: {
const committed = saveAuthProfileStoreIfPersistenceSnapshotMatches({
store,
snapshot: snapshot.persistence,
agentDir,
agentDir: target.kind === "agent" ? target.agentDir : undefined,
...(target.kind === "shared" ? { stateDir: target.stateDir } : {}),
});
// Persisted rows commit before runtime publication. Record their exact
// ownership first so a publication failure can still roll them back.
@@ -951,7 +965,8 @@ export async function runSecretsApply(params: {
restoreAuthProfileStorePersistenceSnapshot(
snapshot.persistence,
snapshot.owned,
snapshot.agentDir,
snapshot.target.kind === "agent" ? snapshot.target.agentDir : undefined,
snapshot.target.kind === "shared" ? { stateDir: snapshot.target.stateDir } : {},
);
} catch {
// Best effort only; preserve original error.
+67 -2
View File
@@ -2,14 +2,21 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { useAutoCleanupTempDirTracker } from "../../test/helpers/temp-dir.js";
import {
noteCommittedSharedAuthStoreOwnership,
resolveSharedAuthStorePath,
} from "../agents/auth-profiles/path-resolve.js";
import {
resolveAuthProfileDatabasePath,
writePersistedAuthProfileStoreRaw,
} from "../agents/auth-profiles/sqlite.js";
import { closeOpenClawAgentDatabasesForTest } from "../state/openclaw-agent-db.js";
import { closeOpenClawStateDatabaseForTest } from "../state/openclaw-state-db.js";
import {
closeOpenClawStateDatabaseForTest,
openOpenClawStateDatabase,
} from "../state/openclaw-state-db.js";
import { runSecretsAudit } from "./audit.js";
import { writeSecretStoreEntry } from "./store/secret-store.js";
@@ -268,6 +275,7 @@ describe("secrets audit", () => {
});
afterEach(async () => {
vi.unstubAllEnvs();
closeOpenClawAgentDatabasesForTest();
closeOpenClawStateDatabaseForTest();
await fs.rm(fixture.rootDir, { recursive: true, force: true });
@@ -702,6 +710,63 @@ describe("secrets audit", () => {
expect(authPlaintextPaths).toEqual(["profiles.openai:plaintext-with-ref.key"]);
});
it("reads a relocated shared store from the explicitly routed state root", async () => {
const ambientStateDir = path.join(fixture.rootDir, "ambient-state");
const ambientAgentDir = path.join(ambientStateDir, "agents", "main", "agent");
vi.stubEnv("OPENCLAW_STATE_DIR", ambientStateDir);
writePersistedAuthProfileStoreRaw(
{
version: 1,
profiles: {
"openai:ambient": {
type: "api_key",
provider: "openai",
key: "sk-ambient-plaintext",
},
},
},
ambientAgentDir,
);
const stateDatabase = openOpenClawStateDatabase({ env: fixture.env }).db;
stateDatabase
.prepare(
`INSERT INTO config_machine_state (state_key, value_json, updated_at_ms)
VALUES ('auth.sharedStore', ?, 1)`,
)
.run(JSON.stringify({ location: "state-db" }));
stateDatabase
.prepare(
"INSERT INTO auth_profile_stores (store_key, store_json, updated_at) VALUES (?, ?, 1)",
)
.run(
"shared",
JSON.stringify({
version: 1,
profiles: {
"openai:target": {
type: "api_key",
provider: "openai",
key: "sk-target-plaintext",
},
},
}),
);
noteCommittedSharedAuthStoreOwnership({ location: "state-db" }, fixture.env);
const report = await runSecretsAudit({ env: fixture.env });
const sharedFindings = report.findings
.filter(
(entry) =>
entry.code === "PLAINTEXT_FOUND" &&
entry.file === resolveSharedAuthStorePath(fixture.env),
)
.map((entry) => entry.jsonPath);
expect(sharedFindings).toContain("profiles.openai:target.key");
expect(sharedFindings).not.toContain("profiles.openai:ambient.key");
expect(report.filesScanned).not.toContain(resolveAuthProfileDatabasePath(ambientAgentDir));
});
it("exempts direct routing headers but audits request headers in openclaw config", async () => {
await writeJsonFile(fixture.configPath, {
models: {
+37 -37
View File
@@ -1,13 +1,13 @@
/** Audits configured secrets and reports plaintext/ref migration status. */
import fs from "node:fs";
import os from "node:os";
import {
listLegacyAuthProfileArchives,
listLegacyAuthProfileSources,
} from "../agents/auth-profiles/legacy-source-diagnostic.js";
import { resolveSharedMainAuthAgentDir } from "../agents/auth-profiles/shared-main-dir.js";
import {
readPersistedAuthProfileStoreRaw,
resolveAuthProfileDatabasePath,
readPersistedSharedAuthProfileStoreRaw,
} from "../agents/auth-profiles/sqlite.js";
import {
isNonSecretApiKeyMarker,
@@ -23,7 +23,7 @@ import { runTasksWithConcurrency } from "../utils/run-with-concurrency.js";
import { findSecretStorePlaintextResidueFindings } from "./audit-store.js";
import type { PlaintextAssignment } from "./audit-store.js";
import { iterateAuthProfileCredentials } from "./auth-profiles-scan.js";
import { listAuthProfileStoreAgentDirs } from "./auth-store-paths.js";
import { listAuthProfileStoreTargets, type AuthProfileStoreTarget } from "./auth-store-paths.js";
import { createSecretsConfigIO } from "./config-io.js";
import { getSkippedExecRefStaticError, selectRefsForExecPolicy } from "./exec-resolution-policy.js";
import { isLikelySensitiveModelProviderHeaderName } from "./model-provider-header-policy.js";
@@ -106,11 +106,7 @@ type ProviderAuthState = {
modes: Set<"api_key" | "token" | "oauth">;
};
type SecretDefaults = {
env?: string;
file?: string;
exec?: string;
};
type SecretDefaults = { env?: string; file?: string; exec?: string };
type AuditCollector = {
findings: SecretsAuditFinding[];
@@ -251,43 +247,46 @@ function collectConfigSecrets(params: {
}
}
function collectAuthStoreSecrets(params: {
agentDir: string;
collector: AuditCollector;
defaults?: SecretDefaults;
}): void {
const authStorePath = resolveAuthProfileDatabasePath(params.agentDir);
function collectAuthStoreSecrets(
target: AuthProfileStoreTarget,
collector: AuditCollector,
defaults?: SecretDefaults,
): void {
const authStorePath = target.path;
if (!fs.existsSync(authStorePath)) {
return;
}
const parsed = readPersistedAuthProfileStoreRaw(params.agentDir);
const parsed =
target.kind === "shared"
? readPersistedSharedAuthProfileStoreRaw(target.env)
: readPersistedAuthProfileStoreRaw(target.agentDir);
if (!isRecord(parsed) || !isRecord(parsed.profiles)) {
return;
}
params.collector.filesScanned.add(authStorePath);
collector.filesScanned.add(authStorePath);
for (const entry of iterateAuthProfileCredentials(parsed.profiles)) {
if (entry.kind === "api_key" || entry.kind === "token") {
const { ref } = resolveSecretInputRef({
value: entry.value,
refValue: entry.refValue,
defaults: params.defaults,
defaults,
});
const authoredValueRef = coerceSecretRef(entry.value, params.defaults);
const authoredValueRef = coerceSecretRef(entry.value, defaults);
if (ref) {
params.collector.refAssignments.push({
collector.refAssignments.push({
file: authStorePath,
path: `profiles.${entry.profileId}.${entry.valueField}`,
ref,
expected: "string",
provider: entry.provider,
});
trackAuthProviderState(params.collector, entry.provider, entry.kind);
trackAuthProviderState(collector, entry.provider, entry.kind);
}
if (authoredValueRef) {
continue;
}
if (isNonEmptyString(entry.value)) {
addFinding(params.collector, {
addFinding(collector, {
code: "PLAINTEXT_FOUND",
severity: "warn",
file: authStorePath,
@@ -299,12 +298,12 @@ function collectAuthStoreSecrets(params: {
provider: entry.provider,
profileId: entry.profileId,
});
trackAuthProviderState(params.collector, entry.provider, entry.kind);
trackAuthProviderState(collector, entry.provider, entry.kind);
}
continue;
}
if (entry.hasAccess || entry.hasRefresh) {
addFinding(params.collector, {
addFinding(collector, {
code: "LEGACY_RESIDUE",
severity: "info",
file: authStorePath,
@@ -313,7 +312,7 @@ function collectAuthStoreSecrets(params: {
provider: entry.provider,
profileId: entry.profileId,
});
trackAuthProviderState(params.collector, entry.provider, "oauth");
trackAuthProviderState(collector, entry.provider, "oauth");
}
}
}
@@ -415,8 +414,9 @@ function collectLegacyAuthSourceFindings(params: {
collector: AuditCollector;
}): void {
const seen = new Set<string>();
const agentDirs = listAuthProfileStoreAgentDirs(params.config, params.stateDir);
for (const agentDir of agentDirs) {
const targets = listAuthProfileStoreTargets(params.config, params.stateDir, params.env);
for (const target of targets) {
const agentDir = target.kind === "agent" ? target.agentDir : undefined;
for (const source of listLegacyAuthProfileSources({ agentDir, env: params.env })) {
if (seen.has(source.path)) {
continue;
@@ -431,7 +431,13 @@ function collectLegacyAuthSourceFindings(params: {
});
}
}
for (const archive of listLegacyAuthProfileArchives({ agentDirs, env: params.env })) {
const sharedMainDir = resolveSharedMainAuthAgentDir(params.env);
for (const archive of listLegacyAuthProfileArchives({
agentDirs: targets
.flatMap((target) => (target.kind === "agent" ? [target.agentDir] : []))
.concat(sharedMainDir),
env: params.env,
})) {
if (seen.has(archive.path)) {
continue;
}
@@ -635,9 +641,7 @@ export async function runSecretsAudit(
} = {},
): Promise<SecretsAuditReport> {
const env = params.env ?? process.env;
const allowExec = Boolean(params.allowExec);
const io = createSecretsConfigIO({ env });
const snapshot = await io.readConfigFileSnapshot();
const snapshot = await createSecretsConfigIO({ env }).readConfigFileSnapshot();
const configPath = resolveUserPath(snapshot.path);
const defaults = snapshot.valid ? snapshot.config.secrets?.defaults : undefined;
@@ -666,12 +670,8 @@ export async function runSecretsAudit(
collector,
env,
});
for (const agentDir of listAuthProfileStoreAgentDirs(config, stateDir)) {
collectAuthStoreSecrets({
agentDir,
collector,
defaults,
});
for (const target of listAuthProfileStoreTargets(config, stateDir, env)) {
collectAuthStoreSecrets(target, collector, defaults);
}
for (const modelsJsonPath of listAgentModelsJsonPaths(config, stateDir, env)) {
collectModelsJsonSecrets({
@@ -683,7 +683,7 @@ export async function runSecretsAudit(
collector,
config,
env,
allowExec,
allowExec: Boolean(params.allowExec),
});
resolution = {
refsChecked: unresolvedRefResult.refsChecked,
+32 -17
View File
@@ -3,43 +3,58 @@ import fs from "node:fs";
import path from "node:path";
import { listAgentIds, resolveAgentDir } from "../agents/agent-scope.js";
import { resolveSharedAuthStorePath } from "../agents/auth-profiles/path-resolve.js";
import { resolveAuthProfileDatabasePath } from "../agents/auth-profiles/sqlite.js";
import type { OpenClawConfig } from "../config/types.openclaw.js";
import { resolveUserPath } from "../utils.js";
/**
* Lists deduplicated auth-profile store agent dirs that may contain SecretRefs.
* Covers implicit main, discovered state-dir agents, and config-declared agent dirs.
*/
export function listAuthProfileStoreAgentDirs(config: OpenClawConfig, stateDir: string): string[] {
const paths = new Set<string>();
export type AuthProfileStoreTarget =
| { kind: "shared"; path: string; env: NodeJS.ProcessEnv; stateDir: string }
| { kind: "agent"; path: string; agentDir: string };
/** Lists canonical auth-profile databases that may contain SecretRefs. */
export function listAuthProfileStoreTargets(
config: OpenClawConfig,
stateDir: string,
env: NodeJS.ProcessEnv = process.env,
): AuthProfileStoreTarget[] {
const targets = new Map<string, AuthProfileStoreTarget>();
// Scope default auth store discovery to the provided stateDir instead of
// ambient process env, so scans do not include unrelated host-global stores.
const scopedEnv = {
...process.env,
...env,
OPENCLAW_STATE_DIR: stateDir,
OPENCLAW_AGENT_DIR: undefined,
};
paths.add(path.dirname(resolveSharedAuthStorePath(scopedEnv)));
const addTarget = (target: AuthProfileStoreTarget) => {
const key = path.resolve(target.path);
if (targets.get(key)?.kind === "shared") {
return;
}
targets.set(key, target);
};
addTarget({
kind: "shared",
path: resolveSharedAuthStorePath(scopedEnv),
env: scopedEnv,
stateDir,
});
const agentsRoot = path.join(resolveUserPath(stateDir), "agents");
const agentsRoot = path.join(resolveUserPath(stateDir, scopedEnv), "agents");
if (fs.existsSync(agentsRoot)) {
for (const entry of fs.readdirSync(agentsRoot, { withFileTypes: true })) {
if (!entry.isDirectory()) {
continue;
}
paths.add(path.join(agentsRoot, entry.name, "agent"));
const agentDir = path.join(agentsRoot, entry.name, "agent");
addTarget({ kind: "agent", agentDir, path: resolveAuthProfileDatabasePath(agentDir) });
}
}
// Configured agent dirs may live outside stateDir; include them after state-dir discovery.
for (const agentId of listAgentIds(config)) {
if (agentId === "main") {
paths.add(path.dirname(resolveSharedAuthStorePath(scopedEnv)));
continue;
}
const agentDir = resolveAgentDir(config, agentId);
paths.add(resolveUserPath(agentDir));
const agentDir = resolveUserPath(resolveAgentDir(config, agentId, scopedEnv), scopedEnv);
addTarget({ kind: "agent", agentDir, path: resolveAuthProfileDatabasePath(agentDir) });
}
return [...paths];
return [...targets.values()];
}
@@ -14,7 +14,6 @@ function createExtensionCodexAppServerAttemptExtraVitestConfig(
"extensions/codex/src/app-server/run-attempt.hooks.test.ts",
"extensions/codex/src/app-server/run-attempt.native-hook-relay.test.ts",
"extensions/codex/src/app-server/run-attempt-runtime.authority.test.ts",
"extensions/codex/src/app-server/run-attempt-state.test.ts",
"extensions/codex/src/app-server/run-attempt.steering.test.ts",
"extensions/codex/src/app-server/run-attempt.turn-watches.test.ts",
"extensions/codex/src/app-server/run-attempt.usage-limits.test.ts",