diff --git a/docs/cli/claws.md b/docs/cli/claws.md index ec1bc59b2619..60fdbf93e51f 100644 --- a/docs/cli/claws.md +++ b/docs/cli/claws.md @@ -63,12 +63,17 @@ concise handoff with evidence. ``` OpenClaw automatically discovers the optional `profiles/openclaw.yml` file. -There is no manifest pointer. Other harnesses may discover their own +No manifest pointer is required. Other harnesses may discover their own conventional profile, such as `profiles/codex.yml`, without changing the portable manifest. -Experimental packages that used `metadata.openclaw.config` must move that file -to `profiles/openclaw.yml` and remove the metadata entry. +The older `metadata.openclaw.config` pointer is deprecated but still read, so +packages published against it keep working. Reading one reports a +`deprecated_openclaw_profile_pointer` warning; move that file to +`profiles/openclaw.yml` and remove the metadata entry. A pointer that is not a +package-relative `.yml`/`.yaml` path is rejected, and a pointer that references +a different file while `profiles/openclaw.yml` also exists is rejected as a +conflict. ```yaml schemaVersion: 1 @@ -110,6 +115,35 @@ The conventional profile is limited to 256 KiB, must be JSON-compatible YAML, ma not use aliases, anchors, tags, or merge keys, and must be a regular, non-symlinked, non-hardlinked file inside the package. +An OpenClaw profile may also declare harness-specific extension requirements: + +```yaml +schemaVersion: 1 +agent: {} +extensions: + - id: incident-tools + kind: plugin + format: claude + source: clawhub + ref: "@acme/incident-tools" + version: 2.0.0 +``` + +`format` asserts the artifact format that OpenClaw must detect (`openclaw`, +`claude`, `codex`, or `cursor`). The canonical plugin preflight resolves the +exact artifact and reports which components the current OpenClaw adapter maps +and which remain unavailable. Missing identity, integrity, format detection, or +adapter identity blocks apply. Extension-backed plugins use the existing +plugin installer and ownership model; they are shared host requirements, not +Claw-owned members or a second package system. + +OpenClaw ignores foreign harness profiles during apply. Package integrity still +covers every published package byte, while a development snapshot binds the +portable manifest, bootstrap and workspace sources, and the selected OpenClaw +profile. Status and doctor report adapter mapping drift or unavailable +inspection. Export writes extension-backed plugins to `profiles/openclaw.yml` +and does not duplicate them in the portable `packages` list. + Package and workspace paths must remain inside the package root. Manifests are limited to 1 MiB, package metadata to 256 KiB, and workspace sources enforce separate per-file and aggregate limits. Workspace sources also reject symlinked @@ -174,10 +208,13 @@ Skills and plugins use exact ClawHub versions: The dry run uses the existing skill and plugin preflight paths to resolve the exact artifact, integrity, and any ClawHub trust warning before consent. The -warning remains visible in the integrity-bound plan. Apply installs missing artifacts -or reuses matching ones and records whether the Claw introduced or referenced -each resource. Plugins remain process-wide OpenClaw capabilities rather than -per-agent installations. +warning remains visible in the integrity-bound plan. Each requirement is shown +as satisfied, missing-installable, conflicting, or setup-required. The exact +plan consent approves missing installs; OpenClaw completes those canonical +plugin actions before creating the agent or workspace. Apply reuses matching +artifacts and records whether the Claw introduced or referenced each resource. +Plugins remain process-wide OpenClaw capabilities rather than per-agent +installations. Cron jobs declare scheduled work for the new agent: @@ -222,7 +259,9 @@ removal follow the same ownership policy as other Claw resources. ## Inspect and preview -Validate the source without planning local changes: +Validate the source without planning local changes. For OpenClaw profile +extensions, inspect also performs the canonical read-only artifact probe and +reports mapped and unavailable components: ```bash openclaw claws inspect ./incident-triage.claw.json @@ -252,11 +291,11 @@ defaults collide with local state. For disposable profiles and parallel validati pass an explicit `--workspace`; `OPENCLAW_STATE_DIR` relocates runtime state but does not change the default workspace location. -Adding a Claw creates the new agent and workspace configuration, seeds optional -first-run instructions, writes declared workspace assets, installs or reuses -declared skill and plugin artifacts, and records package, MCP, and cron -provenance. Existing files are not overwritten, and retries fail closed when -owned content drifted. +Adding a Claw first realizes consented shared plugin requirements, then creates +the new agent and workspace configuration, seeds optional first-run +instructions, writes declared workspace assets, realizes workspace skills, and +records package, MCP, and cron provenance. Existing files are not overwritten, +and retries fail closed when owned content drifted. ## Inspect installed state @@ -300,9 +339,9 @@ The plan compares current provenance and live state with the target manifest. It reports agent, workspace, package, MCP, cron, and ownership changes, including capability escalations and blockers. Capability escalations have separate machine-readable records and `!` lines with exact redacted effects in -human output. Resolved package integrity, install identity, and any trust -warning are included. Removing a package declaration releases this Claw's edge -without uninstalling the artifact during update. The eventual +human output. Resolved package integrity, install identity, trust warnings, and +remaining local setup prerequisites are included. Removing a package declaration +releases this Claw's edge without uninstalling the artifact during update. The eventual exact `planIntegrity` confirmation binds that disclosed set as well as ordinary content changes. Hosts may use the same records for a separate dialog or an aggregate multi-agent review. Apply the exact reviewed plan with explicit @@ -339,8 +378,9 @@ The default removes eligible managed state and releases referenced state. Modified files and resources with another current owner are retained or blocked. Cleanup choices are part of the plan digest; `--yes` never broadens them. Globally installed plugins are retained while this Claw's reference is -released; use the ordinary plugin lifecycle separately when you intend to -uninstall a process-wide plugin. +released. Removal reports which retained requirements Claw add introduced; use +the ordinary plugin lifecycle separately when you intend to uninstall a +process-wide plugin. To remove unchanged Claw-introduced references that have no other current owner, include `--remove-unused` in both preview and apply. To select exact diff --git a/scripts/check-kysely-guardrails.mjs b/scripts/check-kysely-guardrails.mjs index 08bea26594ff..8baf6da45d81 100644 --- a/scripts/check-kysely-guardrails.mjs +++ b/scripts/check-kysely-guardrails.mjs @@ -74,6 +74,7 @@ const rawSqliteAllowPathGroups = { ], "agent auth profile read-only bootstrap": ["src/agents/auth-profiles/sqlite.ts"], "read-only shared state database access": [ + "src/claws/package-resume.ts", "src/state/openclaw-agent-db-readonly.ts", "src/state/openclaw-state-db-readonly.ts", ], diff --git a/src/claws/add.ts b/src/claws/add.ts index 9a845de4d791..9f4055d3f7f0 100644 --- a/src/claws/add.ts +++ b/src/claws/add.ts @@ -1,4 +1,5 @@ // Applies the package, agent, workspace, and managed-file slices of a consented Claw add plan. +import type { Stats } from "node:fs"; import { lstat, mkdir, rmdir } from "node:fs/promises"; import { dirname, resolve } from "node:path"; import { stableStringify } from "@openclaw/normalization-core"; @@ -107,6 +108,16 @@ function hasUnsupportedMutationActions(plan: ClawAddPlan): boolean { ); } +function planWithPackageActions( + plan: ClawAddPlan, + predicate: (action: ClawAddPlan["actions"][number]) => boolean, +): ClawAddPlan { + return { + ...plan, + actions: plan.actions.filter((action) => action.kind !== "package" || predicate(action)), + }; +} + function statusAtLeast(status: ClawInstallStatus, phase: ClawInstallStatus): boolean { const order: Record = { pending: 0, @@ -224,38 +235,124 @@ export async function applyClawAddPlan( throw new ClawAddMutationError("provenance_failed", (error as Error).message); } - const installPackages = options.installPackages ?? installClawPackages; - let packages: PersistedClawPackageRef[] = []; - const workspace = resolve(resolveUserPath(plan.agent.workspace)); const workspacePhaseRecorded = statusAtLeast(installRecord.status, "workspace_ready"); - const workspaceState = workspacePhaseRecorded - ? await lstat(workspace).catch((error: unknown) => { - if ( - typeof error === "object" && - error !== null && - "code" in error && - error.code === "ENOENT" - ) { - return undefined; - } - throw error; - }) - : undefined; + let workspaceState: Stats | undefined; + try { + assertWorkspacePathUnchanged(workspace); + workspaceState = await lstat(workspace).catch((error: unknown) => { + if ( + typeof error === "object" && + error !== null && + "code" in error && + error.code === "ENOENT" + ) { + return undefined; + } + throw error; + }); + } catch (error) { + clearUnownedInstallRecord(plan.agent.finalId, ["pending", "partial"], options); + if (error instanceof ClawAddMutationError) { + throw error; + } + throw new ClawAddMutationError( + "workspace_parent_failed", + `Could not inspect workspace ${JSON.stringify(workspace)}: ${(error as Error).message}`, + ); + } + + if (!workspacePhaseRecorded && workspaceState) { + markInstallStatus(plan.agent.finalId, "partial", ["pending", "partial"], options); + return partialResult({ + plan, + installRecord, + workspaceCreated: false, + configCommitted: false, + packages: [], + error: { + code: "workspace_collision", + message: `Workspace ${JSON.stringify(workspace)} was created after planning.`, + }, + nowMs: options.nowMs, + }); + } if (workspaceState && !workspaceState.isDirectory()) { throw new ClawAddMutationError( "workspace_collision", `Workspace ${JSON.stringify(workspace)} is no longer a directory.`, ); } + let workspaceCreated = workspaceState?.isDirectory() ?? false; let configCommitted = statusAtLeast(installRecord.status, "config_committed"); + const installPackages = options.installPackages ?? installClawPackages; + let packages: PersistedClawPackageRef[] = []; + const preserveRecordedPhaseOrMarkPartial = (): ClawInstallStatus => { + if (workspacePhaseRecorded) { + return installRecord.status; + } + markInstallStatus(plan.agent.finalId, "partial", ["pending", "partial"], options); + return "partial"; + }; + + const hostRequirementPlan = planWithPackageActions( + plan, + (action) => action.details?.kind === "plugin", + ); + const hostRequirementActions = hostRequirementPlan.actions.filter( + (action) => action.kind === "package", + ); + if (hostRequirementActions.length > 0) { + try { + packages = await installPackages(hostRequirementPlan, options); + } catch (error) { + const packageError = + error instanceof ClawPackageInstallError + ? error + : new ClawPackageInstallError( + "package_install_failed", + error instanceof Error ? error.message : String(error), + packages, + ); + const installStatus = preserveRecordedPhaseOrMarkPartial(); + return partialResult({ + plan, + installRecord, + workspaceCreated, + configCommitted, + packages: packageError.installedPackages, + installStatus, + error: { code: packageError.code, message: packageError.message }, + nowMs: options.nowMs, + }); + } + } try { assertWorkspacePathUnchanged(workspace); await mkdir(dirname(workspace), { recursive: true }); assertWorkspacePathUnchanged(workspace); } catch (error) { + if (packages.length > 0) { + const installStatus = preserveRecordedPhaseOrMarkPartial(); + return partialResult({ + plan, + installRecord, + workspaceCreated, + configCommitted, + packages, + installStatus, + error: { + code: error instanceof ClawAddMutationError ? error.code : "workspace_parent_failed", + message: + error instanceof ClawAddMutationError + ? error.message + : `Could not create parent directory for workspace ${JSON.stringify(workspace)}: ${(error as Error).message}`, + }, + nowMs: options.nowMs, + }); + } clearUnownedInstallRecord(plan.agent.finalId, ["pending", "partial"], options); if (error instanceof ClawAddMutationError) { throw error; @@ -310,19 +407,32 @@ export async function applyClawAddPlan( } } + // Seed and attest the consented package bootstrap while the workspace is still + // private. Committing the agent config first makes the agent routable, so a + // concurrent `sessions.create` can stock-seed BOOTSTRAP.md and strand the add at + // `config_committed` with a seed conflict that no retry can clear. try { await (options.seedPackageBootstrap ?? seedClawPackageBootstrap)(plan, { ...options, ...(options.nowMs !== undefined ? { nowMs: options.nowMs } : {}), }); } catch (error) { + const installStatus: ClawInstallStatus = configCommitted + ? "config_committed" + : "workspace_ready"; + markInstallStatus( + plan.agent.finalId, + installStatus, + configCommitted ? ["config_committed"] : ["workspace_ready", "config_committed"], + options, + ); return partialResult({ plan, installRecord, workspaceCreated, configCommitted, packages, - installStatus: "workspace_ready", + installStatus, error: { code: error instanceof ClawBootstrapWriteError ? error.code : "bootstrap_write_failed", message: error instanceof Error ? error.message : String(error), @@ -472,7 +582,17 @@ export async function applyClawAddPlan( try { // Skills require their workspace. Recurring work is enabled only after all // package mutation succeeds. - packages = await installPackages(plan, options); + const workspacePackagePlan = planWithPackageActions( + plan, + (action) => action.details?.kind !== "plugin", + ); + const workspacePackageActions = workspacePackagePlan.actions.filter( + (action) => action.kind === "package", + ); + if (workspacePackageActions.length > 0) { + const workspacePackages = await installPackages(workspacePackagePlan, options); + packages = [...packages, ...workspacePackages]; + } } catch (error) { const packageError = error instanceof ClawPackageInstallError @@ -480,7 +600,7 @@ export async function applyClawAddPlan( : new ClawPackageInstallError( "package_install_failed", error instanceof Error ? error.message : String(error), - packages, + [], ); return partialResult({ plan, @@ -488,7 +608,7 @@ export async function applyClawAddPlan( workspaceCreated, configCommitted, workspaceFiles, - packages: packageError.installedPackages, + packages: [...packages, ...packageError.installedPackages], installStatus: "config_committed", error: { code: packageError.code, message: packageError.message }, nowMs: options.nowMs, diff --git a/src/claws/application-plan.ts b/src/claws/application-plan.ts new file mode 100644 index 000000000000..74a899ab435e --- /dev/null +++ b/src/claws/application-plan.ts @@ -0,0 +1,227 @@ +import { createHash } from "node:crypto"; +import { stableStringify } from "@openclaw/normalization-core"; +import type { + ClawAddCapabilityChange, + ClawAddPlanAction, + ClawDiagnostic, + ClawExtensionPlan, + ClawLocalPrerequisite, + ClawOpenClawExtension, + ClawOpenClawProfile, + ClawPackage, + ClawPackagePreflight, + ClawPackagePreflightResult, +} from "./types.js"; + +export function clawProfileExtensionPackages( + profile: ClawOpenClawProfile | undefined, +): ClawPackage[] { + return (profile?.extensions ?? []).map((extension) => ({ + kind: "plugin", + source: extension.source, + ref: extension.ref, + version: extension.version, + })); +} + +function blocker(code: string, path: string, message: string): ClawDiagnostic { + return { level: "error", code, phase: "plan", path, message }; +} + +export function findClawExtensionPackageCollisions(params: { + packages: ClawPackage[]; + extensions: ClawOpenClawExtension[]; +}): Array<{ index: number; diagnostic: ClawDiagnostic }> { + const declaredPackageIds = new Set(params.packages.map((pkg) => `${pkg.kind}:${pkg.ref}`)); + const collisions: Array<{ index: number; diagnostic: ClawDiagnostic }> = []; + + for (const [index, extension] of params.extensions.entries()) { + const packageId = `plugin:${extension.ref}`; + if (declaredPackageIds.has(packageId)) { + collisions.push({ + index, + diagnostic: blocker( + "extension_package_collision", + `$.profiles.openclaw.extensions[${index}]`, + `Extension package ${JSON.stringify(packageId)} is already declared by the portable manifest or another profile extension.`, + ), + }); + continue; + } + declaredPackageIds.add(packageId); + } + + return collisions; +} + +function extensionCapabilityChange(params: { + extension: ClawOpenClawExtension; + preflight: ClawPackagePreflightResult; +}): ClawAddCapabilityChange { + const effect = { + id: params.extension.id, + source: params.extension.source, + ref: params.extension.ref, + version: params.extension.version, + expectedFormat: params.extension.format, + detectedFormat: params.preflight.detectedFormat ?? "unresolved", + integrity: params.preflight.integrity ?? "unresolved", + mapped: params.preflight.mapped ?? [], + unavailable: params.preflight.unavailable ?? [], + adapterIdentity: params.preflight.adapterIdentity ?? "unresolved", + ...(params.preflight.installId ? { installId: params.preflight.installId } : {}), + ...(params.preflight.warning ? { riskWarning: params.preflight.warning } : {}), + }; + const change = { + kind: "package" as const, + id: `extension:${params.extension.id}`, + path: `openclaw.extensions.${params.extension.id}`, + action: params.preflight.action === "reuse" ? ("reuse" as const) : ("install" as const), + reason: + params.preflight.action === "reuse" + ? "The OpenClaw profile requires access to an existing native extension." + : "The OpenClaw profile requires installation of native extension content or executable code.", + effect, + }; + return { + ...change, + classification: "escalation", + requiresDistinctConsent: true, + digest: `sha256:${createHash("sha256").update(stableStringify(effect)).digest("hex")}`, + }; +} + +export async function planClawExtensions(params: { + extensions: ClawOpenClawExtension[]; + workspace: string; + packagePreflight?: ClawPackagePreflight; +}): Promise<{ + extensions: ClawExtensionPlan[]; + actions: ClawAddPlanAction[]; + capabilityChanges: ClawAddCapabilityChange[]; + requirements: ClawLocalPrerequisite[]; + blockers: ClawDiagnostic[]; +}> { + const extensions: ClawExtensionPlan[] = []; + const actions: ClawAddPlanAction[] = []; + const capabilityChanges: ClawAddCapabilityChange[] = []; + const requirements: ClawLocalPrerequisite[] = []; + const blockers: ClawDiagnostic[] = []; + + for (const [index, extension] of params.extensions.entries()) { + const preflight: ClawPackagePreflightResult = params.packagePreflight + ? await params.packagePreflight( + { + kind: "plugin", + source: extension.source, + ref: extension.ref, + version: extension.version, + }, + params.workspace, + ) + : { + ok: false as const, + code: "package_install_unavailable", + message: "Extension preflight is unavailable.", + }; + const completeProvenance = + preflight.ok && + Boolean( + preflight.integrity && + preflight.installId && + preflight.action && + preflight.detectedFormat && + preflight.adapterIdentity, + ); + const incompleteProvenance = + preflight.ok && !completeProvenance + ? blocker( + "extension_provenance_incomplete", + `$.profiles.openclaw.extensions[${index}]`, + `Extension ${JSON.stringify(extension.id)} did not resolve complete canonical identity and adapter provenance.`, + ) + : undefined; + const formatMismatch = + preflight.ok && completeProvenance && preflight.detectedFormat !== extension.format + ? blocker( + "extension_format_mismatch", + `$.profiles.openclaw.extensions[${index}].format`, + `Extension ${JSON.stringify(extension.id)} declares format ${JSON.stringify(extension.format)}, but the canonical plugin detector found ${JSON.stringify(preflight.detectedFormat ?? "unknown")}.`, + ) + : undefined; + const diagnostic = !preflight.ok + ? blocker( + preflight.code ?? "extension_preflight_failed", + `$.profiles.openclaw.extensions[${index}]`, + preflight.message ?? "Extension preflight failed.", + ) + : (incompleteProvenance ?? formatMismatch); + if (diagnostic) { + blockers.push(diagnostic); + } + if (preflight.ok && preflight.requirements) { + requirements.push(...preflight.requirements); + } + const requirementState: ClawExtensionPlan["requirementState"] = diagnostic + ? "conflicting" + : preflight.action === "install" + ? "missing-installable" + : preflight.requirements && preflight.requirements.length > 0 + ? "setup-required" + : "satisfied"; + const extensionPlan: ClawExtensionPlan = { + ...extension, + ...(preflight.detectedFormat ? { detectedFormat: preflight.detectedFormat } : {}), + ...(preflight.integrity ? { integrity: preflight.integrity } : {}), + ...(preflight.installId ? { installId: preflight.installId } : {}), + ...(preflight.action ? { ownerAction: preflight.action } : {}), + requirementState, + mapped: preflight.mapped ?? [], + unavailable: preflight.unavailable ?? [], + ...(preflight.adapterIdentity ? { adapterIdentity: preflight.adapterIdentity } : {}), + blocked: Boolean(diagnostic), + }; + extensions.push(extensionPlan); + actions.push({ + kind: "package", + id: `plugin:${extension.ref}`, + action: preflight.ok && preflight.action === "reuse" ? "reuse" : "install", + target: `${extension.source}:${extension.ref}@${extension.version}`, + ...(preflight.integrity ? { digest: preflight.integrity } : {}), + details: { + kind: "plugin", + source: extension.source, + ref: extension.ref, + version: extension.version, + ...(preflight.integrity ? { integrity: preflight.integrity } : {}), + ...(preflight.installId ? { installId: preflight.installId } : {}), + ...(preflight.action ? { ownerAction: preflight.action } : {}), + requirementState, + ...(preflight.requirements ? { prerequisites: preflight.requirements } : {}), + ...(completeProvenance + ? { + extension: { + id: extension.id, + format: extension.format, + detectedFormat: preflight.detectedFormat!, + mapped: preflight.mapped ?? [], + unavailable: preflight.unavailable ?? [], + adapterIdentity: preflight.adapterIdentity!, + }, + } + : {}), + expectedState: !preflight.ok + ? "unresolved" + : preflight.action === "reuse" + ? "present-exact" + : "absent", + ...(preflight.warning ? { riskWarning: preflight.warning } : {}), + }, + blocked: extensionPlan.blocked, + ...(diagnostic ? { reason: diagnostic.message } : {}), + }); + capabilityChanges.push(extensionCapabilityChange({ extension, preflight })); + } + + return { extensions, actions, capabilityChanges, requirements, blockers }; +} diff --git a/src/claws/application-provenance.ts b/src/claws/application-provenance.ts new file mode 100644 index 000000000000..6716778eb8f5 --- /dev/null +++ b/src/claws/application-provenance.ts @@ -0,0 +1,90 @@ +import { stableStringify } from "@openclaw/normalization-core"; +import { clawProfileExtensionPackages } from "./application-plan.js"; +import type { ClawPackageStatus } from "./lifecycle-status.js"; +import type { PersistedClawPackageRef } from "./provenance.js"; +import type { + ClawAddPlanAction, + ClawDiagnostic, + ClawManifest, + ClawOpenClawProfile, + ClawPackage, + ClawPackagePreflight, + ClawPackagePreflightResult, +} from "./types.js"; + +export function isApplicationUpdateBlocker(entry: ClawDiagnostic): boolean { + return ( + entry.code !== "workspace_collision" && + entry.code !== "agent_id_collision" && + !entry.path.startsWith("$.packages") + ); +} + +export function clawPackageKey(value: Pick): string { + return `${value.kind}:${value.ref}`; +} + +export function recordingClawPackagePreflight( + preflight: ClawPackagePreflight | undefined, + workspace: string, + results: Map, + currentPackages: ReadonlyMap, +): ClawPackagePreflight { + return async (pkg) => { + const result = preflight + ? await preflight(pkg, workspace) + : { + ok: false as const, + code: "package_install_unavailable", + message: "Package preflight is unavailable.", + }; + const current = currentPackages.get(clawPackageKey(pkg)); + const normalized = + !result.ok && + pkg.kind === "plugin" && + result.code === "plugin_version_conflict" && + current?.state === "present" && + current.origin === "claw-introduced" && + !current.independentOwner && + current.version !== pkg.version && + result.installedVersion === current.version + ? { ...result, ok: true as const, action: "install" as const } + : result; + results.set(clawPackageKey(pkg), normalized); + return normalized; + }; +} + +export function clawTargetPackages( + manifest: ClawManifest, + profile: ClawOpenClawProfile | undefined, +) { + return new Map( + [...manifest.packages, ...clawProfileExtensionPackages(profile)].map( + (pkg) => [clawPackageKey(pkg), pkg] as const, + ), + ); +} + +export function clawWorkspaceActionsById(actions: ClawAddPlanAction[]) { + return new Map( + actions + .filter((action) => action.kind === "workspaceFile") + .map((action) => [action.id, action] as const), + ); +} + +export function clawPackageActionsById(actions: ClawAddPlanAction[]) { + return new Map( + actions + .filter((action) => action.kind === "package") + .map((action) => [action.id, action] as const), + ); +} + +export function clawExtensionProvenanceChanged( + current: PersistedClawPackageRef["extension"], + target: ClawAddPlanAction | undefined, +): boolean { + return stableStringify(current ?? null) !== stableStringify(target?.details?.extension ?? null); +} diff --git a/src/claws/application-schema.test.ts b/src/claws/application-schema.test.ts new file mode 100644 index 000000000000..7dbe60799e67 --- /dev/null +++ b/src/claws/application-schema.test.ts @@ -0,0 +1,293 @@ +import { mkdir, realpath, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { afterEach, describe, expect, it } from "vitest"; +import { useAutoCleanupTempDirTracker } from "../../test/helpers/temp-dir.js"; +import { clawProfileExtensionPackages } from "./application-plan.js"; +import { buildClawAddPlan } from "./lifecycle.js"; +import { parseClawManifest, parseClawOpenClawProfile } from "./schema.js"; +import type { ClawManifest, ClawSourceIdentity } from "./types.js"; + +const tempDirs = useAutoCleanupTempDirTracker(afterEach); + +function requireManifest(value: unknown): ClawManifest { + const result = parseClawManifest(value); + if (!result.ok) { + throw new Error(JSON.stringify(result.diagnostics)); + } + return result.manifest; +} + +async function createPlanSource(): Promise<{ source: ClawSourceIdentity; workspace: string }> { + const root = tempDirs.make("openclaw-claw-application-plan-"); + await mkdir(join(root, "workspace", "schemas"), { recursive: true }); + await writeFile(join(root, "workspace", "schemas", "market.json"), "{}\n", "utf8"); + return { + source: { + kind: "package", + name: "@acme/market-analyst", + version: "1.0.0", + packageRoot: root, + manifestPath: join(root, "CLAW.md"), + integrityKind: "development-snapshot", + integrity: "sha256:test", + byteLength: 0, + }, + workspace: join(await realpath(root), "new-workspace"), + }; +} + +const extension = { + id: "market-data", + kind: "plugin", + format: "claude", + source: "clawhub", + ref: "@acme/market-data", + version: "2.0.1", +} as const; + +describe("Claw application schema v1", () => { + it("accepts strict native extension assertions without a schema bump", () => { + expect( + parseClawOpenClawProfile({ + schemaVersion: 1, + agent: { tools: { profile: "coding" } }, + extensions: [extension], + }), + ).toMatchObject({ + ok: true, + profile: { schemaVersion: 1, extensions: [{ id: "market-data", format: "claude" }] }, + }); + }); + + it("rejects duplicate extensions and unknown formats", () => { + expect( + parseClawOpenClawProfile({ schemaVersion: 1, agent: {}, extensions: [extension, extension] }) + .ok, + ).toBe(false); + expect( + parseClawOpenClawProfile({ + schemaVersion: 1, + agent: {}, + extensions: [{ ...extension, format: "future" }], + }).ok, + ).toBe(false); + }); +}); + +describe("Claw application planning v1", () => { + it("projects profile extensions onto canonical plugin package identities", () => { + expect( + clawProfileExtensionPackages({ schemaVersion: 1, agent: {}, extensions: [extension] }), + ).toEqual([ + { + kind: "plugin", + source: "clawhub", + ref: "@acme/market-data", + version: "2.0.1", + }, + ]); + expect(clawProfileExtensionPackages(undefined)).toEqual([]); + }); + + it("plans a canonical extension and an ordinary managed schema asset", async () => { + const { source, workspace } = await createPlanSource(); + const manifest = requireManifest({ + schemaVersion: 1, + agent: { id: "market-analyst" }, + workspace: { + files: [{ source: "workspace/schemas/market.json", path: "schemas/market.json" }], + }, + }); + const plan = await buildClawAddPlan({ + manifest, + openClawProfile: { schemaVersion: 1, agent: {}, extensions: [extension] }, + source, + context: { + workspace, + packagePreflight: async () => ({ + ok: true, + action: "install", + integrity: `sha256:${"b".repeat(64)}`, + installId: "market-data", + detectedFormat: "claude", + mapped: ["commands", "skills"], + unavailable: ["agents"], + adapterIdentity: "openclaw/test", + }), + }, + }); + + expect(plan.extensions).toEqual([ + expect.objectContaining({ + id: "market-data", + detectedFormat: "claude", + mapped: ["commands", "skills"], + unavailable: ["agents"], + requirementState: "missing-installable", + blocked: false, + }), + ]); + expect(plan.actions).toContainEqual( + expect.objectContaining({ + kind: "package", + id: "plugin:@acme/market-data", + blocked: false, + details: expect.objectContaining({ + extension: expect.objectContaining({ + id: "market-data", + adapterIdentity: "openclaw/test", + }), + }), + }), + ); + expect(plan.actions).toContainEqual( + expect.objectContaining({ kind: "workspaceFile", id: "schemas/market.json" }), + ); + }); + + it("reports a reused extension with remaining local setup as setup-required", async () => { + const { source, workspace } = await createPlanSource(); + const prerequisite = { + kind: "plugin-setup" as const, + plugin: "market-data", + provider: "market-data", + envVars: ["MARKET_DATA_TOKEN"], + authMethods: [], + }; + const plan = await buildClawAddPlan({ + manifest: requireManifest({ schemaVersion: 1, agent: { id: "market-analyst" } }), + openClawProfile: { schemaVersion: 1, agent: {}, extensions: [extension] }, + source, + context: { + workspace, + packagePreflight: async () => ({ + ok: true, + action: "reuse", + integrity: `sha256:${"b".repeat(64)}`, + installId: "market-data", + detectedFormat: "claude", + mapped: ["skills"], + unavailable: [], + adapterIdentity: "openclaw/test", + requirements: [prerequisite], + }), + }, + }); + + expect(plan.extensions?.[0]).toMatchObject({ + requirementState: "setup-required", + ownerAction: "reuse", + }); + expect(plan.actions).toContainEqual( + expect.objectContaining({ + kind: "package", + action: "reuse", + details: expect.objectContaining({ requirementState: "setup-required" }), + }), + ); + expect(plan.readiness).toEqual({ ready: false, requirements: [prerequisite] }); + }); + + it("blocks incomplete adapter provenance", async () => { + const { source, workspace } = await createPlanSource(); + const plan = await buildClawAddPlan({ + manifest: requireManifest({ schemaVersion: 1, agent: { id: "market-analyst" } }), + openClawProfile: { schemaVersion: 1, agent: {}, extensions: [extension] }, + source, + context: { + workspace, + packagePreflight: async () => ({ + ok: true, + action: "install", + integrity: `sha256:${"b".repeat(64)}`, + installId: "market-data", + detectedFormat: "claude", + mapped: ["skills"], + unavailable: [], + }), + }, + }); + + expect(plan.blockers).toContainEqual( + expect.objectContaining({ code: "extension_provenance_incomplete" }), + ); + expect(plan.actions).toContainEqual( + expect.objectContaining({ id: "plugin:@acme/market-data", blocked: true }), + ); + }); + + it("blocks duplicate portable and profile package declarations", async () => { + const { source, workspace } = await createPlanSource(); + const manifest = requireManifest({ + schemaVersion: 1, + agent: { id: "market-analyst" }, + packages: [ + { + kind: "plugin", + source: "clawhub", + ref: "@acme/market-data", + version: "2.0.1", + }, + ], + }); + const plan = await buildClawAddPlan({ + manifest, + openClawProfile: { schemaVersion: 1, agent: {}, extensions: [extension] }, + source, + context: { + workspace, + packagePreflight: async () => ({ + ok: true, + action: "install", + integrity: `sha256:${"b".repeat(64)}`, + installId: "market-data", + detectedFormat: "claude", + mapped: ["skills"], + unavailable: [], + adapterIdentity: "openclaw/test", + }), + }, + }); + + expect(plan.blockers).toContainEqual( + expect.objectContaining({ code: "extension_package_collision" }), + ); + expect(plan.actions.filter((action) => action.id === "plugin:@acme/market-data")).toHaveLength( + 1, + ); + }); + + it("blocks a declared format that differs from canonical detection", async () => { + const { source, workspace } = await createPlanSource(); + const plan = await buildClawAddPlan({ + manifest: requireManifest({ schemaVersion: 1, agent: { id: "market-analyst" } }), + openClawProfile: { + schemaVersion: 1, + agent: {}, + extensions: [{ ...extension, format: "codex" }], + }, + source, + context: { + workspace, + packagePreflight: async () => ({ + ok: true, + action: "install", + integrity: `sha256:${"b".repeat(64)}`, + installId: "market-data", + detectedFormat: "claude", + mapped: ["skills"], + unavailable: [], + adapterIdentity: "openclaw/test", + }), + }, + }); + + expect(plan.extensions?.[0]?.blocked).toBe(true); + expect(plan.blockers).toContainEqual( + expect.objectContaining({ + code: "extension_format_mismatch", + path: "$.profiles.openclaw.extensions[0].format", + }), + ); + }); +}); diff --git a/src/claws/bootstrap.test.ts b/src/claws/bootstrap.test.ts index de7447f5136e..b090be92bdc6 100644 --- a/src/claws/bootstrap.test.ts +++ b/src/claws/bootstrap.test.ts @@ -139,7 +139,43 @@ describe("package-root BOOTSTRAP.md", () => { }); }); - it("keeps a failed pre-publication bootstrap seed removable", async () => { + it("seeds the consented package bootstrap before the agent config is published", async () => { + const root = await createPackage(); + const read = await readClawManifestFile(root); + if (!read.ok || !read.packageBootstrap) { + throw new Error("expected package bootstrap"); + } + const workspace = join(root, "workspace"); + const env = { OPENCLAW_STATE_DIR: join(root, "state") }; + const plan = await buildClawAddPlan({ + manifest: read.manifest, + clawMarkdownBody: read.clawMarkdownBody, + packageBootstrap: read.packageBootstrap, + source: read.source, + context: { workspace }, + }); + let config: OpenClawConfig = {}; + const order: string[] = []; + + const added = await applyClawAddPlan(plan, { + env, + nowMs: 1_000, + consentPlanIntegrity: plan.planIntegrity, + commitConfig: async (transform) => { + order.push("config-commit"); + config = transform(config); + }, + seedPackageBootstrap: async (seedPlan, seedOptions) => { + order.push("bootstrap-seed"); + return await seedClawPackageBootstrap(seedPlan, seedOptions); + }, + }); + + expect(added.status).toBe("complete"); + expect(order).toEqual(["bootstrap-seed", "config-commit"]); + }); + + it("keeps the agent unpublished and resumable when package bootstrap seeding fails", async () => { const root = await createPackage(); const read = await readClawManifestFile(root); if (!read.ok || !read.packageBootstrap) { @@ -174,15 +210,75 @@ describe("package-root BOOTSTRAP.md", () => { error: { code: "bootstrap_write_failed" }, }); expect(config).toEqual({}); + expect(readClawInstallRecord("bootstrap-worker", { env })?.status).toBe("workspace_ready"); expect(readWorkspaceStateSnapshot(workspace, { env }).setup.bootstrapSeededAt).toBeUndefined(); await expect(readFile(join(workspace, "BOOTSTRAP.md"), "utf8")).rejects.toThrow(); + + const resumed = await applyClawAddPlan(plan, { + env, + nowMs: 2_000, + consentPlanIntegrity: plan.planIntegrity, + commitConfig: async (transform) => { + config = transform(config); + }, + }); + + expect(resumed.status).toBe("complete"); + await expect(readFile(join(workspace, "BOOTSTRAP.md"), "utf8")).resolves.toContain( + "which repositories", + ); await expect(readClawStatus("bootstrap-worker", { env, config })).resolves.toMatchObject({ - records: [ - { - bootstrapState: "missing", - bootstrap: { state: "missing", path: "BOOTSTRAP.md" }, - }, - ], + records: [{ bootstrapState: "pending" }], + }); + }); + + it("recovers from a stock bootstrap seeded by a concurrent session", async () => { + const root = await createPackage(); + const read = await readClawManifestFile(root); + if (!read.ok || !read.packageBootstrap) { + throw new Error("expected package bootstrap"); + } + const workspace = join(root, "workspace"); + const env = { OPENCLAW_STATE_DIR: join(root, "state") }; + const plan = await buildClawAddPlan({ + manifest: read.manifest, + clawMarkdownBody: read.clawMarkdownBody, + packageBootstrap: read.packageBootstrap, + source: read.source, + context: { workspace }, + }); + let config: OpenClawConfig = {}; + + const added = await applyClawAddPlan(plan, { + env, + nowMs: 1_000, + consentPlanIntegrity: plan.planIntegrity, + commitConfig: async (transform) => { + config = transform(config); + }, + seedPackageBootstrap: async (seedPlan, seedOptions) => { + await writeFile(join(workspace, "BOOTSTRAP.md"), "# Stock onboarding\n", "utf8"); + return await seedClawPackageBootstrap(seedPlan, seedOptions); + }, + }); + + expect(added).toMatchObject({ status: "partial", configCommitted: false }); + expect(config).toEqual({}); + expect(readClawInstallRecord("bootstrap-worker", { env })?.status).toBe("workspace_ready"); + + await rm(join(workspace, "BOOTSTRAP.md")); + const resumed = await applyClawAddPlan(plan, { + env, + nowMs: 2_000, + consentPlanIntegrity: plan.planIntegrity, + commitConfig: async (transform) => { + config = transform(config); + }, + }); + + expect(resumed.status).toBe("complete"); + await expect(readClawStatus("bootstrap-worker", { env, config })).resolves.toMatchObject({ + records: [{ bootstrapState: "pending" }], }); }); diff --git a/src/claws/cron-update.test.ts b/src/claws/cron-update.test.ts index 671086ad2d27..95d6254735e4 100644 --- a/src/claws/cron-update.test.ts +++ b/src/claws/cron-update.test.ts @@ -82,6 +82,7 @@ function plan(actions: ClawUpdatePlan["actions"]): ClawUpdatePlan { }, actions, capabilityChanges: [], + readiness: { ready: true, requirements: [] }, blockers: [], diagnostics: [], }; diff --git a/src/claws/doctor.ts b/src/claws/doctor.ts index 07ab0c72a5b7..2f2502496bf0 100644 --- a/src/claws/doctor.ts +++ b/src/claws/doctor.ts @@ -145,6 +145,17 @@ function collectInstallFindings( ); } for (const pkg of record.packages) { + if (pkg.extensionCompatibility && pkg.extensionCompatibility.state !== "compatible") { + findings.push( + finding({ + message: `Claw extension ${JSON.stringify(pkg.extension?.id ?? pkg.ref)} has ${pkg.extensionCompatibility.state} host compatibility state${pkg.extensionCompatibility.message ? `: ${pkg.extensionCompatibility.message}` : "."}`, + path: `claws.${agentId}.extensions.${pkg.extension?.id ?? pkg.ref}`, + target: `${pkg.source}:${pkg.ref}@${pkg.version}`, + requirement: "Claw extensions should retain their consented canonical capability mapping", + fixHint: "Preview a Claw update before accepting the host's current extension mapping.", + }), + ); + } if (pkg.state === "present") { continue; } diff --git a/src/claws/export.test.ts b/src/claws/export.test.ts index 9f8446d66f38..b8d6433b30b8 100644 --- a/src/claws/export.test.ts +++ b/src/claws/export.test.ts @@ -6,6 +6,7 @@ import { useAutoCleanupTempDirTracker } from "../../test/helpers/temp-dir.js"; import { MAX_WORKSPACE_BOOTSTRAP_FILE_BYTES } from "../agents/workspace-bootstrap-read.js"; import type { McpServerConfig } from "../config/types.mcp.js"; import type { OpenClawConfig } from "../config/types.openclaw.js"; +import { PLUGIN_ARTIFACT_ADAPTER_IDENTITY } from "../plugins/install-artifact-inspection.js"; import { closeOpenClawStateDatabaseForTest } from "../state/openclaw-state-db.js"; import { applyClawAddPlan } from "./add.js"; import { exportClawAgent } from "./export.js"; @@ -300,6 +301,63 @@ describe("exportClawAgent", () => { await expect(readFile(join(out, "workspace", "SOUL.md"), "utf8")).rejects.toThrow(); }); + it("exports extension plugins into profile v1 without duplicating manifest packages", async () => { + const fixture = await installedFixture(); + const integrity = `sha256:${"b".repeat(64)}`; + const extension = { + id: "coding-tools", + format: "claude" as const, + detectedFormat: "claude" as const, + mapped: ["agents", "commands", "skills"], + unavailable: [], + adapterIdentity: PLUGIN_ARTIFACT_ADAPTER_IDENTITY, + }; + persistClawPackageRef( + fixture.plan, + { + kind: "plugin", + source: "clawhub", + ref: "@acme/coding-tools", + version: "1.2.3", + integrity, + extension, + }, + { env: fixture.env, relationship: "referenced" }, + ); + + const result = await exportClawAgent("worker", join(fixture.root, "exported-extension"), { + env: fixture.env, + config: fixture.config, + sourceMcpServers: fixture.sourceMcpServers, + packageDeps: { + resolvePlugin: async () => ({ + status: "found" as const, + pluginId: "coding-tools", + installedVersion: "1.2.3", + record: { source: "clawhub", integrity }, + }), + }, + }); + + expect(result.manifest.packages).toEqual([]); + expect(result.manifest.workspace.files).toContainEqual( + expect.objectContaining({ path: "reference/policy.md" }), + ); + expect(result.openClawProfile).toMatchObject({ + schemaVersion: 1, + extensions: [ + { + id: "coding-tools", + kind: "plugin", + format: "claude", + source: "clawhub", + ref: "@acme/coding-tools", + version: "1.2.3", + }, + ], + }); + }); + it("rejects modified managed content instead of silently creating a snapshot", async () => { const fixture = await installedFixture(); await writeFile(join(fixture.plan.agent.workspace, "SOUL.md"), "operator revision\n", "utf8"); diff --git a/src/claws/export.ts b/src/claws/export.ts index bbaba6be055b..3a613ef6310f 100644 --- a/src/claws/export.ts +++ b/src/claws/export.ts @@ -24,7 +24,9 @@ import { CLAW_SCHEMA_VERSION, type ClawManifest, type ClawMcpServer, + type ClawOpenClawExtension, type ClawOpenClawProfile, + type ClawPackagePreflight, } from "./types.js"; export const CLAW_EXPORT_RESULT_SCHEMA_VERSION = "openclaw.clawExportResult.v1" as const; @@ -76,7 +78,10 @@ function portableAgent(agent: AgentConfig, avatar: string | undefined): ClawMani }; } -function portableOpenClawProfile(agent: AgentConfig): ClawOpenClawProfile | undefined { +function portableOpenClawProfile( + agent: AgentConfig, + extensions: ClawOpenClawExtension[], +): ClawOpenClawProfile | undefined { const tools = { ...(agent.tools?.profile ? { profile: agent.tools.profile } : {}), ...(agent.tools?.allow?.length ? { allow: agent.tools.allow } : {}), @@ -160,7 +165,9 @@ function portableOpenClawProfile(agent: AgentConfig): ClawOpenClawProfile | unde } : {}), }; - return Object.keys(settings).length > 0 ? { schemaVersion: 1, agent: settings } : undefined; + return extensions.length > 0 || Object.keys(settings).length > 0 + ? { schemaVersion: 1, agent: settings, extensions } + : undefined; } function normalizedRelativePath(value: string): string { @@ -259,6 +266,7 @@ export async function exportClawAgent( options: OpenClawStateDatabaseOptions & { config: OpenClawConfig; packageDeps?: PackageRemovalDeps; + packagePreflight?: ClawPackagePreflight; sourceMcpServers?: Record>; }, ): Promise { @@ -305,11 +313,16 @@ export async function exportClawAgent( `Cannot export drifted managed files: ${driftedFiles.map((file) => `${file.path} (${file.state})`).join(", ")}.`, ); } - const driftedPackages = record.packages.filter((pkg) => pkg.state !== "present"); + const driftedPackages = record.packages.filter( + (pkg) => + pkg.state !== "present" || + (pkg.extensionCompatibility !== undefined && + pkg.extensionCompatibility.state !== "compatible"), + ); if (driftedPackages.length > 0) { throw new ClawExportError( "packages_drifted", - `Cannot export drifted packages: ${driftedPackages.map((pkg) => `${pkg.kind}:${pkg.ref}@${pkg.version} (${pkg.state})`).join(", ")}.`, + `Cannot export drifted packages: ${driftedPackages.map((pkg) => `${pkg.kind}:${pkg.ref}@${pkg.version} (${pkg.extensionCompatibility?.state ?? pkg.state})`).join(", ")}.`, ); } const unresolvedCronJobs = record.cronJobs.filter( @@ -384,27 +397,40 @@ export async function exportClawAgent( const configuredMcpServers = normalizeConfiguredMcpServers( options.sourceMcpServers ?? options.config.mcp?.servers, ); - const openClawProfile = portableOpenClawProfile(agent); + const extensions = record.packages + .filter((pkg) => pkg.extension) + .map((pkg) => ({ + id: pkg.extension!.id, + kind: "plugin" as const, + format: pkg.extension!.format, + source: pkg.source, + ref: pkg.ref, + version: pkg.version, + })) + .toSorted((left, right) => comparePortableText(left.id, right.id)); + const openClawProfile = portableOpenClawProfile(agent, extensions); const openClawProfilePath = "profiles/openclaw.yml"; const openClawProfileRaw = openClawProfile ? Buffer.from(stringifyYaml(openClawProfile)) : undefined; + const portablePackages = record.packages + .filter((pkg) => !pkg.extension) + .map((pkg) => ({ + kind: pkg.kind, + source: pkg.source, + ref: pkg.ref, + version: pkg.version, + })) + .toSorted((left, right) => { + const leftIdentity = `${left.kind}:${left.ref}:${left.version}`; + const rightIdentity = `${right.kind}:${right.ref}:${right.version}`; + return comparePortableText(leftIdentity, rightIdentity); + }); const manifest: ClawManifest = { schemaVersion: CLAW_SCHEMA_VERSION, agent: portableAgent(agent, avatar.source), workspace: { bootstrapFiles, files }, - packages: record.packages - .map((pkg) => ({ - kind: pkg.kind, - source: pkg.source, - ref: pkg.ref, - version: pkg.version, - })) - .toSorted((left, right) => { - const leftIdentity = `${left.kind}:${left.ref}:${left.version}`; - const rightIdentity = `${right.kind}:${right.ref}:${right.version}`; - return comparePortableText(leftIdentity, rightIdentity); - }), + packages: portablePackages, mcpServers: Object.fromEntries( record.mcpServers.map((ref) => [ ref.name, diff --git a/src/claws/lifecycle-state.test.ts b/src/claws/lifecycle-state.test.ts index 3d6a31e99900..789f93385f1b 100644 --- a/src/claws/lifecycle-state.test.ts +++ b/src/claws/lifecycle-state.test.ts @@ -195,6 +195,107 @@ describe("Claw status and remove", () => { }); }); + it("reports adapter identity drift for an installed extension without mutating provenance", async () => { + const current = await addFixture(); + const extension = { + id: "audit-tools", + format: "claude" as const, + detectedFormat: "claude" as const, + mapped: ["skills"], + unavailable: ["agents"], + adapterIdentity: "openclaw/previous", + }; + persistClawPackageRef( + current.plan, + { + kind: "plugin", + source: "clawhub", + ref: "audit", + version: "2.0.0", + integrity: packageIntegrity, + extension, + }, + { env: current.env, nowMs: 2, relationship: "referenced" }, + ); + + const status = await readClawStatus("worker", { + env: current.env, + config: current.getConfig(), + packageDeps: { + resolvePlugin: async () => ({ + status: "found" as const, + pluginId: "audit", + installedVersion: "2.0.0", + record: { source: "clawhub", integrity: packageIntegrity }, + }), + }, + }); + + expect(status.summary.driftedPackages).toBe(1); + expect(status.records[0]?.packages[0]).toMatchObject({ + state: "present", + extension, + extensionCompatibility: { + state: "drifted", + mapped: ["agents", "skills"], + unavailable: [], + adapterIdentity: "openclaw/v1", + }, + }); + expect(readClawPackageRefs({ env: current.env })[0]?.extension).toEqual(extension); + }); + + it("reports unavailable extension inspection separately from package drift", async () => { + const current = await addFixture(); + const extension = { + id: "audit-tools", + format: "claude" as const, + detectedFormat: "claude" as const, + mapped: ["skills"], + unavailable: ["agents"], + adapterIdentity: "openclaw/current", + }; + persistClawPackageRef( + current.plan, + { + kind: "plugin", + source: "clawhub", + ref: "audit", + version: "2.0.0", + integrity: packageIntegrity, + extension, + }, + { env: current.env, nowMs: 2, relationship: "referenced" }, + ); + + const status = await readClawStatus("worker", { + env: current.env, + config: current.getConfig(), + packageDeps: { + resolvePlugin: async () => ({ + status: "found" as const, + pluginId: "audit", + installedVersion: "2.0.0", + record: { source: "clawhub", integrity: packageIntegrity }, + }), + }, + packagePreflight: async () => ({ + ok: false, + code: "extension_unavailable", + message: "Canonical extension inspection is unavailable.", + }), + }); + + expect(status.summary).toMatchObject({ driftedPackages: 0, unavailableExtensions: 1 }); + expect(status.records[0]?.packages[0]).toMatchObject({ + state: "present", + extensionCompatibility: { + state: "unavailable", + message: "Canonical extension inspection is unavailable.", + }, + }); + }); + it("counts every non-complete root install as partial", async () => { const current = await fixture(); persistClawInstallRecord(current.plan, { env: current.env, status: "config_committed" }); @@ -835,6 +936,14 @@ describe("Claw status and remove", () => { config, packageDeps, }); + expect(plan.actions).toContainEqual( + expect.objectContaining({ + kind: "packageRef", + action: "release", + reason: expect.stringContaining("Claw add introduced this shared requirement"), + details: expect.objectContaining({ introducedByClawAdd: true }), + }), + ); await expect( applyClawRemovePlan(plan, { diff --git a/src/claws/lifecycle-status.ts b/src/claws/lifecycle-status.ts index bb0c369617b5..27f9e52430da 100644 --- a/src/claws/lifecycle-status.ts +++ b/src/claws/lifecycle-status.ts @@ -1,8 +1,14 @@ +import { stableStringify } from "@openclaw/normalization-core"; import { listAgentEntries } from "../agents/agent-scope.js"; import { getRuntimeConfig } from "../config/config.js"; import { normalizeConfiguredMcpServers } from "../config/mcp-config-normalize.js"; import { listConfiguredMcpServers } from "../config/mcp-config.js"; import type { OpenClawConfig } from "../config/types.openclaw.js"; +import { + inspectBundlePluginArtifact, + inspectNativePluginArtifact, + PLUGIN_ARTIFACT_ADAPTER_IDENTITY, +} from "../plugins/install-artifact-inspection.js"; import type { OpenClawStateDatabaseOptions } from "../state/openclaw-state-db.js"; import { readClawCronRefs, type PersistedClawCronRef } from "./cron.js"; import { digestClawAgentConfig } from "./lifecycle-config-removal.js"; @@ -30,8 +36,9 @@ import { readClawInstallRecords, readClawPackageRefs, type PersistedClawInstall, + type PersistedClawPackageRef, } from "./provenance.js"; -import { CLAW_OUTPUT_STABILITY } from "./types.js"; +import { CLAW_OUTPUT_STABILITY, type ClawPackagePreflight } from "./types.js"; import { readClawWorkspaceFiles } from "./workspace.js"; const CLAW_STATUS_SCHEMA_VERSION = "openclaw.clawStatus.v1" as const; @@ -40,6 +47,89 @@ type ClawMcpServerStatus = PersistedClawMcpServerRef & { state: "present" | "modified" | "missing" | "pending" | "failed"; }; +export type ClawPackageStatus = ClawPackageInspection & { + extensionCompatibility?: { + state: "compatible" | "drifted" | "unavailable"; + detectedFormat?: NonNullable["detectedFormat"]; + mapped: string[]; + unavailable: string[]; + adapterIdentity?: string; + message?: string; + }; +}; + +async function inspectClawPackageCompatibility(params: { + install: PersistedClawInstall; + packageRef: PersistedClawPackageRef; + packageDeps?: PackageRemovalDeps; + packagePreflight?: ClawPackagePreflight; +}): Promise { + const inspected: ClawPackageStatus = await inspectClawPackage( + params.install, + params.packageRef, + params.packageDeps, + ); + if (!params.packageRef.extension || inspected.state !== "present") { + return inspected; + } + let current: { + detectedFormat: NonNullable["detectedFormat"]; + mapped: string[]; + unavailable: string[]; + adapterIdentity: string; + }; + if (params.packagePreflight) { + const preflight = await params.packagePreflight(params.packageRef, params.install.workspace); + if (!preflight.ok) { + inspected.extensionCompatibility = { + state: "unavailable", + mapped: [], + unavailable: [], + message: preflight.message ?? "Canonical extension inspection is unavailable.", + }; + return inspected; + } + current = { + detectedFormat: preflight.detectedFormat!, + mapped: preflight.mapped ?? [], + unavailable: preflight.unavailable ?? [], + adapterIdentity: preflight.adapterIdentity!, + }; + } else { + // Status and doctor stay offline. Artifact identity was verified above, so the + // current adapter can recompute compatibility from persisted capability inventory. + // Update planning supplies packagePreflight and reports unavailable inspection. + const recorded = params.packageRef.extension; + const artifact = + recorded.detectedFormat === "openclaw" + ? inspectNativePluginArtifact() + : inspectBundlePluginArtifact({ + format: recorded.detectedFormat, + capabilities: [...recorded.mapped, ...recorded.unavailable], + }); + current = { + detectedFormat: recorded.detectedFormat, + mapped: artifact.mapped, + unavailable: artifact.unavailable, + adapterIdentity: PLUGIN_ARTIFACT_ADAPTER_IDENTITY, + }; + } + const recorded = { + detectedFormat: params.packageRef.extension.detectedFormat, + mapped: params.packageRef.extension.mapped, + unavailable: params.packageRef.extension.unavailable, + adapterIdentity: params.packageRef.extension.adapterIdentity, + }; + inspected.extensionCompatibility = { + state: stableStringify(current) === stableStringify(recorded) ? "compatible" : "drifted", + detectedFormat: current.detectedFormat, + mapped: current.mapped, + unavailable: current.unavailable, + adapterIdentity: current.adapterIdentity, + }; + return inspected; +} + export type ClawStatusRecord = { install: PersistedClawInstall; orphaned?: boolean; @@ -47,7 +137,7 @@ export type ClawStatusRecord = { bootstrapState: ClawBootstrapStatus["state"]; bootstrap: ClawBootstrapStatus; workspaceFiles: ClawManagedFileStatus[]; - packages: ClawPackageInspection[]; + packages: ClawPackageStatus[]; mcpServers: ClawMcpServerStatus[]; cronJobs: PersistedClawCronRef[]; }; @@ -66,6 +156,7 @@ type ClawStatusResult = { packageRefs: number; missingPackages: number; driftedPackages: number; + unavailableExtensions: number; incompletePackages: number; mcpServerRefs: number; driftedMcpServers: number; @@ -99,6 +190,7 @@ export async function readClawStatus( sourceMcpServers?: Record>; listMcpServers?: typeof listConfiguredMcpServers; packageDeps?: PackageRemovalDeps; + packagePreflight?: ClawPackagePreflight; } = {}, ): Promise { const config = options.config ?? getRuntimeConfig(); @@ -145,6 +237,7 @@ export async function readClawStatus( (install) => !target || install.agentId === target || install.claw.name === target, ); const records: ClawStatusRecord[] = []; + const packagePreflight = options.packagePreflight; for (const install of installs) { const agent = listAgentEntries(config).find((candidate) => candidate.id === install.agentId); const packageRefs = allPackageRefs.filter( @@ -172,8 +265,14 @@ export async function readClawStatus( bootstrap, workspaceFiles: await Promise.all(workspaceFiles.map(inspectClawWorkspaceFile)), packages: await Promise.all( - packageRefs.map((packageRef) => - inspectClawPackage(install, packageRef, options.packageDeps), + packageRefs.map( + async (packageRef) => + await inspectClawPackageCompatibility({ + install, + packageRef, + packageDeps: options.packageDeps, + packagePreflight, + }), ), ), mcpServers: (options.readOnly @@ -202,7 +301,15 @@ export async function readClawStatus( .filter((pkg) => pkg.state === "missing").length, driftedPackages: records .flatMap((record) => record.packages) - .filter((pkg) => pkg.state === "modified" || pkg.state === "ambiguous").length, + .filter( + (pkg) => + pkg.state === "modified" || + pkg.state === "ambiguous" || + pkg.extensionCompatibility?.state === "drifted", + ).length, + unavailableExtensions: records + .flatMap((record) => record.packages) + .filter((pkg) => pkg.extensionCompatibility?.state === "unavailable").length, incompletePackages: records .flatMap((record) => record.packages) .filter((pkg) => pkg.state === "incomplete").length, diff --git a/src/claws/lifecycle.ts b/src/claws/lifecycle.ts index 2783032aa394..a34846ebae44 100644 --- a/src/claws/lifecycle.ts +++ b/src/claws/lifecycle.ts @@ -8,6 +8,7 @@ import { resolvePathViaExistingAncestorSync } from "../infra/boundary-path.js"; import { assertNoSymlinkParents } from "../infra/fs-safe-advanced.js"; import { FsSafeError, root as fsSafeRoot, type Root } from "../infra/fs-safe.js"; import { resolveUserPath } from "../utils.js"; +import { findClawExtensionPackageCollisions, planClawExtensions } from "./application-plan.js"; import { digestClawMcpServer } from "./mcp.js"; import { clawManifestWorkspaceConflictsWithPath } from "./schema.js"; import { MAX_MANAGED_FILE_BYTES, MAX_MANAGED_WORKSPACE_BYTES } from "./source-limits.js"; @@ -22,7 +23,8 @@ import { type ClawManifest, type ClawLocalPrerequisite, type ClawOpenClawProfile, - type ClawPackage, + type ClawPackagePreflight, + type ClawPackagePreflightResult, type ClawSourceIdentity, type ClawWorkspaceSourceSnapshot, } from "./types.js"; @@ -48,20 +50,7 @@ export type ClawAddPlanContext = { existingWorkspacePaths?: Iterable; existingMcpServerNames?: Iterable; existingMcpServers?: Record>; - packagePreflight?: ( - pkg: ClawPackage, - workspace: string, - ) => Promise<{ - ok: boolean; - action?: "install" | "reuse"; - integrity?: string; - installId?: string; - warning?: string; - requirements?: ClawLocalPrerequisite[]; - installedVersion?: string; - code?: string; - message?: string; - }>; + packagePreflight?: ClawPackagePreflight; }; function canonicalWorkspacePath(value: string): string { @@ -468,7 +457,7 @@ export async function buildClawAddPlan(params: { } for (const pkg of params.manifest.packages) { - const preflight = context.packagePreflight + const preflight: ClawPackagePreflightResult = context.packagePreflight ? await context.packagePreflight(pkg, workspace) : { ok: false, @@ -491,7 +480,7 @@ export async function buildClawAddPlan(params: { actions.push({ kind: "package", id: `${pkg.kind}:${pkg.ref}`, - action: "install", + action: preflight.ok && preflight.action === "reuse" ? "reuse" : "install", target: `${pkg.source}:${pkg.ref}@${pkg.version}`, digest: preflight.integrity, details: { @@ -506,6 +495,13 @@ export async function buildClawAddPlan(params: { ? "present-exact" : "absent", ownerAction: preflight.action, + requirementState: !preflight.ok + ? "conflicting" + : preflight.action === "install" + ? "missing-installable" + : preflight.requirements && preflight.requirements.length > 0 + ? "setup-required" + : "satisfied", }, blocked: !preflight.ok, ...(diagnostic ? { reason: diagnostic.message } : {}), @@ -515,8 +511,11 @@ export async function buildClawAddPlan(params: { kind: "package", id: `${pkg.kind}:${pkg.ref}`, path: `packages.${pkg.kind}.${pkg.ref}`, - action: "install", - reason: "The Claw declares downloadable package content or executable code.", + action: preflight.ok && preflight.action === "reuse" ? "reuse" : "install", + reason: + preflight.ok && preflight.action === "reuse" + ? "The Claw requires access to an existing package capability." + : "The Claw requires downloadable package content or executable code.", effect: { kind: pkg.kind, source: pkg.source, @@ -530,6 +529,28 @@ export async function buildClawAddPlan(params: { ); } + const extensionPlan = await planClawExtensions({ + extensions: params.openClawProfile?.extensions ?? [], + workspace, + packagePreflight: context.packagePreflight, + }); + const extensions = extensionPlan.extensions; + const extensionCollisions = findClawExtensionPackageCollisions({ + packages: params.manifest.packages, + extensions: params.openClawProfile?.extensions ?? [], + }); + const collisionIndexes = new Set(extensionCollisions.map(({ index }) => index)); + blockers.push(...extensionCollisions.map(({ diagnostic }) => diagnostic)); + for (const [index, action] of extensionPlan.actions.entries()) { + if (collisionIndexes.has(index)) { + continue; + } + actions.push(action); + } + capabilityChanges.push(...extensionPlan.capabilityChanges); + readinessRequirements.push(...extensionPlan.requirements); + blockers.push(...extensionPlan.blockers); + const existingMcpServerNames = new Set(context.existingMcpServerNames ?? []); for (const [name, server] of Object.entries(params.manifest.mcpServers)) { const existingServer = context.existingMcpServers?.[name]; @@ -631,6 +652,7 @@ export async function buildClawAddPlan(params: { actions, capabilityChanges, blockers, + extensions, }), ) .digest("hex")}`; @@ -670,6 +692,7 @@ export async function buildClawAddPlan(params: { ready: readinessRequirements.length === 0, requirements: readinessRequirements, }, + extensions, blockers, diagnostics: params.diagnostics ?? [], }; diff --git a/src/claws/mcp-update.test.ts b/src/claws/mcp-update.test.ts index 305103ba8a22..266fbdd78036 100644 --- a/src/claws/mcp-update.test.ts +++ b/src/claws/mcp-update.test.ts @@ -66,6 +66,7 @@ function plan(actions: ClawUpdatePlan["actions"]): ClawUpdatePlan { }, actions, capabilityChanges: [], + readiness: { ready: true, requirements: [] }, blockers: [], diagnostics: [], }; diff --git a/src/claws/openclaw-profile.test.ts b/src/claws/openclaw-profile.test.ts index 95756c63412e..283d9838cd75 100644 --- a/src/claws/openclaw-profile.test.ts +++ b/src/claws/openclaw-profile.test.ts @@ -177,7 +177,7 @@ describe("OpenClaw profile reader", () => { }); }); - it("fails closed for a retired metadata profile pointer", async () => { + it("fails closed for an escaping metadata profile pointer", async () => { const root = tempDirs.make("openclaw-claw-profile-pointer-"); const path = join(root, "openclaw.claw.json"); await writeFile( @@ -197,13 +197,117 @@ describe("OpenClaw profile reader", () => { ok: false, diagnostics: [ expect.objectContaining({ - code: "legacy_openclaw_profile_pointer", + code: "invalid_openclaw_profile_path", path: "$.metadata.openclaw.config", }), ], }); }); + it("still reads the deprecated metadata profile pointer with a warning", async () => { + const root = tempDirs.make("openclaw-claw-profile-legacy-pointer-"); + await mkdir(join(root, "profiles")); + const path = join(root, "openclaw.claw.json"); + await writeFile( + path, + JSON.stringify({ + schemaVersion: 1, + agent: { id: "triage" }, + metadata: { "openclaw.config": "profiles/triage.openclaw.yml" }, + }), + "utf8", + ); + await writeFile( + join(root, "profiles", "triage.openclaw.yml"), + "schemaVersion: 1\nagent:\n tools:\n profile: coding\n", + "utf8", + ); + + const result = await readClawManifestFile(path); + + expect(result).toMatchObject({ + ok: true, + openClawProfile: { schemaVersion: 1, agent: { tools: { profile: "coding" } } }, + }); + if (!result.ok) { + throw new Error("expected the deprecated pointer to keep resolving"); + } + expect(result.diagnostics).toContainEqual( + expect.objectContaining({ + level: "warning", + code: "deprecated_openclaw_profile_pointer", + path: "$.metadata.openclaw.config", + }), + ); + expect(result.diagnostics.some((entry) => entry.level === "error")).toBe(false); + }); + + it("accepts a deprecated pointer that already targets the conventional profile", async () => { + const root = tempDirs.make("openclaw-claw-profile-legacy-conventional-"); + await mkdir(join(root, "profiles")); + const path = join(root, "openclaw.claw.json"); + await writeFile( + path, + JSON.stringify({ + schemaVersion: 1, + agent: { id: "triage" }, + metadata: { "openclaw.config": "profiles/openclaw.yml" }, + }), + "utf8", + ); + await writeFile( + join(root, "profiles", "openclaw.yml"), + "schemaVersion: 1\nagent:\n tools:\n profile: coding\n", + "utf8", + ); + + const result = await readClawManifestFile(path); + + expect(result).toMatchObject({ ok: true, openClawProfile: { schemaVersion: 1 } }); + }); + + it("fails closed when a deprecated pointer diverges from the conventional profile", async () => { + const root = tempDirs.make("openclaw-claw-profile-conflict-"); + await mkdir(join(root, "profiles")); + const path = join(root, "openclaw.claw.json"); + await writeFile( + path, + JSON.stringify({ + schemaVersion: 1, + agent: { id: "triage" }, + metadata: { "openclaw.config": "profiles/other.openclaw.yml" }, + }), + "utf8", + ); + await writeFile(join(root, "profiles", "openclaw.yml"), "schemaVersion: 1\n", "utf8"); + await writeFile(join(root, "profiles", "other.openclaw.yml"), "schemaVersion: 1\n", "utf8"); + + const result = await readClawManifestFile(path); + + expect(result).toMatchObject({ + ok: false, + diagnostics: [ + expect.objectContaining({ + code: "conflicting_openclaw_profile_pointer", + path: "$.metadata.openclaw.config", + }), + ], + }); + }); + + it("keeps the shipped pointer-based fixtures resolvable", async () => { + const result = await readClawManifestFile("src/claws/fixtures/incident-response.claw.json"); + + expect(result).toMatchObject({ + ok: true, + openClawProfile: { schemaVersion: 1, agent: { tools: { deny: ["exec", "browser"] } } }, + }); + if (!result.ok) { + throw new Error("expected the shipped fixture to remain valid"); + } + expect(result.diagnostics.some((entry) => entry.level === "error")).toBe(false); + }); + it("does not inspect profiles owned by other harnesses", async () => { const root = tempDirs.make("openclaw-claw-foreign-profile-"); await mkdir(join(root, "profiles")); diff --git a/src/claws/openclaw-profile.ts b/src/claws/openclaw-profile.ts index 2ddae1a2cf45..33bb5cb8fdab 100644 --- a/src/claws/openclaw-profile.ts +++ b/src/claws/openclaw-profile.ts @@ -1,16 +1,24 @@ // Safe loader for the conventional package-local OpenClaw profile. import { isScalar, parseDocument, visit } from "yaml"; import { FsSafeError, root as fsSafeRoot } from "../infra/fs-safe.js"; +import { isSafeClawRelativePath } from "./schema-portability.js"; import { parseClawOpenClawProfile } from "./schema.js"; import type { ClawDiagnostic, ClawOpenClawProfile } from "./types.js"; const MAX_PROFILE_BYTES = 256 * 1024; const CLAW_PROFILE_PATH = "profiles/openclaw.yml"; +const LEGACY_PROFILE_POINTER_KEY = "openclaw.config"; +const LEGACY_PROFILE_POINTER_PATH = "$.metadata.openclaw.config"; +const CONVENTIONAL_PROFILE_PATH = "$.profiles.openclaw"; function diagnostic(code: string, message: string, path = "$"): ClawDiagnostic { return { level: "error", code, phase: "parse", path, message }; } +function warning(code: string, message: string, path: string): ClawDiagnostic { + return { level: "warning", code, phase: "parse", path, message }; +} + function parseProfileYaml( raw: string, path: string, @@ -82,28 +90,74 @@ async function readProfileFile(packageRoot: string, path: string): Promise; }): Promise< - | { ok: true; profile?: ClawOpenClawProfile; raw?: Buffer; path?: string } + | { + ok: true; + profile?: ClawOpenClawProfile; + raw?: Buffer; + path?: string; + diagnostics?: ClawDiagnostic[]; + } | { ok: false; diagnostics: ClawDiagnostic[] } > { - if (Object.hasOwn(params.metadata ?? {}, "openclaw.config")) { - return { - ok: false, - diagnostics: [ - diagnostic( - "legacy_openclaw_profile_pointer", - "metadata.openclaw.config is no longer supported; move the profile to profiles/openclaw.yml and remove the metadata entry.", - "$.metadata.openclaw.config", - ), - ], - }; - } - const declaredPath = CLAW_PROFILE_PATH; const packageFiles = await fsSafeRoot(params.packageRoot); - if (!(await packageFiles.exists(declaredPath))) { + const conventionalExists = await packageFiles.exists(CLAW_PROFILE_PATH); + const legacyPointer = params.metadata?.[LEGACY_PROFILE_POINTER_KEY]; + const diagnostics: ClawDiagnostic[] = []; + let declaredPath = CLAW_PROFILE_PATH; + let diagnosticPath = CONVENTIONAL_PROFILE_PATH; + + if (legacyPointer !== undefined) { + if ( + legacyPointer.includes("\\") || + !isSafeClawRelativePath(legacyPointer) || + !/\.ya?ml$/i.test(legacyPointer) + ) { + return { + ok: false, + diagnostics: [ + diagnostic( + "invalid_openclaw_profile_path", + `metadata.${LEGACY_PROFILE_POINTER_KEY} must reference a forward-slash package-relative .yml or .yaml file.`, + LEGACY_PROFILE_POINTER_PATH, + ), + ], + }; + } + if (conventionalExists && legacyPointer !== CLAW_PROFILE_PATH) { + return { + ok: false, + diagnostics: [ + diagnostic( + "conflicting_openclaw_profile_pointer", + `metadata.${LEGACY_PROFILE_POINTER_KEY} references ${legacyPointer} while ${CLAW_PROFILE_PATH} also exists; keep only ${CLAW_PROFILE_PATH}.`, + LEGACY_PROFILE_POINTER_PATH, + ), + ], + }; + } + declaredPath = legacyPointer; + diagnosticPath = LEGACY_PROFILE_POINTER_PATH; + diagnostics.push( + warning( + "deprecated_openclaw_profile_pointer", + `metadata.${LEGACY_PROFILE_POINTER_KEY} is deprecated; move the profile to ${CLAW_PROFILE_PATH} and remove the metadata entry.`, + LEGACY_PROFILE_POINTER_PATH, + ), + ); + } else if (!conventionalExists) { return { ok: true }; } @@ -129,7 +183,7 @@ export async function readClawOpenClawProfile(params: { : tooLarge ? `The OpenClaw profile exceeds ${MAX_PROFILE_BYTES} bytes.` : `Could not read ${declaredPath}: ${(error as Error).message}`, - "$.profiles.openclaw", + diagnosticPath, ), ], }; @@ -145,9 +199,15 @@ export async function readClawOpenClawProfile(params: { ok: false, diagnostics: parsed.diagnostics.map((entry) => ({ ...entry, - path: `$.profiles.openclaw${entry.path.slice(1)}`, + path: `${diagnosticPath}${entry.path.slice(1)}`, })), }; } - return { ok: true, profile: parsed.profile, raw, path: declaredPath }; + return { + ok: true, + profile: parsed.profile, + raw, + path: declaredPath, + ...(diagnostics.length > 0 ? { diagnostics } : {}), + }; } diff --git a/src/claws/package-extension-provenance.ts b/src/claws/package-extension-provenance.ts new file mode 100644 index 000000000000..c46a0265b1dd --- /dev/null +++ b/src/claws/package-extension-provenance.ts @@ -0,0 +1,113 @@ +import type { ClawAppliedExtension, ClawPackage } from "./types.js"; + +export const CLAW_PACKAGE_REF_SCHEMA_VERSION = "openclaw.clawPackageRef.v1" as const; +export type ClawPackageRefStatus = "pending" | "complete" | "failed" | "rolled_back"; +export type ClawPackageRelationship = "managed" | "referenced"; +export type ClawPackageOrigin = "claw-introduced" | "pre-existing"; + +export type PersistedClawPackageRef = { + schemaVersion: typeof CLAW_PACKAGE_REF_SCHEMA_VERSION; + agentId: string; + clawName: string; + kind: ClawPackage["kind"]; + source: ClawPackage["source"]; + ref: string; + version: string; + integrity: string; + status: ClawPackageRefStatus; + relationship: ClawPackageRelationship; + origin: ClawPackageOrigin; + independentOwner: boolean; + extension?: ClawAppliedExtension; + installedAtMs: number; + updatedAtMs: number; +}; + +export type PackageRefRow = { + schema_version: string; + agent_id: string; + claw_name: string; + package_kind: ClawPackage["kind"]; + package_source: ClawPackage["source"]; + package_ref: string; + package_version: string; + package_integrity: string; + package_status: ClawPackageRefStatus; + relationship: ClawPackageRelationship; + origin: ClawPackageOrigin; + independent_owner: number | bigint; + extension_id: string | null; + extension_format: ClawAppliedExtension["format"] | null; + extension_detected_format: ClawAppliedExtension["detectedFormat"] | null; + extension_mapped_json: string | null; + extension_unavailable_json: string | null; + extension_adapter_identity: string | null; + installed_at_ms: number | bigint; + updated_at_ms: number | bigint; +}; + +function parsePackageRefExtension(row: PackageRefRow): ClawAppliedExtension | undefined { + const values = [ + row.extension_id, + row.extension_format, + row.extension_detected_format, + row.extension_mapped_json, + row.extension_unavailable_json, + row.extension_adapter_identity, + ]; + if (values.every((value) => value === null)) { + return undefined; + } + if (values.some((value) => value === null)) { + throw new Error( + `Claw package reference ${row.package_kind}:${row.package_ref} has incomplete extension provenance.`, + ); + } + const formats = new Set(["openclaw", "claude", "codex", "cursor"]); + if (!formats.has(row.extension_format!) || !formats.has(row.extension_detected_format!)) { + throw new Error( + `Claw package reference ${row.package_kind}:${row.package_ref} has unsupported extension format provenance.`, + ); + } + const mapped = JSON.parse(row.extension_mapped_json!) as unknown; + const unavailable = JSON.parse(row.extension_unavailable_json!) as unknown; + if ( + !Array.isArray(mapped) || + !mapped.every((value) => typeof value === "string") || + !Array.isArray(unavailable) || + !unavailable.every((value) => typeof value === "string") + ) { + throw new Error( + `Claw package reference ${row.package_kind}:${row.package_ref} has invalid extension inventory provenance.`, + ); + } + return { + id: row.extension_id!, + format: row.extension_format!, + detectedFormat: row.extension_detected_format!, + mapped, + unavailable, + adapterIdentity: row.extension_adapter_identity!, + }; +} + +export function rowToPackageRef(row: PackageRefRow): PersistedClawPackageRef { + const extension = parsePackageRefExtension(row); + return { + schemaVersion: CLAW_PACKAGE_REF_SCHEMA_VERSION, + agentId: row.agent_id, + clawName: row.claw_name, + kind: row.package_kind, + source: row.package_source, + ref: row.package_ref, + version: row.package_version, + integrity: row.package_integrity, + status: row.package_status, + relationship: row.relationship, + origin: row.origin, + independentOwner: Number(row.independent_owner) === 1, + ...(extension ? { extension } : {}), + installedAtMs: Number(row.installed_at_ms), + updatedAtMs: Number(row.updated_at_ms), + }; +} diff --git a/src/claws/package-remove-plan.ts b/src/claws/package-remove-plan.ts index 8e72e8bf66be..ab96e13c5dc9 100644 --- a/src/claws/package-remove-plan.ts +++ b/src/claws/package-remove-plan.ts @@ -52,6 +52,7 @@ export function projectClawPackageRemovePlan(params: { status: pkg.status, relationship: pkg.relationship, origin: pkg.origin, + introducedByClawAdd: pkg.origin === "claw-introduced", independentOwner: pkg.independentOwner, affectedClawAgentIds: decision.affectedClawAgentIds, cleanupMode: params.cleanup?.mode ?? "retain", diff --git a/src/claws/package-remove.test.ts b/src/claws/package-remove.test.ts index 11996f109d6e..139683904e42 100644 --- a/src/claws/package-remove.test.ts +++ b/src/claws/package-remove.test.ts @@ -115,7 +115,8 @@ describe("Claw package removal", () => { expect(decisions).toMatchObject([ { action: "retain", - reason: "Referenced resources are retained unless a cleanup mode selects them.", + reason: + "Claw add introduced this shared requirement; removal releases its dependency edge and retains the artifact. Use its canonical owner separately to uninstall it.", }, ]); }); @@ -306,7 +307,8 @@ describe("Claw package removal", () => { expect(decisions).toMatchObject([ { action: "retain", - reason: "Referenced resources are retained unless a cleanup mode selects them.", + reason: + "Claw add introduced this shared requirement; removal releases its dependency edge and retains the artifact. Use its canonical owner separately to uninstall it.", }, ]); }); @@ -327,7 +329,8 @@ describe("Claw package removal", () => { expect(decisions).toMatchObject([ { action: "retain", - reason: "Referenced resources are retained unless a cleanup mode selects them.", + reason: + "Claw add introduced this shared requirement; removal releases its dependency edge and retains the artifact. Use its canonical owner separately to uninstall it.", }, ]); }); @@ -353,7 +356,8 @@ describe("Claw package removal", () => { expect(decisions).toMatchObject([ { action: "retain", - reason: "Referenced resources are retained unless a cleanup mode selects them.", + reason: + "Claw add introduced this shared requirement; removal releases its dependency edge and retains the artifact. Use its canonical owner separately to uninstall it.", }, ]); }); diff --git a/src/claws/package-remove.ts b/src/claws/package-remove.ts index 6b619f865669..1f2d9a5e7930 100644 --- a/src/claws/package-remove.ts +++ b/src/claws/package-remove.ts @@ -262,7 +262,11 @@ export async function planClawPackageRemovals( continue; } if (!managedCleanup && !explicitlySelected && cleanup.mode !== "remove-if-unused") { - retain("Referenced resources are retained unless a cleanup mode selects them."); + retain( + packageRef.origin === "claw-introduced" + ? "Claw add introduced this shared requirement; removal releases its dependency edge and retains the artifact. Use its canonical owner separately to uninstall it." + : "Referenced resources are retained unless a separate cleanup mode selects them.", + ); continue; } if (!explicitlySelected && affectedClawAgentIds.length > 0) { diff --git a/src/claws/package-resume.test.ts b/src/claws/package-resume.test.ts new file mode 100644 index 000000000000..def44176882d --- /dev/null +++ b/src/claws/package-resume.test.ts @@ -0,0 +1,160 @@ +import { access, readFile } from "node:fs/promises"; +import { join } from "node:path"; +import { afterEach, describe, expect, it } from "vitest"; +import { useAutoCleanupTempDirTracker } from "../../test/helpers/temp-dir.js"; +import { + closeOpenClawStateDatabaseForTest, + openOpenClawStateDatabase, +} from "../state/openclaw-state-db.js"; +import { + findResumableIntroducedPluginRequirement, + readClawResumeStateReadOnly, +} from "./package-resume.js"; +import type { PersistedClawPackageRef } from "./provenance.js"; + +const tempDirs = useAutoCleanupTempDirTracker(afterEach); + +afterEach(() => closeOpenClawStateDatabaseForTest()); + +/** Reproduces a current-main same-version database that predates the additive columns. */ +function createBaseShapeClawState(env: { OPENCLAW_STATE_DIR: string }): string { + const database = openOpenClawStateDatabase({ env }); + const databasePath = database.path; + database.db.exec(` + INSERT INTO claw_installs ( + agent_id, schema_version, source_kind, claw_name, claw_version, package_root, + manifest_path, integrity_kind, integrity, source_byte_length, manifest_schema_version, + plan_integrity, workspace, agent_config_digest, agent_owned_paths_json, status, + added_at_ms, updated_at_ms + ) VALUES ( + 'incident-2', 'openclaw.clawInstallRecord.v1', 'package', 'incident-claw', '1.0.0', + '/packages/incident', '/packages/incident/CLAW.md', 'artifact', 'sha256:aa', 10, 1, + 'sha256:bb', '/workspaces/incident', 'sha256:cc', '[]', 'config_committed', 1000, 2000 + ); + INSERT INTO claw_package_refs ( + agent_id, package_kind, package_source, package_ref, package_version, package_integrity, + schema_version, claw_name, package_status, relationship, origin, independent_owner, + installed_at_ms, updated_at_ms + ) VALUES ( + 'incident-2', 'plugin', 'clawhub', '@owner/audit', '2.0.1', + 'sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', + 'openclaw.clawPackageRef.v1', 'incident-claw', 'complete', 'referenced', + 'claw-introduced', 0, 1000, 2000 + ); + ALTER TABLE claw_installs DROP COLUMN bootstrap_source_path; + ALTER TABLE claw_installs DROP COLUMN bootstrap_content_digest; + ALTER TABLE claw_package_refs DROP COLUMN extension_id; + ALTER TABLE claw_package_refs DROP COLUMN extension_format; + ALTER TABLE claw_package_refs DROP COLUMN extension_detected_format; + ALTER TABLE claw_package_refs DROP COLUMN extension_mapped_json; + ALTER TABLE claw_package_refs DROP COLUMN extension_unavailable_json; + ALTER TABLE claw_package_refs DROP COLUMN extension_adapter_identity; + `); + closeOpenClawStateDatabaseForTest(); + return databasePath; +} + +const integrity = "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; +const pkg = { + kind: "plugin" as const, + source: "clawhub" as const, + ref: "@owner/audit", + version: "2.0.1", +}; +const preflight = { + ok: true as const, + action: "reuse" as const, + integrity, + installedIntegrity: integrity, + installedAt: new Date(1_500).toISOString(), + detectedFormat: "claude" as const, + mapped: ["commands", "skills"], + unavailable: ["agents"], + adapterIdentity: "openclaw/v1", +}; +const ref: PersistedClawPackageRef = { + schemaVersion: "openclaw.clawPackageRef.v1", + agentId: "incident-2", + clawName: "incident-claw", + ...pkg, + integrity, + status: "complete", + relationship: "referenced", + origin: "claw-introduced", + independentOwner: false, + extension: { + id: "audit-tools", + format: "claude", + detectedFormat: "claude", + mapped: ["commands", "skills"], + unavailable: ["agents"], + adapterIdentity: "openclaw/v1", + }, + installedAtMs: 1_000, + updatedAtMs: 2_000, +}; + +describe("findResumableIntroducedPluginRequirement", () => { + it("recognizes an exact retained requirement from the incomplete attempt", () => { + expect( + findResumableIntroducedPluginRequirement({ + agentId: "incident-2", + pkg, + preflight, + refs: [ref], + }), + ).toEqual(ref); + }); + + it("rejects independently owned and newer plugin installations", () => { + expect( + findResumableIntroducedPluginRequirement({ + agentId: "incident-2", + pkg, + preflight, + refs: [{ ...ref, independentOwner: true }], + }), + ).toBeUndefined(); + expect( + findResumableIntroducedPluginRequirement({ + agentId: "incident-2", + pkg, + preflight: { ...preflight, installedAt: new Date(3_000).toISOString() }, + refs: [ref], + }), + ).toBeUndefined(); + }); + + it("rejects changed extension capability mappings", () => { + expect( + findResumableIntroducedPluginRequirement({ + agentId: "incident-2", + pkg, + preflight: { ...preflight, mapped: ["skills"] }, + refs: [ref], + }), + ).toBeUndefined(); + }); + it("does not create a state database while checking for a resumable preview", async () => { + const databasePath = join(tempDirs.make("openclaw-claw-resume-"), "missing.sqlite"); + + await expect( + readClawResumeStateReadOnly("incident-2", { path: databasePath }), + ).resolves.toBeUndefined(); + await expect(access(databasePath)).rejects.toThrow(); + }); + + it("previews a same-version base-shape database without mutating it", async () => { + const env = { OPENCLAW_STATE_DIR: tempDirs.make("openclaw-claw-resume-base-") }; + const databasePath = createBaseShapeClawState(env); + const before = await readFile(databasePath); + + const state = await readClawResumeStateReadOnly("incident-2", { path: databasePath }); + + expect(state?.record).toMatchObject({ agentId: "incident-2", status: "config_committed" }); + expect(state?.record.bootstrap).toBeUndefined(); + expect(state?.packageRefs).toHaveLength(1); + expect(state?.packageRefs[0]?.extension).toBeUndefined(); + await expect(readFile(databasePath)).resolves.toEqual(before); + }); +}); diff --git a/src/claws/package-resume.ts b/src/claws/package-resume.ts new file mode 100644 index 000000000000..302e336de482 --- /dev/null +++ b/src/claws/package-resume.ts @@ -0,0 +1,117 @@ +import { stableStringify } from "@openclaw/normalization-core"; +import { normalizeClawHubSha256Integrity } from "../infra/clawhub.js"; +import { + openExistingOpenClawStateDatabaseReadOnly, + type OpenClawStateDatabaseOptions, +} from "../state/openclaw-state-db.js"; +import { + readClawInstallRecordFromDatabase, + readClawPackageRefs, + type PersistedClawInstall, + type PersistedClawPackageRef, +} from "./provenance.js"; +import type { ClawPackage, ClawPackagePreflightResult } from "./types.js"; + +function ownerInstallIsNewerThanRef( + installedAt: string | undefined, + ref: PersistedClawPackageRef, +): boolean { + const timestamp = Date.parse(installedAt ?? ""); + return Number.isFinite(timestamp) && timestamp > ref.updatedAtMs; +} + +function persistedExtensionMatchesPreflight( + ref: PersistedClawPackageRef, + preflight: ClawPackagePreflightResult, +): boolean { + if (!ref.extension) { + return true; + } + if (!preflight.ok) { + return false; + } + return ( + stableStringify({ + detectedFormat: ref.extension.detectedFormat, + mapped: ref.extension.mapped, + unavailable: ref.extension.unavailable, + adapterIdentity: ref.extension.adapterIdentity, + }) === + stableStringify({ + detectedFormat: preflight.detectedFormat, + mapped: preflight.mapped ?? [], + unavailable: preflight.unavailable ?? [], + adapterIdentity: preflight.adapterIdentity, + }) + ); +} + +export function findResumableIntroducedPluginRequirement(params: { + agentId: string; + pkg: ClawPackage; + preflight: ClawPackagePreflightResult; + refs: readonly PersistedClawPackageRef[]; + expectedIntegrity?: string; +}): PersistedClawPackageRef | undefined { + if (params.pkg.kind !== "plugin" || !params.preflight.ok || params.preflight.action !== "reuse") { + return undefined; + } + const expectedRawIntegrity = params.expectedIntegrity ?? params.preflight.integrity; + if (!expectedRawIntegrity || !params.preflight.installedIntegrity) { + return undefined; + } + const expectedIntegrity = normalizeClawHubSha256Integrity(expectedRawIntegrity); + const installedIntegrity = normalizeClawHubSha256Integrity(params.preflight.installedIntegrity); + if (!expectedIntegrity || installedIntegrity !== expectedIntegrity) { + return undefined; + } + const ref = params.refs.find( + (candidate) => + candidate.agentId === params.agentId && + candidate.kind === params.pkg.kind && + candidate.source === params.pkg.source && + candidate.ref === params.pkg.ref && + candidate.version === params.pkg.version && + normalizeClawHubSha256Integrity(candidate.integrity) === expectedIntegrity && + candidate.status === "complete" && + candidate.relationship === "referenced" && + candidate.origin === "claw-introduced" && + !candidate.independentOwner && + persistedExtensionMatchesPreflight(candidate, params.preflight), + ); + return ref && !ownerInstallIsNewerThanRef(params.preflight.installedAt, ref) ? ref : undefined; +} + +export async function readClawResumeStateReadOnly( + agentId: string, + options: OpenClawStateDatabaseOptions = {}, +): Promise< + | { + record: PersistedClawInstall; + packageRefs: PersistedClawPackageRef[]; + } + | undefined +> { + const database = await openExistingOpenClawStateDatabaseReadOnly(options); + if (!database) { + return undefined; + } + try { + const hasInstallTable = database.db + .prepare("SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = 'claw_installs'") + .get(); + if (!hasInstallTable) { + return undefined; + } + const record = readClawInstallRecordFromDatabase(database.db, agentId); + if (!record) { + return undefined; + } + return { + record, + packageRefs: readClawPackageRefs({ ...options, database, readOnly: true, agentId }), + }; + } finally { + database.walMaintenance.close(); + } +} diff --git a/src/claws/package-setup-requirements.ts b/src/claws/package-setup-requirements.ts new file mode 100644 index 000000000000..f5a92933c4ee --- /dev/null +++ b/src/claws/package-setup-requirements.ts @@ -0,0 +1,35 @@ +import type { PluginManifestSetup } from "../plugins/manifest.js"; +import { resolveLocalProviderAuthEvidence } from "../secrets/provider-auth-evidence.js"; +import type { ClawLocalPrerequisite } from "./types.js"; + +export function resolveClawPluginSetupRequirements(params: { + pluginId: string; + setup?: PluginManifestSetup; + env: NodeJS.ProcessEnv; +}): ClawLocalPrerequisite[] { + const providers = params.setup?.providers ?? []; + const hasConfiguredProvider = providers.some( + (provider) => + (provider.envVars ?? []).some((name) => Boolean(params.env[name]?.trim())) || + resolveLocalProviderAuthEvidence(provider.authEvidence, params.env), + ); + if (hasConfiguredProvider) { + return []; + } + return providers.flatMap((provider) => { + const envVars = provider.envVars ?? []; + const authEvidence = provider.authEvidence ?? []; + if (envVars.length === 0 && authEvidence.length === 0) { + return []; + } + return [ + { + kind: "plugin-setup" as const, + plugin: params.pluginId, + provider: provider.id, + envVars, + authMethods: provider.authMethods ?? [], + }, + ]; + }); +} diff --git a/src/claws/package-update-provenance.ts b/src/claws/package-update-provenance.ts index 74de84195d99..56c35f5ec527 100644 --- a/src/claws/package-update-provenance.ts +++ b/src/claws/package-update-provenance.ts @@ -7,7 +7,24 @@ import { import type { PersistedClawPackageRef } from "./provenance.js"; export function digestClawPackageRef(ref: PersistedClawPackageRef): string { - return `sha256:${createHash("sha256").update(stableStringify(ref)).digest("hex")}`; + const persisted = { + schemaVersion: ref.schemaVersion, + agentId: ref.agentId, + clawName: ref.clawName, + kind: ref.kind, + source: ref.source, + ref: ref.ref, + version: ref.version, + integrity: ref.integrity, + status: ref.status, + relationship: ref.relationship, + origin: ref.origin, + independentOwner: ref.independentOwner, + ...(ref.extension ? { extension: ref.extension } : {}), + installedAtMs: ref.installedAtMs, + updatedAtMs: ref.updatedAtMs, + }; + return `sha256:${createHash("sha256").update(stableStringify(persisted)).digest("hex")}`; } export function replaceClawPackageRefExpected( @@ -36,6 +53,12 @@ export function replaceClawPackageRefExpected( AND relationship = @relationship AND origin = @origin AND independent_owner = @independent_owner + AND extension_id IS @extension_id + AND extension_format IS @extension_format + AND extension_detected_format IS @extension_detected_format + AND extension_mapped_json IS @extension_mapped_json + AND extension_unavailable_json IS @extension_unavailable_json + AND extension_adapter_identity IS @extension_adapter_identity AND installed_at_ms = @installed_at_ms AND updated_at_ms = @updated_at_ms`, ) @@ -52,6 +75,16 @@ export function replaceClawPackageRefExpected( relationship: expected.relationship, origin: expected.origin, independent_owner: expected.independentOwner ? 1 : 0, + extension_id: expected.extension?.id ?? null, + extension_format: expected.extension?.format ?? null, + extension_detected_format: expected.extension?.detectedFormat ?? null, + extension_mapped_json: expected.extension + ? JSON.stringify(expected.extension.mapped) + : null, + extension_unavailable_json: expected.extension + ? JSON.stringify(expected.extension.unavailable) + : null, + extension_adapter_identity: expected.extension?.adapterIdentity ?? null, installed_at_ms: expected.installedAtMs, updated_at_ms: expected.updatedAtMs, }); @@ -80,11 +113,15 @@ export function replaceClawPackageRefExpected( `INSERT INTO claw_package_refs ( agent_id, package_kind, package_source, package_ref, package_version, package_integrity, schema_version, claw_name, package_status, relationship, origin, independent_owner, + extension_id, extension_format, extension_detected_format, extension_mapped_json, + extension_unavailable_json, extension_adapter_identity, installed_at_ms, updated_at_ms ) VALUES ( @agent_id, @package_kind, @package_source, @package_ref, @package_version, @package_integrity, @schema_version, @claw_name, @package_status, @relationship, @origin, - @independent_owner, @installed_at_ms, @updated_at_ms + @independent_owner, @extension_id, @extension_format, @extension_detected_format, + @extension_mapped_json, @extension_unavailable_json, @extension_adapter_identity, + @installed_at_ms, @updated_at_ms )`, ) .run({ @@ -100,6 +137,16 @@ export function replaceClawPackageRefExpected( relationship: replacement.relationship, origin: replacement.origin, independent_owner: replacement.independentOwner ? 1 : 0, + extension_id: replacement.extension?.id ?? null, + extension_format: replacement.extension?.format ?? null, + extension_detected_format: replacement.extension?.detectedFormat ?? null, + extension_mapped_json: replacement.extension + ? JSON.stringify(replacement.extension.mapped) + : null, + extension_unavailable_json: replacement.extension + ? JSON.stringify(replacement.extension.unavailable) + : null, + extension_adapter_identity: replacement.extension?.adapterIdentity ?? null, installed_at_ms: replacement.installedAtMs, updated_at_ms: replacement.updatedAtMs, }); diff --git a/src/claws/package-update.test.ts b/src/claws/package-update.test.ts index 7f4c1e59446e..267929d69395 100644 --- a/src/claws/package-update.test.ts +++ b/src/claws/package-update.test.ts @@ -3,7 +3,12 @@ import { digestClawPackageRef } from "./package-update-provenance.js"; import { applyClawPackageUpdate } from "./package-update.js"; import { installClawPackages } from "./packages.js"; import { CLAW_PACKAGE_REF_SCHEMA_VERSION, type PersistedClawPackageRef } from "./provenance.js"; -import { CLAW_OUTPUT_STABILITY, type ClawAddPlan, type ClawManifest } from "./types.js"; +import { + CLAW_OUTPUT_STABILITY, + type ClawAddPlan, + type ClawManifest, + type ResolvedClawPackage, +} from "./types.js"; import { CLAW_UPDATE_PLAN_SCHEMA_VERSION, type ClawUpdatePlan } from "./update-plan.js"; function ref(kind: "skill" | "plugin", name: string, version: string): PersistedClawPackageRef { @@ -50,6 +55,7 @@ function plan(actions: ClawUpdatePlan["actions"]): ClawUpdatePlan { }, actions, capabilityChanges: [], + readiness: { ready: true, requirements: [] }, blockers: [], diagnostics: [], }; @@ -130,6 +136,101 @@ const addPlan: ClawAddPlan = { }; describe("applyClawPackageUpdate", () => { + it("hashes only persisted package provenance from enriched status records", () => { + const persisted = ref("plugin", "audit", "1.0.0"); + expect( + digestClawPackageRef({ + ...persisted, + state: "present", + extensionCompatibility: { state: "compatible" }, + } as typeof persisted), + ).toBe(digestClawPackageRef(persisted)); + }); + + it("adds extension metadata to a reused v1 plugin edge without changing ownership", async () => { + const previous = ref("plugin", "audit", "1.0.0"); + const extension = { + id: "audit-tools", + format: "claude" as const, + detectedFormat: "claude" as const, + mapped: ["skills"], + unavailable: ["agents"], + adapterIdentity: "openclaw/test", + }; + const targetPlan: ClawAddPlan = { + ...addPlan, + actions: [ + { + kind: "package", + id: "plugin:audit", + action: "install", + target: "clawhub:audit@1.0.0", + blocked: false, + details: { + kind: "plugin", + source: "clawhub", + ref: "audit", + version: "1.0.0", + integrity: previous.integrity, + ownerAction: "reuse", + installId: "audit", + extension, + }, + }, + ], + }; + const replaceExpected = vi.fn(); + const installPackages = vi.fn( + async (current: ClawAddPlan, options: Parameters[1]) => { + const persisted = options?.deps?.persistPackageRef; + if (!persisted) { + throw new Error("expected package provenance adapter"); + } + return [ + persisted(current, current.actions[0]!.details as ResolvedClawPackage, { + status: "complete", + relationship: "referenced", + origin: "pre-existing", + independentOwner: true, + }), + ]; + }, + ); + + await applyClawPackageUpdate( + plan([ + { + kind: "package", + id: "plugin:audit", + action: "change", + target: "clawhub:audit@1.0.0", + blocked: false, + reason: "relocated", + currentDigest: digestClawPackageRef(previous), + }, + ]), + { ...manifest, packages: [] }, + targetPlan, + { + installPackages, + readRefs: () => [previous], + replaceExpected, + nowMs: 20, + }, + ); + + expect(replaceExpected).toHaveBeenCalledWith( + previous, + expect.objectContaining({ + extension, + origin: "claw-introduced", + independentOwner: false, + installedAtMs: 10, + }), + expect.any(Object), + ); + }); + it("updates exact references but reports retained artifacts on rollback", async () => { const oldSkill = ref("skill", "triage", "1.0.0"); const legacy = ref("plugin", "legacy", "1.0.0"); diff --git a/src/claws/package-update.ts b/src/claws/package-update.ts index 4d3bb600b561..26ef645c065d 100644 --- a/src/claws/package-update.ts +++ b/src/claws/package-update.ts @@ -44,7 +44,7 @@ function packageKey(value: Pick): string { export async function applyClawPackageUpdate( updatePlan: ClawUpdatePlan, - targetManifest: ClawManifest, + _targetManifest: ClawManifest, targetAddPlan: ClawAddPlan, options: OpenClawStateDatabaseOptions & { installPackages?: typeof installClawPackages; @@ -67,7 +67,6 @@ export async function applyClawPackageUpdate( readRefs({ ...options, agentId: updatePlan.agentId }).map((ref) => [packageKey(ref), ref]), ); const allRefs = readRefs(options); - const targets = new Map(targetManifest.packages.map((pkg) => [packageKey(pkg), pkg])); const undo: Array<() => Promise> = []; const externalMutations: string[] = []; const appliedIds: string[] = []; @@ -114,17 +113,29 @@ export async function applyClawPackageUpdate( appliedIds.push(action.id); continue; } - const target = targets.get(action.id); const targetAction = targetAddPlan.actions.find( (candidate) => candidate.kind === "package" && candidate.id === action.id, ); - if (!target || !targetAction) { + const target = targetAction?.details as + | (ClawPackage & { + integrity?: string; + ownerAction?: "install" | "reuse"; + extension?: PersistedClawPackageRef["extension"]; + }) + | undefined; + if ( + !targetAction || + (target?.kind !== "skill" && target?.kind !== "plugin") || + target.source !== "clawhub" || + !target.ref || + !target.version + ) { throw new ClawPackageUpdateError( `Target package action ${JSON.stringify(action.id)} is missing.`, false, ); } - const targetIntegrity = targetAction.details?.integrity; + const targetIntegrity = target.integrity; if (typeof targetIntegrity !== "string") { throw new ClawPackageUpdateError( `Target package action ${JSON.stringify(action.id)} has no resolved integrity.`, @@ -148,7 +159,11 @@ export async function applyClawPackageUpdate( ); } const nowMs = options.nowMs ?? Date.now(); - const reusesExistingArtifact = targetAction.details?.ownerAction === "reuse"; + const reusesExistingArtifact = target.ownerAction === "reuse"; + const preservesExistingEdge = + reusesExistingArtifact && + previous?.version === target.version && + previous.integrity === targetIntegrity; let claimed: PersistedClawPackageRef = { schemaVersion: CLAW_PACKAGE_REF_SCHEMA_VERSION, agentId: updatePlan.agentId, @@ -159,10 +174,22 @@ export async function applyClawPackageUpdate( version: target.version, integrity: targetIntegrity, status: "pending", - relationship: target.kind === "skill" ? "managed" : "referenced", - origin: reusesExistingArtifact ? "pre-existing" : "claw-introduced", - independentOwner: reusesExistingArtifact, - installedAtMs: nowMs, + relationship: + preservesExistingEdge && previous + ? previous.relationship + : target.kind === "skill" + ? "managed" + : "referenced", + origin: + preservesExistingEdge && previous + ? previous.origin + : reusesExistingArtifact + ? "pre-existing" + : "claw-introduced", + independentOwner: + preservesExistingEdge && previous ? previous.independentOwner : reusesExistingArtifact, + ...(target.extension ? { extension: target.extension } : {}), + installedAtMs: preservesExistingEdge && previous ? previous.installedAtMs : nowMs, updatedAtMs: nowMs, }; replaceExpected(previous, claimed, options); @@ -199,9 +226,15 @@ export async function applyClawPackageUpdate( const next = { ...claimed, status: persistOptions?.status ?? "complete", - relationship: persistOptions?.relationship ?? claimed.relationship, - origin: persistOptions?.origin ?? claimed.origin, - independentOwner: persistOptions?.independentOwner ?? claimed.independentOwner, + relationship: preservesExistingEdge + ? claimed.relationship + : (persistOptions?.relationship ?? claimed.relationship), + origin: preservesExistingEdge + ? claimed.origin + : (persistOptions?.origin ?? claimed.origin), + independentOwner: preservesExistingEdge + ? claimed.independentOwner + : (persistOptions?.independentOwner ?? claimed.independentOwner), updatedAtMs: nowMs, }; replaceExpected(claimed, next, options); diff --git a/src/claws/packages.test.ts b/src/claws/packages.test.ts index bfc31cae56b4..3fb55ff88d25 100644 --- a/src/claws/packages.test.ts +++ b/src/claws/packages.test.ts @@ -2,6 +2,7 @@ import { mkdtemp, rm, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { describe, expect, it, vi } from "vitest"; +import { PLUGIN_ARTIFACT_ADAPTER_IDENTITY } from "../plugins/install-artifact-inspection.js"; import { installClawPackages, preflightClawPackage } from "./packages.js"; import type { PersistedClawPackageRef } from "./provenance.js"; import type { ClawAddPlan, ResolvedClawPackage } from "./types.js"; @@ -130,11 +131,13 @@ describe("preflightClawPackage plugin setup requirements", () => { }, ], }; + const artifactInspection = { format: "openclaw" as const, mapped: ["plugin"], unavailable: [] }; const preflightPlugin = vi.fn().mockResolvedValue({ ok: true, action: "install" }); const probePluginSetup = vi.fn().mockResolvedValue({ ok: true, pluginId: "evidence", setup, + artifactInspection, clawhub: { integrity }, }); @@ -178,6 +181,7 @@ describe("preflightClawPackage plugin setup requirements", () => { { id: "second", envVars: ["SECOND_API_KEY"] }, ], }, + artifactInspection, clawhub: { integrity }, }); @@ -196,6 +200,7 @@ describe("preflightClawPackage plugin setup requirements", () => { setup: { providers: [{ id: "oauth-only", authMethods: ["oauth"] }], }, + artifactInspection, clawhub: { integrity }, }); @@ -229,6 +234,7 @@ describe("preflightClawPackage plugin setup requirements", () => { }, ], }, + artifactInspection, clawhub: { integrity }, }); @@ -248,6 +254,184 @@ describe("preflightClawPackage plugin setup requirements", () => { }); }); +describe("preflightClawPackage isolated plugin inspection", () => { + it("rejects generic agent bundles outside the Claw schema-v1 format contract", async () => { + const probeAgentBundle = vi.fn(async () => ({ + ok: true as const, + pluginId: "audit", + packageName: "@owner/audit", + targetDir: "/tmp/extensions/audit", + extensions: [], + artifactInspection: { + format: "agent" as const, + mapped: ["skills"], + unavailable: [], + }, + clawhub: { + source: "clawhub" as const, + clawhubUrl: "https://clawhub.ai", + clawhubPackage: "@owner/audit", + clawhubFamily: "code-plugin" as const, + integrity, + }, + })); + + await expect( + preflightClawPackage(pluginPackage, "/tmp/workspace", { + deps: { + preflightPlugin: vi.fn(async () => ({ + ok: true as const, + action: "install" as const, + request: {} as never, + })), + probePlugin: probeAgentBundle, + }, + }), + ).resolves.toEqual({ + ok: false, + code: "plugin_artifact_format_unsupported", + message: "Plugin @owner/audit@2.0.1 uses unsupported Claw extension format agent.", + }); + }); + + it("preserves live extension-directory conflict checks for a new plugin install", async () => { + const createProbeExtensionsDir = vi.fn(); + const liveProbe = vi.fn(async () => ({ + ok: false as const, + code: "plugin_target_exists" as never, + error: "plugin already exists: /tmp/extensions/audit", + })); + + await expect( + preflightClawPackage(pluginPackage, "/tmp/workspace", { + deps: { + preflightPlugin: vi.fn(async () => ({ + ok: true as const, + action: "install" as const, + request: {} as never, + })), + probePlugin: liveProbe, + createProbeExtensionsDir, + }, + }), + ).resolves.toMatchObject({ + ok: false, + message: "plugin already exists: /tmp/extensions/audit", + }); + expect(liveProbe).toHaveBeenCalledWith( + expect.not.objectContaining({ extensionsDir: expect.anything() }), + ); + expect(createProbeExtensionsDir).not.toHaveBeenCalled(); + }); + + it("preserves canonical inspection when an installed plugin version conflicts", async () => { + const probePluginConflict = vi.fn(async () => ({ + ok: true as const, + pluginId: "audit", + packageName: "@owner/audit", + targetDir: "/tmp/claw-plugin-probe/audit", + extensions: [], + artifactInspection: { + format: "claude" as const, + mapped: ["commands", "skills"], + unavailable: ["agents"], + }, + clawhub: { + source: "clawhub" as const, + clawhubUrl: "https://clawhub.ai", + clawhubPackage: "@owner/audit", + clawhubFamily: "code-plugin" as const, + integrity, + }, + })); + + await expect( + preflightClawPackage(pluginPackage, "/tmp/workspace", { + deps: { + preflightPlugin: vi.fn(async () => ({ + ok: false as const, + code: "plugin_version_conflict" as const, + error: "Installed plugin has a different version.", + installedVersion: "1.0.0", + expectedVersion: pluginPackage.version, + request: {} as never, + })), + probePlugin: probePluginConflict, + createProbeExtensionsDir: vi.fn(async () => "/tmp/claw-plugin-probe"), + removeProbeExtensionsDir: vi.fn(async () => undefined), + }, + }), + ).resolves.toMatchObject({ + ok: false, + code: "plugin_version_conflict", + installedVersion: "1.0.0", + integrity: `sha256-${Buffer.from("a".repeat(64), "hex").toString("base64")}`, + installId: "audit", + detectedFormat: "claude", + mapped: ["commands", "skills"], + unavailable: ["agents"], + adapterIdentity: expect.stringMatching(/^openclaw\//), + }); + }); + + it("inspects an exact installed plugin outside its live extension directory", async () => { + const removeProbeExtensionsDir = vi.fn(async () => { + throw new Error("temporary directory is still busy"); + }); + const isolatedProbe = vi.fn(async () => ({ + ok: true as const, + pluginId: "audit", + packageName: "@owner/audit", + targetDir: "/tmp/claw-plugin-probe/audit", + extensions: [], + artifactInspection: { + format: "openclaw" as const, + mapped: ["plugin"], + unavailable: [], + }, + clawhub: { + source: "clawhub" as const, + clawhubUrl: "https://clawhub.ai", + clawhubPackage: "@owner/audit", + clawhubFamily: "code-plugin" as const, + integrity, + }, + })); + + await expect( + preflightClawPackage(pluginPackage, "/tmp/workspace", { + deps: { + preflightPlugin: vi.fn(async () => ({ + ok: true as const, + action: "reuse" as const, + request: {} as never, + installedId: "audit", + installedVersion: "2.0.1", + installedIntegrity: integrity, + installedAt: "2026-08-06T00:00:00.000Z", + })), + probePlugin: isolatedProbe, + createProbeExtensionsDir: vi.fn(async () => "/tmp/claw-plugin-probe"), + removeProbeExtensionsDir, + }, + }), + ).resolves.toMatchObject({ + ok: true, + action: "reuse", + installId: "audit", + installedIntegrity: integrity, + installedAt: "2026-08-06T00:00:00.000Z", + detectedFormat: "openclaw", + mapped: ["plugin"], + unavailable: [], + }); + expect(isolatedProbe).toHaveBeenCalledWith( + expect.objectContaining({ extensionsDir: "/tmp/claw-plugin-probe", dryRun: true }), + ); + expect(removeProbeExtensionsDir).toHaveBeenCalledWith("/tmp/claw-plugin-probe"); + }); +}); + describe("installClawPackages", () => { it("installs skill packages into the planned workspace with the resolved digest", async () => { const skillIntegrity = `sha256-${Buffer.from("a".repeat(64), "hex").toString("base64")}`; @@ -315,6 +499,7 @@ describe("installClawPackages", () => { }); it("installs plugins through the shared plugin surface", async () => { + probePlugin.mockClear(); const installPlugin = vi.fn().mockResolvedValue(undefined); const persistPackageRef = vi.fn().mockReturnValue({ kind: "plugin", @@ -348,6 +533,9 @@ describe("installClawPackages", () => { clawManaged: true, }), ); + expect(probePlugin).toHaveBeenCalledWith( + expect.not.objectContaining({ extensionsDir: expect.anything() }), + ); expect(persistPackageRef).toHaveBeenCalledWith( expect.anything(), expect.objectContaining({ @@ -362,7 +550,56 @@ describe("installClawPackages", () => { ); }); + it("resumes an exact Claw-introduced plugin requirement without reinstalling", async () => { + probePlugin.mockClear(); + const introduced = pluginPackageRef("@owner/audit", { + version: pluginPackage.version, + integrity, + }); + const installPlugin = vi.fn(); + const persistPackageRef = vi.fn().mockReturnValue(introduced); + + const result = await installClawPackages(plan([pluginPackage]), { + deps: { + installPlugin, + probePlugin, + preflightPlugin: vi.fn().mockResolvedValue({ + ok: true, + action: "reuse", + installedId: "audit", + installedIntegrity: integrity, + }), + persistPackageRef, + completePackageRef, + readPackageRefs: vi.fn().mockReturnValue([introduced]), + acquirePackageLease, + }, + }); + + expect(result).toEqual([introduced]); + expect(installPlugin).not.toHaveBeenCalled(); + expect(persistPackageRef).toHaveBeenCalledWith( + expect.anything(), + expect.anything(), + expect.objectContaining({ + status: "complete", + relationship: "referenced", + origin: "claw-introduced", + independentOwner: false, + }), + ); + }); + it("records a dependency ref without reinstalling an exact reused plugin", async () => { + probePlugin.mockClear(); + const extension = { + id: "audit-tools", + format: "claude" as const, + detectedFormat: "claude" as const, + mapped: ["skills"], + unavailable: ["agents"], + adapterIdentity: PLUGIN_ARTIFACT_ADAPTER_IDENTITY, + }; const installPlugin = vi.fn(); const persistPackageRef = vi.fn().mockReturnValue({ kind: "plugin" }); const preflightPlugin = vi.fn().mockResolvedValue({ @@ -371,11 +608,30 @@ describe("installClawPackages", () => { installedId: "audit", installedIntegrity: "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", }); + const probePluginForExtension = vi.fn().mockResolvedValue({ + ok: true, + pluginId: "audit", + packageName: "@owner/audit", + targetDir: "/tmp/plugin", + extensions: [], + artifactInspection: { + format: "claude", + mapped: ["skills"], + unavailable: ["agents"], + }, + clawhub: { + source: "clawhub", + clawhubUrl: "https://clawhub.ai", + clawhubPackage: "@owner/audit", + clawhubFamily: "code-plugin", + integrity, + }, + }); - await installClawPackages(plan([pluginPackage], "reuse"), { + await installClawPackages(plan([{ ...pluginPackage, extension }], "reuse"), { deps: { installPlugin, - probePlugin, + probePlugin: probePluginForExtension, preflightPlugin, persistPackageRef, completePackageRef, @@ -385,10 +641,14 @@ describe("installClawPackages", () => { }); expect(installPlugin).not.toHaveBeenCalled(); + expect(probePluginForExtension).toHaveBeenCalledWith( + expect.objectContaining({ extensionsDir: expect.any(String) }), + ); expect(persistPackageRef).toHaveBeenCalledWith( expect.anything(), expect.objectContaining({ integrity: "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + extension, }), expect.objectContaining({ status: "complete", @@ -399,6 +659,59 @@ describe("installClawPackages", () => { ); }); + it("rejects changed extension inspection before recording reused plugin provenance", async () => { + const extension = { + id: "audit-tools", + format: "claude" as const, + detectedFormat: "claude" as const, + mapped: ["skills"], + unavailable: ["agents"], + adapterIdentity: PLUGIN_ARTIFACT_ADAPTER_IDENTITY, + }; + const persistPackageRef = vi.fn(); + + await expect( + installClawPackages(plan([{ ...pluginPackage, extension }], "reuse"), { + deps: { + installPlugin: vi.fn(), + probePlugin: vi.fn().mockResolvedValue({ + ok: true, + pluginId: "audit", + packageName: "@owner/audit", + targetDir: "/tmp/plugin", + extensions: [], + artifactInspection: { + format: "claude", + mapped: ["skills", "commands"], + unavailable: ["agents"], + }, + clawhub: { + source: "clawhub", + clawhubUrl: "https://clawhub.ai", + clawhubPackage: "@owner/audit", + clawhubFamily: "code-plugin", + integrity, + }, + }), + preflightPlugin: vi.fn().mockResolvedValue({ + ok: true, + action: "reuse", + installedId: "audit", + installedIntegrity: integrity, + }), + persistPackageRef, + completePackageRef, + readPackageRefs: vi.fn().mockReturnValue([]), + acquirePackageLease, + }, + }), + ).rejects.toMatchObject({ + code: "package_owner_state_changed", + message: expect.stringContaining("identity or trust state changed after planning"), + }); + expect(persistPackageRef).not.toHaveBeenCalled(); + }); + it("inherits Claw-introduced origin when another Claw already owns the plugin", async () => { const persistPackageRef = vi.fn().mockReturnValue({ kind: "plugin" }); const existing = { diff --git a/src/claws/packages.ts b/src/claws/packages.ts index 95f706893399..0151f3407654 100644 --- a/src/claws/packages.ts +++ b/src/claws/packages.ts @@ -1,15 +1,18 @@ +import { mkdtemp, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { stableStringify } from "@openclaw/normalization-core"; import { runPluginInstallCommand } from "../cli/plugins-install-command.js"; import { runPluginUninstallCommand } from "../cli/plugins-uninstall-command.js"; import { normalizeClawHubSha256Integrity } from "../infra/clawhub.js"; import { installPluginFromClawHub } from "../plugins/clawhub.js"; -import type { PluginManifestSetup } from "../plugins/manifest.js"; +import { PLUGIN_ARTIFACT_ADAPTER_IDENTITY } from "../plugins/install-artifact-inspection.js"; import { preflightPluginInstall, resolveInstalledClawHubPlugin, } from "../plugins/plugin-install-preflight.js"; import { withPluginLifecycleLease } from "../plugins/plugin-lifecycle-lease.js"; import { defaultRuntime, type RuntimeEnv } from "../runtime.js"; -import { resolveLocalProviderAuthEvidence } from "../secrets/provider-auth-evidence.js"; import { installSkillFromClawHub, preflightSkillFromClawHub } from "../skills/lifecycle/clawhub.js"; import { acquireClawPackageLifecycleLease, @@ -17,6 +20,8 @@ import { type MaintainedClawPackageLifecycleLease, } from "../state/claw-package-lifecycle-lease.js"; import type { OpenClawStateDatabaseOptions } from "../state/openclaw-state-db.js"; +import { findResumableIntroducedPluginRequirement } from "./package-resume.js"; +import { resolveClawPluginSetupRequirements } from "./package-setup-requirements.js"; import { persistClawPackageRef, readClawPackageRefs, @@ -26,8 +31,8 @@ import { import type { ClawAddPlan, ClawAddPlanAction, - ClawLocalPrerequisite, ClawPackage, + ClawPackagePreflightResult, ResolvedClawPackage, } from "./types.js"; @@ -96,6 +101,7 @@ function packageFromAction(action: ClawAddPlanAction): PlannedClawPackage { version: details.version, integrity: details.integrity, ownerAction: details.ownerAction, + ...(details.extension ? { extension: details.extension } : {}), ...(details.installId ? { installId: details.installId } : {}), ...(details.riskWarning ? { riskWarning: details.riskWarning } : {}), }; @@ -123,59 +129,42 @@ function ownerInstallIsNewerThanRefs( ); } -type ClawPackagePreflightResult = - | { - ok: true; - action: "install" | "reuse"; - integrity: string; - installId?: string; - warning?: string; - requirements?: ClawLocalPrerequisite[]; - } - | { - ok: false; - code: string; - message: string; - installedVersion?: string; - integrity?: string; - installId?: string; - warning?: string; - }; +type ClawPluginProbeDeps = { + probePlugin?: typeof installPluginFromClawHub; + createProbeExtensionsDir?: () => Promise; + removeProbeExtensionsDir?: (path: string) => Promise; +}; -function resolveClawPluginSetupRequirements(params: { - pluginId: string; - setup?: PluginManifestSetup; - env: NodeJS.ProcessEnv; -}): ClawLocalPrerequisite[] { - const providers = params.setup?.providers ?? []; - // Providers are alternative setup routes for one plugin. Any configured - // route satisfies readiness; otherwise expose every route to the operator. - const hasConfiguredProvider = providers.some( - (provider) => - (provider.envVars ?? []).some((name) => Boolean(params.env[name]?.trim())) || - resolveLocalProviderAuthEvidence(provider.authEvidence, params.env), - ); - if (hasConfiguredProvider) { - return []; +async function probeClawPluginArtifact( + pkg: ClawPackage, + isolateFromLiveExtensions: boolean, + deps: ClawPluginProbeDeps, +): Promise>> { + const probePlugin = deps.probePlugin ?? installPluginFromClawHub; + const request = { + spec: `clawhub:${pkg.ref}@${pkg.version}`, + dryRun: true, + acknowledgeClawHubRisk: true, + } as const; + if (!isolateFromLiveExtensions) { + return await probePlugin(request); } - return providers.flatMap((provider) => { - const envVars = provider.envVars ?? []; - const authEvidence = provider.authEvidence ?? []; - // Dry-run has no persisted setup state, so only gate on credential evidence - // it can actually observe. Auth methods remain descriptive metadata. - if (envVars.length === 0 && authEvidence.length === 0) { - return []; + const probeExtensionsDir = await ( + deps.createProbeExtensionsDir ?? + (async () => await mkdtemp(join(tmpdir(), "openclaw-claw-plugin-probe-"))) + )(); + try { + return await probePlugin({ ...request, extensionsDir: probeExtensionsDir }); + } finally { + try { + await ( + deps.removeProbeExtensionsDir ?? + (async (path: string) => await rm(path, { recursive: true, force: true })) + )(probeExtensionsDir); + } catch { + // Temporary probe cleanup must not replace the canonical preflight result. } - return [ - { - kind: "plugin-setup" as const, - plugin: params.pluginId, - provider: provider.id, - envVars, - authMethods: provider.authMethods ?? [], - }, - ]; - }); + } } export async function preflightClawPackage( @@ -183,7 +172,7 @@ export async function preflightClawPackage( workspaceDir: string, options: { env?: NodeJS.ProcessEnv; - deps?: Pick; + deps?: Pick & ClawPluginProbeDeps; } = {}, ): Promise { if (pkg.kind === "skill") { @@ -207,14 +196,28 @@ export async function preflightClawPackage( message: result.error, }; } - const probe = await (options.deps?.probePlugin ?? installPluginFromClawHub)({ - spec: `clawhub:${pkg.ref}@${pkg.version}`, - dryRun: true, - acknowledgeClawHubRisk: true, - }); + const probe = await probeClawPluginArtifact( + pkg, + !(result.ok && result.action === "install"), + options.deps ?? {}, + ); if (!probe.ok) { return { ok: false, code: probe.code ?? "plugin_preflight_failed", message: probe.error }; } + if (!probe.artifactInspection) { + return { + ok: false, + code: "plugin_artifact_inspection_unavailable", + message: `Plugin ${pkg.ref}@${pkg.version} did not return canonical artifact inspection.`, + }; + } + if (probe.artifactInspection.format === "agent") { + return { + ok: false, + code: "plugin_artifact_format_unsupported", + message: `Plugin ${pkg.ref}@${pkg.version} uses unsupported Claw extension format agent.`, + }; + } const integrity = probe.clawhub.integrity ? normalizeClawHubSha256Integrity(probe.clawhub.integrity) : null; @@ -225,6 +228,11 @@ export async function preflightClawPackage( message: `Plugin ${pkg.ref}@${pkg.version} did not resolve an artifact integrity.`, }; } + const requirements = resolveClawPluginSetupRequirements({ + pluginId: probe.pluginId, + setup: probe.setup, + env: options.env ?? process.env, + }); if (!result.ok) { return { ok: false, @@ -232,6 +240,11 @@ export async function preflightClawPackage( installedVersion: result.installedVersion, integrity, installId: probe.pluginId, + ...(requirements.length > 0 ? { requirements } : {}), + detectedFormat: probe.artifactInspection.format, + mapped: probe.artifactInspection.mapped, + unavailable: probe.artifactInspection.unavailable, + adapterIdentity: PLUGIN_ARTIFACT_ADAPTER_IDENTITY, ...(probe.warning ? { warning: probe.warning } : {}), message: `Plugin ${pkg.ref}@${pkg.version} conflicts with installed version ${result.installedVersion}.`, }; @@ -248,17 +261,20 @@ export async function preflightClawPackage( message: `Plugin ${pkg.ref}@${pkg.version} is installed as ${result.installedId} with integrity ${result.installedIntegrity ?? "unknown"}, expected ${probe.pluginId} with ${integrity}.`, }; } - const requirements = resolveClawPluginSetupRequirements({ - pluginId: probe.pluginId, - setup: probe.setup, - env: options.env ?? process.env, - }); return { ok: true, action: result.action, integrity, installId: probe.pluginId, + ...(result.action === "reuse" && result.installedIntegrity + ? { installedIntegrity: result.installedIntegrity } + : {}), + ...(result.action === "reuse" && result.installedAt ? { installedAt: result.installedAt } : {}), ...(requirements.length > 0 ? { requirements } : {}), + detectedFormat: probe.artifactInspection.format, + mapped: probe.artifactInspection.mapped, + unavailable: probe.artifactInspection.unavailable, + adapterIdentity: PLUGIN_ARTIFACT_ADAPTER_IDENTITY, ...(probe.warning ? { warning: probe.warning } : {}), }; } @@ -396,28 +412,6 @@ async function installClawPackagesUnlocked( continue; } - const probe = await probePlugin({ - spec: `clawhub:${pkg.ref}@${pkg.version}`, - dryRun: true, - acknowledgeClawHubRisk: true, - }); - if (!probe.ok) { - throw new Error(probe.error); - } - const probeIntegrity = probe.clawhub.integrity - ? normalizeClawHubSha256Integrity(probe.clawhub.integrity) - : null; - if ( - probe.pluginId !== pkg.installId || - probeIntegrity !== normalizeClawHubSha256Integrity(pkg.integrity) || - probe.warning !== pkg.riskWarning - ) { - throw new ClawPackageInstallError( - "package_owner_state_changed", - `Plugin ${pkg.ref}@${pkg.version} identity or trust state changed after planning; run add --dry-run again.`, - installedPackages, - ); - } const preflight = await preflightPlugin({ clawhubPackage: pkg.ref, rawSpec: `clawhub:${pkg.ref}@${pkg.version}`, @@ -431,13 +425,70 @@ async function installClawPackagesUnlocked( : preflight.error, ); } - if (preflight.action !== pkg.ownerAction) { + const resumableRequirement = + pkg.ownerAction === "install" && preflight.action === "reuse" + ? findResumableIntroducedPluginRequirement({ + agentId: plan.agent.finalId, + pkg, + preflight, + expectedIntegrity: pkg.integrity, + refs: readPackageRefs({ + ...options, + agentId: plan.agent.finalId, + kind: pkg.kind, + source: pkg.source, + ref: pkg.ref, + version: pkg.version, + }), + }) + : undefined; + if (preflight.action !== pkg.ownerAction && !resumableRequirement) { throw new ClawPackageInstallError( "package_owner_state_changed", `Plugin ${pkg.ref}@${pkg.version} owner state changed from ${pkg.ownerAction} to ${preflight.action}; run add --dry-run again.`, installedPackages, ); } + const probe = await probeClawPluginArtifact(pkg, preflight.action === "reuse", { + probePlugin, + }); + packageLease.assertCurrent(); + if (!probe.ok) { + throw new Error(probe.error); + } + const probeIntegrity = probe.clawhub.integrity + ? normalizeClawHubSha256Integrity(probe.clawhub.integrity) + : null; + const plannedExtensionInspection = pkg.extension + ? { + detectedFormat: pkg.extension.detectedFormat, + mapped: pkg.extension.mapped, + unavailable: pkg.extension.unavailable, + adapterIdentity: pkg.extension.adapterIdentity, + } + : undefined; + const probedExtensionInspection = probe.artifactInspection + ? { + detectedFormat: probe.artifactInspection.format, + mapped: probe.artifactInspection.mapped, + unavailable: probe.artifactInspection.unavailable, + adapterIdentity: PLUGIN_ARTIFACT_ADAPTER_IDENTITY, + } + : undefined; + if ( + probe.pluginId !== pkg.installId || + probeIntegrity !== normalizeClawHubSha256Integrity(pkg.integrity) || + probe.warning !== pkg.riskWarning || + (plannedExtensionInspection && + stableStringify(probedExtensionInspection) !== + stableStringify(plannedExtensionInspection)) + ) { + throw new ClawPackageInstallError( + "package_owner_state_changed", + `Plugin ${pkg.ref}@${pkg.version} identity or trust state changed after planning; run add --dry-run again.`, + installedPackages, + ); + } if (!pkg.installId) { throw new ClawPackageInstallError( "plugin_identity_unresolved", @@ -458,6 +509,18 @@ async function installClawPackagesUnlocked( installedPackages, ); } + if (resumableRequirement) { + installedPackages.push( + persistPackageRef(plan, pkg, { + ...options, + status: "complete", + relationship: resumableRequirement.relationship, + origin: resumableRequirement.origin, + independentOwner: resumableRequirement.independentOwner, + }), + ); + continue; + } const existingRefs = readPackageRefs({ ...options, kind: pkg.kind, diff --git a/src/claws/provenance-legacy-columns.ts b/src/claws/provenance-legacy-columns.ts new file mode 100644 index 000000000000..b2b4fd2a82fc --- /dev/null +++ b/src/claws/provenance-legacy-columns.ts @@ -0,0 +1,32 @@ +// Projects additive Claw provenance columns that only writable opens can ensure. +import type { DatabaseSync } from "node:sqlite"; + +function canSelect(db: DatabaseSync, table: string, projection: string): boolean { + try { + db /* sqlite-allow-raw: capability probe for lazily added Claw provenance columns. */ + .prepare(`SELECT ${projection} FROM ${table} LIMIT 0`); + return true; + } catch { + return false; + } +} + +/** + * Read-only opens never run the additive column migration, so a same-version + * database written before a column existed must still answer planning reads. + * Absent columns project as SQL NULL, which the row parsers already treat as + * "no recorded provenance". + */ +export function legacySafeColumnProjection( + db: DatabaseSync, + table: "claw_installs" | "claw_package_refs", + columns: readonly string[], +): string { + const full = columns.join(", "); + if (canSelect(db, table, full)) { + return full; + } + return columns + .map((column) => (canSelect(db, table, column) ? column : `NULL AS ${column}`)) + .join(", "); +} diff --git a/src/claws/provenance.test-helpers.ts b/src/claws/provenance.test-helpers.ts new file mode 100644 index 000000000000..9a9a299e2748 --- /dev/null +++ b/src/claws/provenance.test-helpers.ts @@ -0,0 +1,71 @@ +import { join } from "node:path"; +import { openOpenClawStateDatabase } from "../state/openclaw-state-db.js"; +import { buildClawAddPlan } from "./lifecycle.js"; +import { parseClawManifest } from "./schema.js"; +import type { ClawOpenClawProfile, ClawSourceIdentity } from "./types.js"; + +export async function makeProvenancePlan( + root: string, + manifestValue: unknown, + options: { + workspace?: string; + openClawProfile?: ClawOpenClawProfile; + packagePreflight?: NonNullable< + Parameters[0]["context"] + >["packagePreflight"]; + } = {}, +) { + const parsed = parseClawManifest(manifestValue); + if (!parsed.ok) { + throw new Error(JSON.stringify(parsed.diagnostics)); + } + const source: ClawSourceIdentity = { + kind: "package", + name: "@acme/worker", + version: "1.0.0", + packageRoot: root, + manifestPath: join(root, "openclaw.claw.json"), + integrityKind: "artifact", + integrity: "sha256:manifest", + byteLength: 123, + }; + const plan = await buildClawAddPlan({ + manifest: parsed.manifest, + openClawProfile: options.openClawProfile, + source, + context: { + workspace: options.workspace ?? join(root, "workspace-worker"), + ...(options.packagePreflight ? { packagePreflight: options.packagePreflight } : {}), + }, + }); + return { root, plan }; +} + +export function stateEnv(root: string) { + return { OPENCLAW_STATE_DIR: join(root, "state") }; +} + +export function readInstallRow(agentId: string, root: string) { + return openOpenClawStateDatabase({ env: stateEnv(root) }) + .db.prepare( + `SELECT agent_id, schema_version, claw_name, claw_version, integrity, plan_integrity, + workspace, agent_config_digest, agent_owned_paths_json, status, added_at_ms + FROM claw_installs + WHERE agent_id = ?`, + ) + .get(agentId) as + | { + agent_id: string; + schema_version: string; + claw_name: string; + claw_version: string; + integrity: string; + plan_integrity: string; + workspace: string; + agent_config_digest: string; + agent_owned_paths_json: string; + status: string; + added_at_ms: number | bigint; + } + | undefined; +} diff --git a/src/claws/provenance.test.ts b/src/claws/provenance.test.ts index 3b3c48aab107..4b58624fcaa6 100644 --- a/src/claws/provenance.test.ts +++ b/src/claws/provenance.test.ts @@ -1,18 +1,16 @@ // Tests root Claw install ownership and the narrow agent/workspace mutation slice. import { access, mkdir, rmdir, symlink, writeFile } from "node:fs/promises"; import { join } from "node:path"; -import { afterEach, describe, expect, it } from "vitest"; +import { afterEach, describe, expect, it, vi } from "vitest"; import { useAutoCleanupTempDirTracker } from "../../test/helpers/temp-dir.js"; import type { OpenClawConfig } from "../config/types.openclaw.js"; -import { - closeOpenClawStateDatabaseForTest, - openOpenClawStateDatabase, -} from "../state/openclaw-state-db.js"; +import { closeOpenClawStateDatabaseForTest } from "../state/openclaw-state-db.js"; import { applyClawAddPlan, ClawAddMutationError } from "./add.js"; import { ClawCronInstallError } from "./cron.js"; -import { buildClawAddPlan } from "./lifecycle.js"; import { replaceClawPackageRefExpected } from "./package-update-provenance.js"; +import { ClawPackageInstallError } from "./packages.js"; import { + clawInstallRecordMatchesPlan, persistClawInstallRecord, persistClawPackageRef, readClawInstallRecord, @@ -21,8 +19,7 @@ import { updateClawInstallRecordStatus, updateClawPackageRefStatus, } from "./provenance.js"; -import { parseClawManifest } from "./schema.js"; -import type { ClawOpenClawProfile, ClawSourceIdentity } from "./types.js"; +import { makeProvenancePlan, readInstallRow, stateEnv } from "./provenance.test-helpers.js"; const tempDirs = useAutoCleanupTempDirTracker(afterEach); @@ -32,59 +29,10 @@ afterEach(() => { async function makePlan( manifestValue: unknown = { schemaVersion: 1, agent: { id: "worker" } }, - options: { workspace?: string; openClawProfile?: ClawOpenClawProfile } = {}, + options: Parameters[2] = {}, ) { const root = tempDirs.make("openclaw-claw-add-"); - const parsed = parseClawManifest(manifestValue); - if (!parsed.ok) { - throw new Error(JSON.stringify(parsed.diagnostics)); - } - const source: ClawSourceIdentity = { - kind: "package", - name: "@acme/worker", - version: "1.0.0", - packageRoot: root, - manifestPath: join(root, "openclaw.claw.json"), - integrityKind: "artifact", - integrity: "sha256:manifest", - byteLength: 123, - }; - const plan = await buildClawAddPlan({ - manifest: parsed.manifest, - openClawProfile: options.openClawProfile, - source, - context: { workspace: options.workspace ?? join(root, "workspace-worker") }, - }); - return { root, plan }; -} - -function stateEnv(root: string) { - return { OPENCLAW_STATE_DIR: join(root, "state") }; -} - -function readInstallRow(agentId: string, root: string) { - return openOpenClawStateDatabase({ env: stateEnv(root) }) - .db.prepare( - `SELECT agent_id, schema_version, claw_name, claw_version, integrity, plan_integrity, - workspace, agent_config_digest, agent_owned_paths_json, status, added_at_ms - FROM claw_installs - WHERE agent_id = ?`, - ) - .get(agentId) as - | { - agent_id: string; - schema_version: string; - claw_name: string; - claw_version: string; - integrity: string; - plan_integrity: string; - workspace: string; - agent_config_digest: string; - agent_owned_paths_json: string; - status: string; - added_at_ms: number | bigint; - } - | undefined; + return await makeProvenancePlan(root, manifestValue, options); } describe("Claw root install provenance", () => { @@ -126,6 +74,34 @@ describe("Claw root install provenance", () => { expect(readClawPackageRefs({ env: stateEnv(root) })).toEqual([replayed]); }); + it("round-trips canonical extension inventory on the shared plugin dependency edge", async () => { + const { root, plan } = await makePlan(); + const extension = { + id: "coding-tools", + format: "claude" as const, + detectedFormat: "claude" as const, + mapped: ["commands", "skills"], + unavailable: ["agents"], + adapterIdentity: "openclaw/test", + }; + + const persisted = persistClawPackageRef( + plan, + { + kind: "plugin", + source: "clawhub", + ref: "@acme/coding-tools", + version: "1.2.3", + integrity: `sha256:${"b".repeat(64)}`, + extension, + }, + { env: stateEnv(root), nowMs: 42, relationship: "referenced" }, + ); + + expect(persisted.extension).toEqual(extension); + expect(readClawPackageRefs({ env: stateEnv(root) })).toEqual([persisted]); + }); + it("persists package identity, agent ownership, workspace, and config digest", async () => { const { root, plan } = await makePlan(); @@ -181,6 +157,9 @@ describe("Claw root install provenance", () => { }); expect(resumed).toEqual(first); + expect(clawInstallRecordMatchesPlan(first, { ...plan, planIntegrity: "sha256:changed" })).toBe( + false, + ); expect(readClawInstallRecord("worker", { env: stateEnv(root) })).toMatchObject({ agentId: "worker", status: "pending", @@ -349,6 +328,283 @@ describe("Claw root install provenance", () => { }); describe("applyClawAddPlan", () => { + it("realizes shared plugin requirements before creating the agent workspace", async () => { + const { root, plan } = await makePlan( + { + schemaVersion: 1, + agent: { id: "worker" }, + packages: [ + { + kind: "plugin", + source: "clawhub", + ref: "@acme/audit", + version: "1.0.0", + }, + ], + }, + { + packagePreflight: async () => ({ + ok: true, + action: "install", + integrity: `sha256:${"a".repeat(64)}`, + installId: "audit", + }), + }, + ); + const order: string[] = []; + + const result = await applyClawAddPlan(plan, { + consentPlanIntegrity: plan.planIntegrity, + env: stateEnv(root), + installPackages: async () => { + await expect(access(plan.agent.workspace)).rejects.toThrow(); + order.push("requirement"); + return []; + }, + commitConfig: async (transform) => { + order.push("agent"); + transform({}); + }, + }); + + expect(result.status).toBe("complete"); + expect(order).toEqual(["requirement", "agent"]); + }); + + it("retains an introduced shared requirement when later agent creation fails", async () => { + const { root, plan } = await makePlan( + { + schemaVersion: 1, + agent: { id: "worker" }, + packages: [ + { + kind: "plugin", + source: "clawhub", + ref: "@acme/audit", + version: "1.0.0", + }, + ], + }, + { + packagePreflight: async () => ({ + ok: true, + action: "install", + integrity: `sha256:${"a".repeat(64)}`, + installId: "audit", + }), + }, + ); + const requirement = { + schemaVersion: "openclaw.clawPackageRef.v1" as const, + agentId: "worker", + clawName: "@acme/worker", + kind: "plugin" as const, + source: "clawhub" as const, + ref: "@acme/audit", + version: "1.0.0", + integrity: `sha256:${"a".repeat(64)}`, + status: "complete" as const, + relationship: "referenced" as const, + origin: "claw-introduced" as const, + independentOwner: false, + installedAtMs: 1, + updatedAtMs: 1, + }; + + const result = await applyClawAddPlan(plan, { + consentPlanIntegrity: plan.planIntegrity, + env: stateEnv(root), + installPackages: async () => [requirement], + commitConfig: async () => { + throw new Error("config unavailable"); + }, + }); + + expect(result).toMatchObject({ + status: "partial", + workspaceCreated: false, + configCommitted: false, + packages: [{ ref: "@acme/audit", origin: "claw-introduced" }], + error: { code: "config_commit_failed" }, + }); + await expect(access(plan.agent.workspace)).rejects.toThrow(); + }); + + it("reports retained plugin requirements when a later workspace package fails", async () => { + const { root, plan } = await makePlan( + { + schemaVersion: 1, + agent: { id: "worker" }, + packages: [ + { + kind: "plugin", + source: "clawhub", + ref: "@acme/audit", + version: "1.0.0", + }, + { + kind: "skill", + source: "clawhub", + ref: "research", + version: "1.0.0", + }, + ], + }, + { + packagePreflight: async (pkg) => ({ + ok: true, + action: "install", + integrity: + pkg.kind === "plugin" ? `sha256:${"a".repeat(64)}` : `sha256:${"b".repeat(64)}`, + ...(pkg.kind === "plugin" ? { installId: "audit" } : {}), + }), + }, + ); + const requirement = { + schemaVersion: "openclaw.clawPackageRef.v1" as const, + agentId: "worker", + clawName: "@acme/worker", + kind: "plugin" as const, + source: "clawhub" as const, + ref: "@acme/audit", + version: "1.0.0", + integrity: `sha256:${"a".repeat(64)}`, + status: "complete" as const, + relationship: "referenced" as const, + origin: "claw-introduced" as const, + independentOwner: false, + installedAtMs: 1, + updatedAtMs: 1, + }; + const failedSkill = { + ...requirement, + kind: "skill" as const, + ref: "research", + integrity: `sha256:${"b".repeat(64)}`, + status: "failed" as const, + relationship: "managed" as const, + }; + const installPackages = vi + .fn() + .mockResolvedValueOnce([requirement]) + .mockRejectedValueOnce( + new ClawPackageInstallError("package_install_failed", "skill installer failed", [ + failedSkill, + ]), + ); + + const result = await applyClawAddPlan(plan, { + consentPlanIntegrity: plan.planIntegrity, + env: stateEnv(root), + installPackages, + commitConfig: async (transform) => { + transform({}); + }, + }); + + expect(installPackages).toHaveBeenCalledTimes(2); + expect(result).toMatchObject({ + status: "partial", + packages: [ + { kind: "plugin", ref: "@acme/audit", status: "complete" }, + { kind: "skill", ref: "research", status: "failed" }, + ], + error: { code: "package_install_failed", message: "skill installer failed" }, + }); + }); + + it("stops before agent mutation when a shared requirement fails", async () => { + const { root, plan } = await makePlan( + { + schemaVersion: 1, + agent: { id: "worker" }, + packages: [ + { + kind: "plugin", + source: "clawhub", + ref: "@acme/audit", + version: "1.0.0", + }, + ], + }, + { + packagePreflight: async () => ({ + ok: true, + action: "install", + integrity: `sha256:${"a".repeat(64)}`, + installId: "audit", + }), + }, + ); + const commitConfig = vi.fn(); + + const result = await applyClawAddPlan(plan, { + consentPlanIntegrity: plan.planIntegrity, + env: stateEnv(root), + installPackages: async () => { + throw new ClawPackageInstallError("package_install_failed", "installer failed", []); + }, + commitConfig, + }); + + expect(result).toMatchObject({ + status: "partial", + workspaceCreated: false, + configCommitted: false, + error: { code: "package_install_failed", message: "installer failed" }, + }); + expect(commitConfig).not.toHaveBeenCalled(); + await expect(access(plan.agent.workspace)).rejects.toThrow(); + }); + + it("preserves a config-committed phase when a resumed host requirement fails", async () => { + const { root, plan } = await makePlan( + { + schemaVersion: 1, + agent: { id: "worker" }, + packages: [ + { + kind: "plugin", + source: "clawhub", + ref: "@acme/audit", + version: "1.0.0", + }, + ], + }, + { + packagePreflight: async () => ({ + ok: true, + action: "install", + integrity: `sha256:${"a".repeat(64)}`, + installId: "audit", + }), + }, + ); + await mkdir(plan.agent.workspace, { recursive: true }); + persistClawInstallRecord(plan, { + env: stateEnv(root), + status: "config_committed", + nowMs: 1, + }); + + const result = await applyClawAddPlan(plan, { + consentPlanIntegrity: plan.planIntegrity, + env: stateEnv(root), + installPackages: async () => { + throw new ClawPackageInstallError("package_install_failed", "installer failed", []); + }, + }); + + expect(result).toMatchObject({ + status: "partial", + workspaceCreated: true, + configCommitted: true, + installRecord: { status: "config_committed" }, + error: { code: "package_install_failed", message: "installer failed" }, + }); + expect(readInstallRow("worker", root)?.status).toBe("config_committed"); + }); + it("appends one agent, preserves defaults and existing agents, and creates a new workspace", async () => { const { root, plan } = await makePlan( { @@ -543,13 +799,36 @@ describe("applyClawAddPlan", () => { }); it("records a partial add when the workspace appears after planning", async () => { - const { root, plan } = await makePlan(); + const { root, plan } = await makePlan( + { + schemaVersion: 1, + agent: { id: "worker" }, + packages: [ + { + kind: "plugin", + source: "clawhub", + ref: "@acme/audit", + version: "1.0.0", + }, + ], + }, + { + packagePreflight: async () => ({ + ok: true, + action: "install", + integrity: `sha256:${"a".repeat(64)}`, + installId: "audit", + }), + }, + ); + const installPackages = vi.fn(); await mkdir(plan.agent.workspace); await expect( applyClawAddPlan(plan, { consentPlanIntegrity: plan.planIntegrity, env: stateEnv(root), + installPackages, }), ).resolves.toMatchObject({ status: "partial", @@ -558,6 +837,7 @@ describe("applyClawAddPlan", () => { error: { code: "workspace_collision" }, }); expect(readInstallRow("worker", root)?.status).toBe("partial"); + expect(installPackages).not.toHaveBeenCalled(); }); it("records parent-directory creation failures before workspace mutation", async () => { diff --git a/src/claws/provenance.ts b/src/claws/provenance.ts index c74e7c616ee7..62a17f29bfba 100644 --- a/src/claws/provenance.ts +++ b/src/claws/provenance.ts @@ -7,11 +7,25 @@ import { runOpenClawStateWriteTransaction, type OpenClawStateDatabaseOptions, } from "../state/openclaw-state-db.js"; +import { + CLAW_PACKAGE_REF_SCHEMA_VERSION, + rowToPackageRef, + type ClawPackageOrigin, + type ClawPackageRefStatus, + type ClawPackageRelationship, + type PackageRefRow, + type PersistedClawPackageRef, +} from "./package-extension-provenance.js"; import { clawBootstrapProvenanceFromRow, selectClawBootstrapProvenanceColumns, } from "./provenance-bootstrap.js"; +import { legacySafeColumnProjection } from "./provenance-legacy-columns.js"; import type { ClawAddPlan, ClawPackage, ResolvedClawPackage } from "./types.js"; +export { + CLAW_PACKAGE_REF_SCHEMA_VERSION, + type PersistedClawPackageRef, +} from "./package-extension-provenance.js"; const CLAW_INSTALL_RECORD_SCHEMA_VERSION = "openclaw.clawInstallRecord.v1" as const; @@ -155,6 +169,31 @@ function rowToRecord(row: ClawInstallRow): PersistedClawInstall { }; } +export function clawInstallRecordMatchesPlan( + record: PersistedClawInstall, + plan: ClawAddPlan, +): boolean { + const bootstrap = bootstrapProvenance(plan); + return ( + record.schemaVersion === CLAW_INSTALL_RECORD_SCHEMA_VERSION && + record.claw.kind === plan.claw.kind && + record.claw.name === plan.claw.name && + record.claw.version === plan.claw.version && + record.claw.packageRoot === plan.claw.packageRoot && + record.claw.manifestPath === plan.claw.manifestPath && + record.claw.integrityKind === plan.claw.integrityKind && + record.claw.integrity === plan.claw.integrity && + record.claw.byteLength === plan.claw.byteLength && + record.manifestSchemaVersion === plan.manifestSchemaVersion && + record.planIntegrity === plan.planIntegrity && + record.workspace === plan.agent.workspace && + record.agentConfigDigest === digestAgentConfig(plan) && + stableStringify(record.agentOwnedPaths) === stableStringify(agentOwnedPaths(plan)) && + record.bootstrap?.sourcePath === bootstrap?.sourcePath && + record.bootstrap?.contentDigest === bootstrap?.contentDigest + ); +} + function selectClawInstallRow(db: DatabaseSync, agentId: string): ClawInstallRow | undefined { const bootstrapColumns = selectClawBootstrapProvenanceColumns(db); return db /* sqlite-allow-raw: this Claw prototype state-table read is scoped to one owned row. */ @@ -170,6 +209,14 @@ function selectClawInstallRow(db: DatabaseSync, agentId: string): ClawInstallRow .get(agentId) as ClawInstallRow | undefined; } +export function readClawInstallRecordFromDatabase( + db: DatabaseSync, + agentId: string, +): PersistedClawInstall | undefined { + const row = selectClawInstallRow(db, agentId); + return row ? rowToRecord(row) : undefined; +} + function getClawInstallRow( agentId: string, options: OpenClawStateDatabaseOptions, @@ -185,31 +232,8 @@ export function readClawInstallRecord( return row ? rowToRecord(row) : undefined; } -function isSameInstallAttempt( - row: ClawInstallRow, - plan: ClawAddPlan, - agentConfigDigest: string, - ownedPaths: string[], -): boolean { - const bootstrap = bootstrapProvenance(plan); - return ( - row.schema_version === CLAW_INSTALL_RECORD_SCHEMA_VERSION && - row.source_kind === plan.claw.kind && - row.claw_name === plan.claw.name && - row.claw_version === plan.claw.version && - row.package_root === plan.claw.packageRoot && - row.manifest_path === plan.claw.manifestPath && - row.integrity_kind === plan.claw.integrityKind && - row.integrity === plan.claw.integrity && - Number(row.source_byte_length) === plan.claw.byteLength && - Number(row.manifest_schema_version) === plan.manifestSchemaVersion && - row.plan_integrity === plan.planIntegrity && - row.workspace === plan.agent.workspace && - row.agent_config_digest === agentConfigDigest && - row.agent_owned_paths_json === JSON.stringify(ownedPaths) && - row.bootstrap_source_path === (bootstrap?.sourcePath ?? null) && - row.bootstrap_content_digest === (bootstrap?.contentDigest ?? null) - ); +function isSameInstallAttempt(row: ClawInstallRow, plan: ClawAddPlan): boolean { + return clawInstallRecordMatchesPlan(rowToRecord(row), plan); } export function persistClawInstallRecord( @@ -224,10 +248,7 @@ export function persistClawInstallRecord( return runOpenClawStateWriteTransaction(({ db }) => { const existing = selectClawInstallRow(db, plan.agent.finalId); if (existing) { - if ( - existing.status !== "complete" && - isSameInstallAttempt(existing, plan, agentConfigDigest, ownedPaths) - ) { + if (existing.status !== "complete" && isSameInstallAttempt(existing, plan)) { return rowToRecord(existing); } // A nonmatching partial attempt remains durable ownership evidence. A later @@ -347,7 +368,6 @@ export function readClawInstallRecords( ): PersistedClawInstall[] { const database = openOpenClawStateDatabase(options); const bootstrapColumns = selectClawBootstrapProvenanceColumns(database.db); - // sqlite-allow-raw: read-only Claw install inventory ordered by stable agent id. const rows = database.db /* sqlite-allow-raw: read-only Claw install inventory ordered by stable agent id. */ .prepare( @@ -364,45 +384,6 @@ export function readClawInstallRecords( return rows.map(rowToInstall); } -export const CLAW_PACKAGE_REF_SCHEMA_VERSION = "openclaw.clawPackageRef.v1" as const; -type ClawPackageRefStatus = "pending" | "complete" | "failed" | "rolled_back"; -type ClawPackageRelationship = "managed" | "referenced"; -type ClawPackageOrigin = "claw-introduced" | "pre-existing"; - -export type PersistedClawPackageRef = { - schemaVersion: typeof CLAW_PACKAGE_REF_SCHEMA_VERSION; - agentId: string; - clawName: string; - kind: ClawPackage["kind"]; - source: ClawPackage["source"]; - ref: string; - version: string; - integrity: string; - status: ClawPackageRefStatus; - relationship: ClawPackageRelationship; - origin: ClawPackageOrigin; - independentOwner: boolean; - installedAtMs: number; - updatedAtMs: number; -}; - -type PackageRefRow = { - schema_version: string; - agent_id: string; - claw_name: string; - package_kind: ClawPackage["kind"]; - package_source: ClawPackage["source"]; - package_ref: string; - package_version: string; - package_integrity: string; - package_status: ClawPackageRefStatus; - relationship: ClawPackageRelationship; - origin: ClawPackageOrigin; - independent_owner: number | bigint; - installed_at_ms: number | bigint; - updated_at_ms: number | bigint; -}; - export function updateClawInstallRecord( plan: ClawAddPlan, options: OpenClawStateDatabaseOptions & { @@ -493,25 +474,6 @@ export function updateClawInstallRecord( }; } -function rowToPackageRef(row: PackageRefRow): PersistedClawPackageRef { - return { - schemaVersion: CLAW_PACKAGE_REF_SCHEMA_VERSION, - agentId: row.agent_id, - clawName: row.claw_name, - kind: row.package_kind, - source: row.package_source, - ref: row.package_ref, - version: row.package_version, - integrity: row.package_integrity, - status: row.package_status, - relationship: row.relationship, - origin: row.origin, - independentOwner: Number(row.independent_owner) === 1, - installedAtMs: Number(row.installed_at_ms), - updatedAtMs: Number(row.updated_at_ms), - }; -} - export function persistClawPackageRef( plan: ClawAddPlan, pkg: ResolvedClawPackage, @@ -537,6 +499,7 @@ export function persistClawPackageRef( relationship: options.relationship ?? (pkg.kind === "skill" ? "managed" : "referenced"), origin: options.origin ?? "claw-introduced", independentOwner: options.independentOwner ?? false, + ...(pkg.extension ? { extension: pkg.extension } : {}), installedAtMs: nowMs, updatedAtMs: nowMs, }; @@ -545,7 +508,8 @@ export function persistClawPackageRef( .prepare( `SELECT schema_version, agent_id, claw_name, package_kind, package_source, package_ref, package_version, package_integrity, package_status, relationship, origin, - independent_owner, + independent_owner, extension_id, extension_format, extension_detected_format, + extension_mapped_json, extension_unavailable_json, extension_adapter_identity, installed_at_ms, updated_at_ms FROM claw_package_refs WHERE agent_id = @agent_id @@ -584,6 +548,12 @@ export function persistClawPackageRef( relationship = @relationship, origin = @origin, independent_owner = @independent_owner, + extension_id = @extension_id, + extension_format = @extension_format, + extension_detected_format = @extension_detected_format, + extension_mapped_json = @extension_mapped_json, + extension_unavailable_json = @extension_unavailable_json, + extension_adapter_identity = @extension_adapter_identity, updated_at_ms = @updated_at_ms WHERE agent_id = @agent_id AND package_kind = @package_kind @@ -605,6 +575,14 @@ export function persistClawPackageRef( relationship: record.relationship, origin: record.origin, independent_owner: record.independentOwner ? 1 : 0, + extension_id: record.extension?.id ?? null, + extension_format: record.extension?.format ?? null, + extension_detected_format: record.extension?.detectedFormat ?? null, + extension_mapped_json: record.extension ? JSON.stringify(record.extension.mapped) : null, + extension_unavailable_json: record.extension + ? JSON.stringify(record.extension.unavailable) + : null, + extension_adapter_identity: record.extension?.adapterIdentity ?? null, updated_at_ms: record.updatedAtMs, }); return; @@ -615,13 +593,15 @@ export function persistClawPackageRef( `INSERT INTO claw_package_refs ( agent_id, package_kind, package_source, package_ref, package_version, package_integrity, schema_version, claw_name, package_status, relationship, origin, - independent_owner, + independent_owner, extension_id, extension_format, extension_detected_format, + extension_mapped_json, extension_unavailable_json, extension_adapter_identity, installed_at_ms, updated_at_ms ) VALUES ( @agent_id, @package_kind, @package_source, @package_ref, @package_version, @package_integrity, @schema_version, @claw_name, @package_status, @relationship, @origin, - @independent_owner, + @independent_owner, @extension_id, @extension_format, @extension_detected_format, + @extension_mapped_json, @extension_unavailable_json, @extension_adapter_identity, @installed_at_ms, @updated_at_ms )`, @@ -639,6 +619,14 @@ export function persistClawPackageRef( relationship: record.relationship, origin: record.origin, independent_owner: record.independentOwner ? 1 : 0, + extension_id: record.extension?.id ?? null, + extension_format: record.extension?.format ?? null, + extension_detected_format: record.extension?.detectedFormat ?? null, + extension_mapped_json: record.extension ? JSON.stringify(record.extension.mapped) : null, + extension_unavailable_json: record.extension + ? JSON.stringify(record.extension.unavailable) + : null, + extension_adapter_identity: record.extension?.adapterIdentity ?? null, installed_at_ms: record.installedAtMs, updated_at_ms: record.updatedAtMs, }); @@ -714,12 +702,20 @@ export function readClawPackageRefs( } } const where = conditions.length > 0 ? ` WHERE ${conditions.join(" AND ")}` : ""; + const extensionColumns = legacySafeColumnProjection(database.db, "claw_package_refs", [ + "extension_id", + "extension_format", + "extension_detected_format", + "extension_mapped_json", + "extension_unavailable_json", + "extension_adapter_identity", + ]); const rows = database.db /* sqlite-allow-raw: read-only Claw package reference lookup with closed column filters. */ .prepare( `SELECT schema_version, agent_id, claw_name, package_kind, package_source, package_ref, package_version, package_integrity, package_status, relationship, origin, - independent_owner, + independent_owner, ${extensionColumns}, installed_at_ms, updated_at_ms FROM claw_package_refs${where} diff --git a/src/claws/read-only-state-compat.test.ts b/src/claws/read-only-state-compat.test.ts new file mode 100644 index 000000000000..065b50eeb4e4 --- /dev/null +++ b/src/claws/read-only-state-compat.test.ts @@ -0,0 +1,126 @@ +// Regression coverage for read-only Claw state access on databases that predate +// the additive provenance columns but already report the current schema version. +import { mkdir, readFile, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { afterEach, describe, expect, it } from "vitest"; +import { useAutoCleanupTempDirTracker } from "../../test/helpers/temp-dir.js"; +import { CLAW_LAZY_ADDITIVE_STATE_COLUMNS } from "../state/openclaw-state-db-maintenance.js"; +import { + closeOpenClawStateDatabaseForTest, + openOpenClawStateDatabase, +} from "../state/openclaw-state-db.js"; +import { readClawResumeStateReadOnly } from "./package-resume.js"; +import { parseClawManifest } from "./schema.js"; +import type { ClawSourceIdentity } from "./types.js"; +import { buildClawUpdatePlan } from "./update-plan.js"; + +const tempDirs = useAutoCleanupTempDirTracker(afterEach); + +afterEach(() => closeOpenClawStateDatabaseForTest()); + +function createBaseShapeState(params: { + env: { OPENCLAW_STATE_DIR: string }; + packageRoot: string; + workspace: string; +}): string { + const database = openOpenClawStateDatabase({ env: params.env }); + const databasePath = database.path; + database.db + .prepare( + `INSERT INTO claw_installs ( + agent_id, schema_version, source_kind, claw_name, claw_version, package_root, + manifest_path, integrity_kind, integrity, source_byte_length, manifest_schema_version, + plan_integrity, workspace, agent_config_digest, agent_owned_paths_json, status, + added_at_ms, updated_at_ms + ) VALUES ( + 'legacy-worker', 'openclaw.clawInstallRecord.v1', 'package', '@acme/legacy', '1.0.0', ?, + ?, 'artifact', 'sha256:aa', 10, 1, 'sha256:bb', ?, 'sha256:cc', '[]', 'complete', + 1000, 2000 + )`, + ) + .run(params.packageRoot, join(params.packageRoot, "CLAW.md"), params.workspace); + for (const column of CLAW_LAZY_ADDITIVE_STATE_COLUMNS) { + const [table, name] = column.split("."); + database.db.exec(`ALTER TABLE ${table} DROP COLUMN ${name};`); + } + closeOpenClawStateDatabaseForTest(); + return databasePath; +} + +async function createFixture(label: string): Promise<{ + env: { OPENCLAW_STATE_DIR: string }; + databasePath: string; + packageRoot: string; + workspace: string; +}> { + const root = tempDirs.make(label); + const packageRoot = join(root, "package"); + const workspace = join(root, "workspace"); + await mkdir(packageRoot, { recursive: true }); + await mkdir(workspace, { recursive: true }); + await writeFile(join(packageRoot, "CLAW.md"), "---\nschemaVersion: 1\n---\n", "utf8"); + const env = { OPENCLAW_STATE_DIR: join(root, "state") }; + return { + env, + databasePath: createBaseShapeState({ env, packageRoot, workspace }), + packageRoot, + workspace, + }; +} + +describe("read-only Claw state compatibility", () => { + it("plans an update against a base-shape database without mutating it", async () => { + const fixture = await createFixture("openclaw-claw-base-shape-"); + const before = await readFile(fixture.databasePath); + const parsed = parseClawManifest({ + schemaVersion: 1, + agent: { id: "legacy-worker", name: "Legacy Worker" }, + }); + if (!parsed.ok) { + throw new Error(JSON.stringify(parsed.diagnostics)); + } + const source: ClawSourceIdentity = { + kind: "package", + name: "@acme/legacy", + version: "1.1.0", + packageRoot: fixture.packageRoot, + manifestPath: join(fixture.packageRoot, "CLAW.md"), + integrityKind: "artifact", + integrity: "sha256:dd", + byteLength: 12, + }; + + const plan = await buildClawUpdatePlan({ + agentId: "legacy-worker", + targetManifest: parsed.manifest, + targetSource: source, + config: {}, + sourceMcpServers: {}, + stateOptions: { env: fixture.env }, + packagePreflight: async () => ({ + ok: true as const, + action: "install" as const, + integrity: `sha256:${"a".repeat(64)}`, + }), + }); + + expect(plan.blockers).not.toContainEqual(expect.objectContaining({ code: "claw_not_found" })); + expect(plan.blockers).not.toContainEqual( + expect.objectContaining({ code: "claw_identity_mismatch" }), + ); + await expect(readFile(fixture.databasePath)).resolves.toEqual(before); + }); + + it("resumes a base-shape database without mutating it", async () => { + const fixture = await createFixture("openclaw-claw-base-shape-resume-"); + const before = await readFile(fixture.databasePath); + + const state = await readClawResumeStateReadOnly("legacy-worker", { + path: fixture.databasePath, + }); + + expect(state?.record).toMatchObject({ agentId: "legacy-worker", status: "complete" }); + expect(state?.record.bootstrap).toBeUndefined(); + await expect(readFile(fixture.databasePath)).resolves.toEqual(before); + }); +}); diff --git a/src/claws/reader.ts b/src/claws/reader.ts index c245c8b17014..67764cf5f7d5 100644 --- a/src/claws/reader.ts +++ b/src/claws/reader.ts @@ -653,6 +653,6 @@ export async function readClawManifestFile(path: string): Promise { + const ids = new Set(); + const refs = new Set(); + profile.extensions.forEach((extension, index) => { + if (ids.has(extension.id)) { + ctx.addIssue({ + code: "custom", + path: ["extensions", index, "id"], + message: `Extension id ${JSON.stringify(extension.id)} is declared more than once.`, + }); + } + ids.add(extension.id); + const ref = extension.ref.toLowerCase(); + if (refs.has(ref)) { + ctx.addIssue({ + code: "custom", + path: ["extensions", index, "ref"], + message: `Extension ${JSON.stringify(extension.ref)} is declared more than once.`, + }); + } + refs.add(ref); + }); + }); const workspaceSourceSchema = z.object({ source: packageRelativePath }).strict(); const bootstrapFilesSchema = z diff --git a/src/claws/types.ts b/src/claws/types.ts index e346ec86f10d..7117aa203cf9 100644 --- a/src/claws/types.ts +++ b/src/claws/types.ts @@ -29,6 +29,17 @@ type ClawAgent = { }; }; +type ClawExtensionFormat = "openclaw" | "claude" | "codex" | "cursor"; + +export type ClawOpenClawExtension = { + id: string; + kind: "plugin"; + format: ClawExtensionFormat; + source: "clawhub"; + ref: string; + version: string; +}; + export type ClawOpenClawProfile = { schemaVersion: 1; agent: { @@ -73,6 +84,7 @@ export type ClawOpenClawProfile = { maxMs?: number; }; }; + extensions?: ClawOpenClawExtension[]; }; export const CLAW_BOOTSTRAP_FILE_NAMES = [ @@ -102,7 +114,42 @@ export type ClawPackage = { version: string; }; -export type ResolvedClawPackage = ClawPackage & { integrity: string }; +export type ClawAppliedExtension = { + id: string; + format: ClawExtensionFormat; + detectedFormat: ClawExtensionFormat; + mapped: string[]; + unavailable: string[]; + adapterIdentity: string; +}; + +export type ResolvedClawPackage = ClawPackage & { + integrity: string; + extension?: ClawAppliedExtension; +}; + +export type ClawPackagePreflightResult = { + ok: boolean; + action?: "install" | "reuse"; + integrity?: string; + installId?: string; + warning?: string; + installedIntegrity?: string; + installedAt?: string; + installedVersion?: string; + code?: string; + message?: string; + requirements?: ClawLocalPrerequisite[]; + detectedFormat?: ClawExtensionFormat; + mapped?: string[]; + unavailable?: string[]; + adapterIdentity?: string; +}; + +export type ClawPackagePreflight = ( + pkg: ClawPackage, + workspace: string, +) => Promise; type ClawMcpServerCommon = { toolFilter?: { @@ -195,7 +242,7 @@ export type ClawReadResult = export type ClawAddPlanAction = { kind: "agent" | "workspace" | "bootstrap" | "workspaceFile" | "package" | "mcpServer" | "cronJob"; id: string; - action: "create" | "write" | "install" | "configure" | "schedule"; + action: "create" | "write" | "install" | "reuse" | "configure" | "schedule"; target: string; source?: string; sourceKind?: "clawMarkdownBody"; @@ -205,11 +252,23 @@ export type ClawAddPlanAction = { reason?: string; }; +export type ClawExtensionPlan = ClawOpenClawExtension & { + detectedFormat?: ClawExtensionFormat; + integrity?: string; + installId?: string; + ownerAction?: "install" | "reuse"; + requirementState: "satisfied" | "missing-installable" | "conflicting" | "setup-required"; + mapped: string[]; + unavailable: string[]; + adapterIdentity?: string; + blocked: boolean; +}; + export type ClawAddCapabilityChange = { kind: "agent" | "package" | "mcpServer" | "cronJob"; id: string; path: string; - action: "create" | "install" | "configure" | "schedule"; + action: "create" | "install" | "reuse" | "configure" | "schedule"; classification: "escalation"; requiresDistinctConsent: true; reason: string; @@ -258,6 +317,7 @@ export type ClawAddPlan = { ready: boolean; requirements: ClawLocalPrerequisite[]; }; + extensions?: ClawExtensionPlan[]; blockers: ClawDiagnostic[]; diagnostics: ClawDiagnostic[]; }; diff --git a/src/claws/update-apply.test.ts b/src/claws/update-apply.test.ts index 6ef090317c42..9be3a49b6be4 100644 --- a/src/claws/update-apply.test.ts +++ b/src/claws/update-apply.test.ts @@ -3,12 +3,18 @@ import { afterEach, describe, expect, it, vi } from "vitest"; import { useAutoCleanupTempDirTracker } from "../../test/helpers/temp-dir.js"; import type { OpenClawConfig } from "../config/types.openclaw.js"; import { ClawCronUpdateError } from "./cron-update.js"; +import type { buildClawAddPlan } from "./lifecycle.js"; import { persistClawInstallRecord, readClawInstallRecord, type PersistedClawInstall, } from "./provenance.js"; -import type { ClawAddPlan, ClawManifest, ClawSourceIdentity } from "./types.js"; +import type { + ClawAddPlan, + ClawManifest, + ClawOpenClawProfile, + ClawSourceIdentity, +} from "./types.js"; import { applyClawUpdatePlan } from "./update-apply.js"; import type { ClawUpdatePlan } from "./update-plan.js"; @@ -101,6 +107,7 @@ function plan(actions: ClawUpdatePlan["actions"]): ClawUpdatePlan { }, actions, capabilityChanges: [], + readiness: { ready: true, requirements: [] }, blockers: [], diagnostics: [], }; @@ -168,6 +175,37 @@ describe("applyClawUpdatePlan", () => { expect(readInstall).not.toHaveBeenCalled(); }); + it("rejects setup requirements that changed after consent", async () => { + const updatePlan = plan([]); + const changed = { + ...updatePlan, + readiness: { + ready: false, + requirements: [ + { + kind: "plugin-setup" as const, + plugin: "market-data", + provider: "market-data", + envVars: ["MARKET_DATA_TOKEN"], + authMethods: ["token"], + }, + ], + }, + }; + + await expect( + applyClawUpdatePlan( + updatePlan, + { targetManifest: manifest, targetSource: source }, + { + config: {}, + ...consent(updatePlan), + rebuildPlan: async () => changed, + }, + ), + ).rejects.toMatchObject({ code: "update_changed" }); + }); + it("compare-writes the owned agent and advances root provenance", async () => { const currentAgent = { id: "worker", name: "Worker" }; const currentDigest = `sha256:${createHash("sha256").update(stableStringify(currentAgent)).digest("hex")}`; @@ -216,7 +254,7 @@ describe("applyClawUpdatePlan", () => { }); }); - it("activates cron only after package and agent updates succeed", async () => { + it("activates cron only after owned state and agent updates succeed", async () => { const updatePlan = plan([ { kind: "agent", @@ -266,7 +304,94 @@ describe("applyClawUpdatePlan", () => { }, ); - expect(order).toEqual(["workspace", "mcp", "package", "agent", "cron", "provenance"]); + expect(order).toEqual(["workspace", "mcp", "agent", "cron", "provenance"]); + }); + + it("realizes new plugin requirements before workspace mutation and retains them on failure", async () => { + const targetPackage = { + kind: "plugin" as const, + source: "clawhub" as const, + ref: "github", + version: "1.0.0", + }; + const packageDetails = { + ...targetPackage, + integrity: "sha256:github", + installId: "github", + ownerAction: "install" as const, + }; + const desiredDigest = `sha256:${createHash("sha256") + .update( + stableStringify({ + package: targetPackage, + integrity: packageDetails.integrity, + installId: packageDetails.installId, + riskWarning: undefined, + prerequisites: undefined, + extension: undefined, + }), + ) + .digest("hex")}`; + const updatePlan = plan([ + { + kind: "package", + id: "plugin:github", + action: "add", + target: "packages.plugin:github", + blocked: false, + reason: "target adds a shared plugin requirement", + desiredDigest, + }, + ]); + const packageAddPlan: ClawAddPlan = { + ...addPlan, + actions: [ + { + kind: "package", + id: "plugin:github", + action: "install", + target: "clawhub:github@1.0.0", + details: packageDetails, + blocked: false, + }, + ], + }; + const order: string[] = []; + const requirementRollback = vi.fn(async () => undefined); + + await expect( + applyClawUpdatePlan( + updatePlan, + { + targetManifest: { ...manifest, packages: [targetPackage] }, + targetSource: source, + }, + { + config: {}, + ...consent(updatePlan), + rebuildPlan: vi.fn(async () => updatePlan), + buildAddPlan: vi.fn(async () => packageAddPlan), + readInstall: vi.fn(() => install), + applyPackage: vi.fn(async (phase) => { + order.push("requirement"); + expect( + phase.actions.map((action: ClawAddPlan["actions"][number]) => action.id), + ).toEqual(["plugin:github"]); + return { appliedIds: ["plugin:github"], rollback: requirementRollback }; + }), + applyWorkspace: vi.fn(async () => { + order.push("workspace"); + throw new Error("workspace unavailable"); + }), + }, + ), + ).rejects.toMatchObject({ + code: "update_partial", + message: expect.stringContaining("shared requirements were retained"), + }); + + expect(order).toEqual(["requirement", "workspace"]); + expect(requirementRollback).not.toHaveBeenCalled(); }); it("preserves cron prerequisites when the gateway mutation outcome is uncertain", async () => { @@ -369,6 +494,8 @@ describe("applyClawUpdatePlan", () => { integrity: packageDetails.integrity, installId: undefined, riskWarning: undefined, + prerequisites: undefined, + extension: undefined, }), ) .digest("hex")}`; @@ -445,6 +572,8 @@ describe("applyClawUpdatePlan", () => { integrity: resolved.integrity, installId: resolved.installId, riskWarning: resolved.warning, + prerequisites: undefined, + extension: undefined, }), ) .digest("hex")}`; @@ -501,6 +630,154 @@ describe("applyClawUpdatePlan", () => { expect(applyPackage).toHaveBeenCalledOnce(); }); + it("validates and applies profile extension package updates", async () => { + const packageRoot = tempDirs.make("openclaw-claw-extension-update-"); + const targetSource = { + ...source, + packageRoot, + manifestPath: join(packageRoot, "openclaw.claw.json"), + }; + const extension = { + id: "github-tools", + kind: "plugin" as const, + format: "claude" as const, + source: "clawhub" as const, + ref: "github", + version: "2.0.0", + }; + const extensionProvenance = { + id: extension.id, + format: extension.format, + detectedFormat: "claude" as const, + mapped: ["commands", "skills"], + unavailable: ["agents"], + adapterIdentity: "openclaw/current", + }; + const targetPackage = { + kind: extension.kind, + source: extension.source, + ref: extension.ref, + version: extension.version, + }; + const packageDetails = { + ...targetPackage, + integrity: `sha256:${"a".repeat(64)}`, + installId: "github", + ownerAction: "reuse" as const, + extension: extensionProvenance, + }; + const desiredDigest = `sha256:${createHash("sha256") + .update( + stableStringify({ + package: targetPackage, + integrity: packageDetails.integrity, + installId: packageDetails.installId, + riskWarning: undefined, + prerequisites: undefined, + extension: extensionProvenance, + }), + ) + .digest("hex")}`; + const updatePlan = plan([ + { + kind: "package", + id: "plugin:github", + action: "change", + target: "clawhub:github@2.0.0", + blocked: false, + reason: "target profile changes the extension package", + desiredDigest, + }, + ]); + const targetManifest: ClawManifest = { + ...manifest, + schemaVersion: 1, + packages: [], + }; + const targetOpenClawProfile: ClawOpenClawProfile = { + schemaVersion: 1, + agent: {}, + extensions: [extension], + }; + const targetAddPlan: ClawAddPlan = { + ...addPlan, + manifestSchemaVersion: 1, + actions: [ + { + kind: "package", + id: "plugin:github", + action: "install", + target: "clawhub:github@2.0.0", + details: packageDetails, + blocked: false, + }, + ], + }; + const conflictPreflight = { + ok: false as const, + code: "plugin_version_conflict", + message: "The Claw owns the installed previous version.", + installedVersion: "1.0.0", + integrity: packageDetails.integrity, + installId: packageDetails.installId, + detectedFormat: extensionProvenance.detectedFormat, + mapped: extensionProvenance.mapped, + unavailable: extensionProvenance.unavailable, + adapterIdentity: extensionProvenance.adapterIdentity, + }; + const buildAddPlan = vi.fn(async (params: Parameters[0]) => { + const preflight = await params.context?.packagePreflight?.( + targetPackage, + addPlan.agent.workspace, + ); + expect(preflight).toMatchObject({ + ok: true, + action: "install", + integrity: packageDetails.integrity, + installId: packageDetails.installId, + detectedFormat: extensionProvenance.detectedFormat, + mapped: extensionProvenance.mapped, + unavailable: extensionProvenance.unavailable, + adapterIdentity: extensionProvenance.adapterIdentity, + }); + return targetAddPlan; + }); + const applyPackage = vi.fn(async () => ({ + appliedIds: ["plugin:github"], + rollback: vi.fn(async () => undefined), + })); + + await applyClawUpdatePlan( + updatePlan, + { targetManifest, targetOpenClawProfile, targetSource }, + { + config: {}, + ...consent(updatePlan), + rebuildPlan: vi.fn(async () => updatePlan), + buildAddPlan, + packagePreflight: vi.fn(async () => conflictPreflight), + readInstall: vi.fn(() => install), + persistInstall: vi.fn(() => ({ ...install, claw: source })), + applyWorkspace: vi.fn(async () => ({ + appliedPaths: [], + rollback: vi.fn(async () => undefined), + })), + applyMcp: vi.fn(async () => ({ + appliedNames: [], + rollback: vi.fn(async () => undefined), + })), + applyCron: vi.fn(async () => ({ + appliedIds: [], + rollback: vi.fn(async () => undefined), + })), + applyPackage, + }, + ); + + expect(buildAddPlan).toHaveBeenCalledOnce(); + expect(applyPackage).toHaveBeenCalledOnce(); + }); + it("rolls workspace and MCP changes back when root provenance cannot advance", async () => { const updatePlan = plan([ { diff --git a/src/claws/update-apply.ts b/src/claws/update-apply.ts index 7f2fe3e3f3ad..c6ad8df25f80 100644 --- a/src/claws/update-apply.ts +++ b/src/claws/update-apply.ts @@ -5,6 +5,7 @@ import { transformConfigFileWithRetry } from "../config/config.js"; import type { AgentConfig } from "../config/types.agents.js"; import type { OpenClawConfig } from "../config/types.openclaw.js"; import type { OpenClawStateDatabaseOptions } from "../state/openclaw-state-db.js"; +import { clawTargetPackages } from "./application-provenance.js"; import { applyClawCronUpdate, ClawCronUpdateError, @@ -32,7 +33,6 @@ import { CLAW_OUTPUT_STABILITY, type ClawManifest, type ClawOpenClawProfile, - type ClawPackage, type ClawSourceIdentity, } from "./types.js"; import { buildClawUpdatePlan, type ClawUpdateAction, type ClawUpdatePlan } from "./update-plan.js"; @@ -81,6 +81,7 @@ function comparablePlan(plan: ClawUpdatePlan): unknown { targetClaw: plan.targetClaw, actions: plan.actions, capabilityChanges: plan.capabilityChanges, + readiness: plan.readiness, blockers: plan.blockers, }; } @@ -208,6 +209,11 @@ export async function applyClawUpdatePlan( ...(preflight.integrity ? { integrity: preflight.integrity } : {}), ...(preflight.installId ? { installId: preflight.installId } : {}), ...(preflight.warning ? { warning: preflight.warning } : {}), + ...(preflight.requirements ? { requirements: preflight.requirements } : {}), + ...(preflight.detectedFormat ? { detectedFormat: preflight.detectedFormat } : {}), + ...(preflight.mapped ? { mapped: preflight.mapped } : {}), + ...(preflight.unavailable ? { unavailable: preflight.unavailable } : {}), + ...(preflight.adapterIdentity ? { adapterIdentity: preflight.adapterIdentity } : {}), } : preflight; }, @@ -223,9 +229,7 @@ export async function applyClawUpdatePlan( "The target Claw cannot be safely materialized for update.", ); } - const targetPackages = new Map( - params.targetManifest.packages.map((pkg) => [`${pkg.kind}:${pkg.ref}`, pkg] as const), - ); + const targetPackages = clawTargetPackages(params.targetManifest, params.targetOpenClawProfile); for (const action of fresh.actions.filter( (candidate) => candidate.kind === "package" && @@ -245,6 +249,8 @@ export async function applyClawUpdatePlan( integrity: details?.integrity, installId: details?.installId, riskWarning: details?.riskWarning, + prerequisites: details?.prerequisites, + extension: details?.extension, }) ) { throw new ClawUpdateMutationError( @@ -254,6 +260,41 @@ export async function applyClawUpdatePlan( } } + const applyPackage = options.applyPackage ?? applyClawPackageUpdate; + const requirementActions = fresh.actions.filter( + (action) => + action.kind === "package" && + action.action !== "unchanged" && + action.action !== "release" && + action.action !== "remove" && + targetPackages.get(action.id)?.kind === "plugin", + ); + const remainingPackageActions = fresh.actions.filter( + (action) => action.kind === "package" && !requirementActions.includes(action), + ); + const applyPackageActions = async ( + actions: ClawUpdateAction[], + ): Promise => { + if (actions.length === 0) { + return { appliedIds: [], rollback: async () => undefined }; + } + return await applyPackage({ ...fresh, actions }, params.targetManifest, targetAddPlan, options); + }; + + let requirementExecution: ClawPackageUpdateExecution; + try { + requirementExecution = await applyPackageActions(requirementActions); + } catch (error) { + if (error instanceof ClawPackageUpdateError && error.partial) { + throw partialMutation(error.message); + } + throw new ClawUpdateMutationError( + "package_update_failed", + error instanceof Error ? error.message : String(error), + ); + } + const retainedRequirementMutation = requirementExecution.appliedIds.length > 0; + const applyWorkspace = options.applyWorkspace ?? applyClawWorkspaceUpdate; let workspaceExecution: ClawWorkspaceUpdateExecution; try { @@ -262,6 +303,11 @@ export async function applyClawUpdatePlan( if (error instanceof ClawWorkspaceUpdateError && error.partial) { throw partialMutation(error.message); } + if (retainedRequirementMutation) { + throw partialMutation( + `${error instanceof Error ? error.message : String(error)}; successfully realized shared requirements were retained`, + ); + } throw new ClawUpdateMutationError( "workspace_update_failed", error instanceof Error ? error.message : String(error), @@ -284,16 +330,20 @@ export async function applyClawUpdatePlan( if (partial) { throw partialMutation(`${error.message}; MCP config write outcome is uncertain`); } + if (retainedRequirementMutation) { + throw partialMutation( + `${error instanceof Error ? error.message : String(error)}; successfully realized shared requirements were retained`, + ); + } throw new ClawUpdateMutationError( "mcp_update_failed", error instanceof Error ? error.message : String(error), ); } - const applyPackage = options.applyPackage ?? applyClawPackageUpdate; let packageExecution: ClawPackageUpdateExecution; try { - packageExecution = await applyPackage(fresh, params.targetManifest, targetAddPlan, options); + packageExecution = await applyPackageActions(remainingPackageActions); } catch (error) { const rollbackFailures: string[] = []; try { @@ -318,6 +368,11 @@ export async function applyClawUpdatePlan( `${error instanceof Error ? error.message : String(error)}; ${rollbackFailures.join("; ")}`, ); } + if (retainedRequirementMutation) { + throw partialMutation( + `${error instanceof Error ? error.message : String(error)}; successfully realized shared requirements were retained`, + ); + } throw new ClawUpdateMutationError( "package_update_failed", error instanceof Error ? error.message : String(error), @@ -420,6 +475,11 @@ export async function applyClawUpdatePlan( `${error instanceof Error ? error.message : String(error)}; ${rollbackFailures.join("; ")}`, ); } + if (retainedRequirementMutation) { + throw partialMutation( + `${error instanceof Error ? error.message : String(error)}; successfully realized shared requirements were retained`, + ); + } if (error instanceof ClawUpdateMutationError) { throw error; } @@ -484,6 +544,11 @@ export async function applyClawUpdatePlan( `${error instanceof Error ? error.message : String(error)}; ${rollbackFailures.join("; ")}`, ); } + if (retainedRequirementMutation) { + throw partialMutation( + `${error instanceof Error ? error.message : String(error)}; successfully realized shared requirements were retained`, + ); + } throw new ClawUpdateMutationError( "cron_update_failed", error instanceof Error ? error.message : String(error), @@ -538,6 +603,11 @@ export async function applyClawUpdatePlan( `${error instanceof Error ? error.message : String(error)}; ${rollbackFailures.join("; ")}`, ); } + if (retainedRequirementMutation) { + throw partialMutation( + `${error instanceof Error ? error.message : String(error)}; successfully realized shared requirements were retained`, + ); + } throw new ClawUpdateMutationError( "provenance_update_failed", error instanceof Error ? error.message : String(error), diff --git a/src/claws/update-capability-changes.ts b/src/claws/update-capability-changes.ts index c5833b14d9c6..195151758c4c 100644 --- a/src/claws/update-capability-changes.ts +++ b/src/claws/update-capability-changes.ts @@ -472,6 +472,8 @@ export function packageCapabilityChange(params: { integrity?: string; installId?: string; riskWarning?: string; + currentExtension?: unknown; + desiredExtension?: unknown; }): ClawUpdateCapabilityChange | undefined { if (params.pkg.kind !== "plugin" || params.action === "unchanged") { return undefined; @@ -494,15 +496,28 @@ export function packageCapabilityChange(params: { ...(params.integrity ? { integrity: params.integrity } : {}), ...(params.installId ? { installId: params.installId } : {}), ...(params.riskWarning ? { riskWarning: params.riskWarning } : {}), + ...(params.desiredExtension ? { extension: params.desiredExtension } : {}), }, ...(params.currentVersion ? { - current: capabilityValue(`version ${params.currentVersion}`), + current: capabilityValue( + `version ${params.currentVersion}${params.currentExtension ? "; extension mapping recorded" : ""}`, + { + version: params.currentVersion, + extension: params.currentExtension, + }, + ), } : {}), ...(params.desiredVersion ? { - desired: capabilityValue(`version ${params.desiredVersion}`), + desired: capabilityValue( + `version ${params.desiredVersion}${params.desiredExtension ? "; extension mapping updated" : ""}`, + { + version: params.desiredVersion, + extension: params.desiredExtension, + }, + ), } : {}), }; diff --git a/src/claws/update-plan-empty.ts b/src/claws/update-plan-empty.ts index 423bd62cf467..45fbbab04c8b 100644 --- a/src/claws/update-plan-empty.ts +++ b/src/claws/update-plan-empty.ts @@ -41,6 +41,7 @@ export function makeEmptyClawUpdatePlan(params: { }, actions: [], capabilityChanges: [], + readiness: { ready: true, requirements: [] }, blockers: params.blockers, diagnostics: params.diagnostics ?? [], }; diff --git a/src/claws/update-plan-readiness.test.ts b/src/claws/update-plan-readiness.test.ts new file mode 100644 index 000000000000..cdfd7bcf7492 --- /dev/null +++ b/src/claws/update-plan-readiness.test.ts @@ -0,0 +1,208 @@ +import { afterEach, describe, expect, it } from "vitest"; +import { useAutoCleanupTempDirTracker } from "../../test/helpers/temp-dir.js"; +import { closeOpenClawStateDatabaseForTest } from "../state/openclaw-state-db.js"; +import { parseClawManifest } from "./schema.js"; +import { buildClawUpdatePlan } from "./update-plan.js"; +import { createUpdatePlanFixture, targetSource } from "./update-plan.test-helpers.js"; + +const tempDirs = useAutoCleanupTempDirTracker(afterEach); + +afterEach(() => closeOpenClawStateDatabaseForTest()); + +describe("buildClawUpdatePlan readiness", () => { + it("projects and integrity-binds plugin setup prerequisites", async () => { + const current = await createUpdatePlanFixture(tempDirs.make("openclaw-claw-readiness-")); + const requirement = (envVars: string[]) => ({ + kind: "plugin-setup" as const, + plugin: "obsolete", + provider: "market-data", + envVars, + authMethods: ["token"], + }); + const build = async (envVars: string[]) => + await buildClawUpdatePlan({ + agentId: "worker", + targetManifest: current.manifest, + targetSource: targetSource(current.root, "2.0.0", "sha256:target"), + config: current.config, + sourceMcpServers: current.config.mcp?.servers ?? {}, + stateOptions: { + env: current.env, + packageDeps: { + resolvePlugin: async () => ({ + status: "found" as const, + pluginId: "obsolete", + installedVersion: "1.0.0", + record: { source: "clawhub", integrity: `sha256:${"a".repeat(64)}` }, + }), + }, + }, + packagePreflight: async (pkg) => ({ + ok: true, + action: "reuse", + integrity: `sha256:${"a".repeat(64)}`, + ...(pkg.kind === "plugin" + ? { + installId: pkg.ref, + requirements: [requirement(envVars)], + } + : {}), + }), + }); + + const planned = await build(["MARKET_DATA_TOKEN"]); + const changed = await build(["MARKET_DATA_API_KEY"]); + const plannedPackage = planned.actions.find( + (action) => action.kind === "package" && action.id === "plugin:obsolete", + ); + const changedPackage = changed.actions.find( + (action) => action.kind === "package" && action.id === "plugin:obsolete", + ); + + expect(planned.readiness).toEqual({ + ready: false, + requirements: [requirement(["MARKET_DATA_TOKEN"])], + }); + expect(changed.planIntegrity).not.toBe(planned.planIntegrity); + expect(changedPackage?.desiredDigest).not.toBe(plannedPackage?.desiredDigest); + }); + + it("preserves prerequisites from an accepted owned plugin upgrade conflict", async () => { + const current = await createUpdatePlanFixture(tempDirs.make("openclaw-claw-upgrade-ready-")); + const parsed = parseClawManifest({ + ...current.manifest, + packages: current.manifest.packages.map((pkg) => + pkg.kind === "plugin" ? { ...pkg, version: "2.0.0" } : pkg, + ), + }); + if (!parsed.ok) { + throw new Error(JSON.stringify(parsed.diagnostics)); + } + const setupRequirement = { + kind: "plugin-setup" as const, + plugin: "obsolete", + provider: "market-data", + envVars: ["MARKET_DATA_TOKEN"], + authMethods: ["token"], + }; + + const plan = await buildClawUpdatePlan({ + agentId: "worker", + targetManifest: parsed.manifest, + targetSource: targetSource(current.root, "2.0.0", "sha256:target"), + config: current.config, + sourceMcpServers: current.config.mcp?.servers ?? {}, + stateOptions: { + env: current.env, + packageDeps: { + resolvePlugin: async () => ({ + status: "found" as const, + pluginId: "obsolete", + installedVersion: "1.0.0", + record: { source: "clawhub", integrity: `sha256:${"a".repeat(64)}` }, + }), + }, + }, + packagePreflight: async (pkg) => + pkg.kind === "plugin" + ? { + ok: false, + code: "plugin_version_conflict", + installedVersion: "1.0.0", + integrity: `sha256:${"a".repeat(64)}`, + installId: pkg.ref, + requirements: [setupRequirement], + message: "The Claw owns the installed previous version.", + } + : { + ok: true, + action: "reuse", + integrity: `sha256:${"a".repeat(64)}`, + }, + }); + + expect(plan.actions).toContainEqual( + expect.objectContaining({ id: "plugin:obsolete", action: "change", blocked: false }), + ); + expect(plan.readiness).toEqual({ + ready: false, + requirements: [setupRequirement], + }); + }); + + it("accepts an owned plugin upgrade declared through the OpenClaw profile", async () => { + const current = await createUpdatePlanFixture(tempDirs.make("openclaw-claw-profile-upgrade-")); + const parsed = parseClawManifest({ + ...current.manifest, + packages: current.manifest.packages.filter((pkg) => pkg.kind !== "plugin"), + }); + if (!parsed.ok) { + throw new Error(JSON.stringify(parsed.diagnostics)); + } + + const plan = await buildClawUpdatePlan({ + agentId: "worker", + targetManifest: parsed.manifest, + targetOpenClawProfile: { + schemaVersion: 1, + agent: {}, + extensions: [ + { + id: "obsolete-tools", + kind: "plugin", + format: "claude", + source: "clawhub", + ref: "obsolete", + version: "2.0.0", + }, + ], + }, + targetSource: targetSource(current.root, "2.0.0", "sha256:target"), + config: current.config, + sourceMcpServers: current.config.mcp?.servers ?? {}, + stateOptions: { + env: current.env, + packageDeps: { + resolvePlugin: async () => ({ + status: "found" as const, + pluginId: "obsolete", + installedVersion: "1.0.0", + record: { source: "clawhub", integrity: `sha256:${"a".repeat(64)}` }, + }), + }, + }, + packagePreflight: async (pkg) => ({ + ok: false, + code: "plugin_version_conflict", + installedVersion: "1.0.0", + integrity: `sha256:${"a".repeat(64)}`, + installId: pkg.ref, + detectedFormat: "claude", + mapped: ["skills"], + unavailable: ["agents"], + adapterIdentity: "openclaw/test", + message: "The Claw owns the installed previous version.", + }), + }); + + expect(plan.actions).toContainEqual( + expect.objectContaining({ id: "plugin:obsolete", action: "change", blocked: false }), + ); + expect(plan.blockers).not.toContainEqual( + expect.objectContaining({ + code: "plugin_version_conflict", + path: "$.profiles.openclaw.extensions[0]", + }), + ); + expect( + plan.capabilityChanges.find( + (change) => change.kind === "package" && change.id === "plugin:obsolete", + )?.effect.extension, + ).toMatchObject({ + id: "obsolete-tools", + detectedFormat: "claude", + mapped: ["skills"], + unavailable: ["agents"], + }); + }); +}); diff --git a/src/claws/update-plan-types.ts b/src/claws/update-plan-types.ts index 1e0c06e35ada..593552998b5e 100644 --- a/src/claws/update-plan-types.ts +++ b/src/claws/update-plan-types.ts @@ -1,4 +1,9 @@ -import type { CLAW_OUTPUT_STABILITY, ClawDiagnostic, ClawSourceIdentity } from "./types.js"; +import type { + CLAW_OUTPUT_STABILITY, + ClawDiagnostic, + ClawLocalPrerequisite, + ClawSourceIdentity, +} from "./types.js"; import type { ClawUpdateCapabilityChange } from "./update-capability-changes.js"; export const CLAW_UPDATE_PLAN_SCHEMA_VERSION = "openclaw.clawUpdatePlan.v1" as const; @@ -39,6 +44,10 @@ export type ClawUpdatePlan = { }; actions: ClawUpdateAction[]; capabilityChanges: ClawUpdateCapabilityChange[]; + readiness: { + ready: boolean; + requirements: ClawLocalPrerequisite[]; + }; blockers: ClawDiagnostic[]; diagnostics: ClawDiagnostic[]; }; diff --git a/src/claws/update-plan.test-helpers.ts b/src/claws/update-plan.test-helpers.ts new file mode 100644 index 000000000000..c0ab4f1aa4cd --- /dev/null +++ b/src/claws/update-plan.test-helpers.ts @@ -0,0 +1,115 @@ +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import type { McpServerConfig } from "../config/types.mcp.js"; +import type { OpenClawConfig } from "../config/types.openclaw.js"; +import { applyClawAddPlan } from "./add.js"; +import { buildClawAddPlan } from "./lifecycle.js"; +import { installClawMcpServers } from "./mcp.js"; +import { persistClawPackageRef } from "./provenance.js"; +import { parseClawManifest } from "./schema.js"; +import type { ClawSourceIdentity, ResolvedClawPackage } from "./types.js"; + +export const packagePreflight = async (pkg: { kind: "skill" | "plugin"; ref: string }) => ({ + ok: true as const, + action: "install" as const, + integrity: `sha256:${"a".repeat(64)}`, + ...(pkg.kind === "plugin" ? { installId: pkg.ref } : {}), +}); + +export async function createUpdatePlanFixture(root: string) { + await writeFile(join(root, "SOUL.md"), "base soul\n", "utf8"); + await writeFile(join(root, "OLD.md"), "old\n", "utf8"); + const raw = { + schemaVersion: 1, + agent: { id: "worker", name: "Worker" }, + workspace: { + bootstrapFiles: { "SOUL.md": { source: "SOUL.md" } }, + files: [{ source: "OLD.md", path: "OLD.md" }], + }, + packages: [ + { + kind: "skill", + source: "clawhub", + ref: "triage", + version: "1.0.0", + }, + { + kind: "plugin", + source: "clawhub", + ref: "obsolete", + version: "1.0.0", + }, + ], + mcpServers: { docs: { command: "uvx", args: ["docs-mcp"] } }, + cronJobs: [ + { + id: "daily", + schedule: { cron: "0 9 * * *", timezone: "UTC" }, + session: "isolated", + message: "Base report", + }, + ], + }; + const parsed = parseClawManifest(raw); + if (!parsed.ok) { + throw new Error(JSON.stringify(parsed.diagnostics)); + } + const source: ClawSourceIdentity = { + kind: "package", + name: "@acme/worker", + version: "1.0.0", + packageRoot: root, + manifestPath: join(root, "openclaw.claw.json"), + integrityKind: "artifact", + integrity: "sha256:base", + byteLength: 100, + }; + const env = { OPENCLAW_STATE_DIR: join(root, "state") }; + const addPlan = await buildClawAddPlan({ + manifest: parsed.manifest, + source, + context: { workspace: join(root, "workspace-worker"), packagePreflight }, + }); + if (addPlan.blockers.length > 0) { + throw new Error(JSON.stringify(addPlan.blockers)); + } + let config: OpenClawConfig = {}; + await applyClawAddPlan(addPlan, { + consentPlanIntegrity: addPlan.planIntegrity, + env, + commitConfig: async (transform) => { + config = transform(config); + }, + installPackages: async (plan, options) => + plan.actions + .filter((action) => action.kind === "package") + .map((action) => + persistClawPackageRef(plan, action.details as ResolvedClawPackage, options), + ), + installMcpServers: async (plan, options) => + await installClawMcpServers(plan, { + ...options, + setMcpServer: async ({ name, server }) => { + const servers = { ...config.mcp?.servers, [name]: server as McpServerConfig }; + config.mcp = { ...config.mcp, servers }; + return { ok: true, path: "config", config, mcpServers: servers }; + }, + listMcpServers: async () => ({ ok: true, path: "config", config, mcpServers: {} }), + }), + cronGateway: { add: async () => ({ id: "scheduler-daily" }) }, + }); + return { root, env, config, manifest: parsed.manifest, source, addPlan }; +} + +export function targetSource(root: string, version: string, integrity: string): ClawSourceIdentity { + return { + kind: "package", + name: "@acme/worker", + version, + packageRoot: root, + manifestPath: join(root, "openclaw.claw.json"), + integrityKind: "artifact", + integrity, + byteLength: 100, + }; +} diff --git a/src/claws/update-plan.test.ts b/src/claws/update-plan.test.ts index 5c5ebb8c7adf..9c1a9bfbd440 100644 --- a/src/claws/update-plan.test.ts +++ b/src/claws/update-plan.test.ts @@ -4,130 +4,29 @@ import { join } from "node:path"; import { stableStringify } from "@openclaw/normalization-core"; import { afterEach, describe, expect, it } from "vitest"; import { useAutoCleanupTempDirTracker } from "../../test/helpers/temp-dir.js"; -import type { McpServerConfig } from "../config/types.mcp.js"; -import type { OpenClawConfig } from "../config/types.openclaw.js"; import { requireNodeSqlite } from "../infra/node-sqlite.js"; import { closeOpenClawStateDatabaseForTest, openOpenClawStateDatabase, } from "../state/openclaw-state-db.js"; import { resolveOpenClawStateSqlitePath } from "../state/openclaw-state-db.paths.js"; -import { applyClawAddPlan } from "./add.js"; -import { buildClawAddPlan } from "./lifecycle.js"; -import { installClawMcpServers } from "./mcp.js"; import { persistClawPackageRef } from "./provenance.js"; import { parseClawManifest } from "./schema.js"; -import type { ClawPackage, ClawSourceIdentity, ResolvedClawPackage } from "./types.js"; +import type { ClawPackage } from "./types.js"; import { buildClawUpdatePlan } from "./update-plan.js"; +import { + createUpdatePlanFixture, + packagePreflight, + targetSource, +} from "./update-plan.test-helpers.js"; const tempDirs = useAutoCleanupTempDirTracker(afterEach); afterEach(() => closeOpenClawStateDatabaseForTest()); -const packagePreflight = async (pkg: { kind: "skill" | "plugin"; ref: string }) => ({ - ok: true as const, - action: "install" as const, - integrity: `sha256:${"a".repeat(64)}`, - ...(pkg.kind === "plugin" ? { installId: pkg.ref } : {}), -}); - async function fixture() { const root = tempDirs.make("openclaw-claw-update-"); - await writeFile(join(root, "SOUL.md"), "base soul\n", "utf8"); - await writeFile(join(root, "OLD.md"), "old\n", "utf8"); - const raw = { - schemaVersion: 1, - agent: { id: "worker", name: "Worker" }, - workspace: { - bootstrapFiles: { "SOUL.md": { source: "SOUL.md" } }, - files: [{ source: "OLD.md", path: "OLD.md" }], - }, - packages: [ - { - kind: "skill", - source: "clawhub", - ref: "triage", - version: "1.0.0", - }, - { - kind: "plugin", - source: "clawhub", - ref: "obsolete", - version: "1.0.0", - }, - ], - mcpServers: { docs: { command: "uvx", args: ["docs-mcp"] } }, - cronJobs: [ - { - id: "daily", - schedule: { cron: "0 9 * * *", timezone: "UTC" }, - session: "isolated", - message: "Base report", - }, - ], - }; - const parsed = parseClawManifest(raw); - if (!parsed.ok) { - throw new Error(JSON.stringify(parsed.diagnostics)); - } - const source: ClawSourceIdentity = { - kind: "package", - name: "@acme/worker", - version: "1.0.0", - packageRoot: root, - manifestPath: join(root, "openclaw.claw.json"), - integrityKind: "artifact", - integrity: "sha256:base", - byteLength: 100, - }; - const env = { OPENCLAW_STATE_DIR: join(root, "state") }; - const addPlan = await buildClawAddPlan({ - manifest: parsed.manifest, - source, - context: { workspace: join(root, "workspace-worker"), packagePreflight }, - }); - if (addPlan.blockers.length > 0) { - throw new Error(JSON.stringify(addPlan.blockers)); - } - let config: OpenClawConfig = {}; - await applyClawAddPlan(addPlan, { - consentPlanIntegrity: addPlan.planIntegrity, - env, - commitConfig: async (transform) => { - config = transform(config); - }, - installPackages: async (plan, options) => - plan.actions - .filter((action) => action.kind === "package") - .map((action) => - persistClawPackageRef(plan, action.details as ResolvedClawPackage, options), - ), - installMcpServers: async (plan, options) => - await installClawMcpServers(plan, { - ...options, - setMcpServer: async ({ name, server }) => { - const servers = { ...config.mcp?.servers, [name]: server as McpServerConfig }; - config.mcp = { ...config.mcp, servers }; - return { ok: true, path: "config", config, mcpServers: servers }; - }, - listMcpServers: async () => ({ ok: true, path: "config", config, mcpServers: {} }), - }), - cronGateway: { add: async () => ({ id: "scheduler-daily" }) }, - }); - return { root, env, config, manifest: parsed.manifest, source, addPlan }; -} - -function targetSource(root: string, version: string, integrity: string): ClawSourceIdentity { - return { - kind: "package", - name: "@acme/worker", - version, - packageRoot: root, - manifestPath: join(root, "openclaw.claw.json"), - integrityKind: "artifact", - integrity, - byteLength: 100, - }; + return await createUpdatePlanFixture(root); } describe("buildClawUpdatePlan", () => { @@ -163,6 +62,81 @@ describe("buildClawUpdatePlan", () => { expect((await stat(databasePath)).mtimeMs).toBe(beforeStat.mtimeMs); }); + it("moves a portable plugin dependency into a profile extension edge without reinstalling", async () => { + const current = await fixture(); + const parsed = parseClawManifest({ + ...current.manifest, + packages: current.manifest.packages.filter((pkg) => pkg.ref !== "obsolete"), + }); + if (!parsed.ok) { + throw new Error(JSON.stringify(parsed.diagnostics)); + } + + const plan = await buildClawUpdatePlan({ + agentId: "worker", + targetManifest: parsed.manifest, + targetOpenClawProfile: { + schemaVersion: 1, + agent: {}, + extensions: [ + { + id: "obsolete-tools", + kind: "plugin", + format: "claude", + source: "clawhub", + ref: "obsolete", + version: "1.0.0", + }, + ], + }, + targetSource: targetSource(current.root, "2.0.0", "sha256:target"), + config: current.config, + sourceMcpServers: current.config.mcp?.servers ?? {}, + stateOptions: { + env: current.env, + packageDeps: { + resolvePlugin: async () => ({ + status: "found" as const, + pluginId: "obsolete", + installedVersion: "1.0.0", + record: { source: "clawhub", integrity: `sha256:${"a".repeat(64)}` }, + }), + }, + }, + packagePreflight: async () => ({ + ok: true, + action: "reuse", + integrity: `sha256:${"a".repeat(64)}`, + installId: "obsolete", + detectedFormat: "claude", + mapped: ["skills"], + unavailable: ["agents"], + adapterIdentity: "openclaw/test", + }), + }); + + expect(plan.actions).toContainEqual( + expect.objectContaining({ + kind: "package", + id: "plugin:obsolete", + action: "change", + reason: expect.stringContaining("without reinstalling"), + }), + ); + expect(plan.actions).not.toContainEqual( + expect.objectContaining({ kind: "package", id: "plugin:obsolete", action: "release" }), + ); + const extensionChange = plan.capabilityChanges.find( + (change) => change.kind === "package" && change.id === "plugin:obsolete", + ); + expect(extensionChange?.effect.extension).toMatchObject({ + id: "obsolete-tools", + mapped: ["skills"], + unavailable: ["agents"], + }); + expect(extensionChange?.current?.digest).not.toBe(extensionChange?.desired?.digest); + }); + it("plans missing package restoration without mutating state", async () => { const current = await fixture(); const beforeConfig = structuredClone(current.config); diff --git a/src/claws/update-plan.ts b/src/claws/update-plan.ts index 04438ef2c697..7129496491b4 100644 --- a/src/claws/update-plan.ts +++ b/src/claws/update-plan.ts @@ -9,6 +9,15 @@ import { openExistingOpenClawStateDatabaseReadOnly, type OpenClawStateDatabaseOptions, } from "../state/openclaw-state-db.js"; +import { + clawExtensionProvenanceChanged, + clawPackageActionsById, + clawPackageKey, + clawTargetPackages, + clawWorkspaceActionsById, + isApplicationUpdateBlocker, + recordingClawPackagePreflight, +} from "./application-provenance.js"; import { readClawStatus } from "./lifecycle-state.js"; import { buildClawAddPlan } from "./lifecycle.js"; import { digestClawMcpServer, readClawMcpServerRefsByName } from "./mcp.js"; @@ -20,7 +29,8 @@ import { type ClawDiagnostic, type ClawManifest, type ClawOpenClawProfile, - type ClawPackage, + type ClawPackagePreflight, + type ClawPackagePreflightResult, type ClawSourceIdentity, } from "./types.js"; import { @@ -65,19 +75,7 @@ export async function buildClawUpdatePlan(params: { config: OpenClawConfig; sourceMcpServers: Record>; stateOptions?: OpenClawStateDatabaseOptions & { packageDeps?: PackageRemovalDeps }; - packagePreflight?: ( - pkg: ClawPackage, - workspaceDir: string, - ) => Promise<{ - ok: boolean; - action?: "install" | "reuse"; - code?: string; - message?: string; - installedVersion?: string; - integrity?: string; - installId?: string; - warning?: string; - }>; + packagePreflight?: ClawPackagePreflight; diagnostics?: ClawDiagnostic[]; }): Promise { const ownsDatabase = !params.stateOptions?.database; @@ -133,6 +131,7 @@ export async function buildClawUpdatePlan(params: { ...readOnlyStateOptions, config: params.config, sourceMcpServers: params.sourceMcpServers, + ...(params.packagePreflight ? { packagePreflight: params.packagePreflight } : {}), }); if (status.records.length === 0) { return makeEmptyClawUpdatePlan({ @@ -189,20 +188,10 @@ export async function buildClawUpdatePlan(params: { }); } - const packageKey = (value: { kind: string; ref: string }) => `${value.kind}:${value.ref}`; - const packagePreflights = new Map< - string, - { - ok: boolean; - action?: "install" | "reuse"; - code?: string; - message?: string; - installedVersion?: string; - integrity?: string; - installId?: string; - warning?: string; - } - >(); + const packagePreflights = new Map(); + const currentPackages = new Map( + record.packages.map((pkg) => [clawPackageKey(pkg), pkg] as const), + ); const targetPlan = await buildClawAddPlan({ manifest: params.targetManifest, clawMarkdownBody: params.targetClawMarkdownBody, @@ -213,25 +202,15 @@ export async function buildClawUpdatePlan(params: { context: { agentId, workspace: record.install.workspace, - packagePreflight: async (pkg) => { - const result = params.packagePreflight - ? await params.packagePreflight(pkg, record.install.workspace) - : { - ok: false, - code: "package_install_unavailable", - message: "Package preflight is unavailable.", - }; - packagePreflights.set(packageKey(pkg), result); - return result; - }, + packagePreflight: recordingClawPackagePreflight( + params.packagePreflight, + record.install.workspace, + packagePreflights, + currentPackages, + ), }, }); - const blockers = targetPlan.blockers.filter( - (entry) => - entry.code !== "workspace_collision" && - entry.code !== "agent_id_collision" && - !entry.path.startsWith("$.packages"), - ); + const blockers = targetPlan.blockers.filter(isApplicationUpdateBlocker); const actions: ClawUpdateAction[] = []; const capabilityChanges: ClawUpdateCapabilityChange[] = []; @@ -270,11 +249,7 @@ export async function buildClawUpdatePlan(params: { desiredAgent: targetPlan.agent.config, }); - const targetFiles = new Map( - targetPlan.actions - .filter((action) => action.kind === "workspaceFile") - .map((action) => [action.id, action] as const), - ); + const targetFiles = clawWorkspaceActionsById(targetPlan.actions); const currentFiles = new Map(record.workspaceFiles.map((file) => [file.path, file] as const)); let workspace: Awaited> | undefined; let workspaceState: "present" | "missing" | "unsafe" = "present"; @@ -382,28 +357,19 @@ export async function buildClawUpdatePlan(params: { } const allPackages = readClawPackageRefs(readOnlyStateOptions); - const currentPackages = new Map(record.packages.map((pkg) => [packageKey(pkg), pkg] as const)); - const targetPackages = new Map( - params.targetManifest.packages.map((pkg) => [packageKey(pkg), pkg] as const), - ); + const targetPackages = clawTargetPackages(params.targetManifest, params.targetOpenClawProfile); + const targetPackageActions = clawPackageActionsById(targetPlan.actions); for (const [key, target] of targetPackages) { const current = currentPackages.get(key); const preflight = packagePreflights.get(key); + const targetAction = targetPackageActions.get(key); + const extensionChanged = clawExtensionProvenanceChanged(current?.extension, targetAction); const requiresPackageMutation = !current || (current.origin === "claw-introduced" && !current.independentOwner && (current.state === "missing" || current.version !== target.version)); - const expectedOwnedPluginUpgradeConflict = - target.kind === "plugin" && - current?.state === "present" && - current.origin === "claw-introduced" && - !current.independentOwner && - current.version !== target.version && - preflight?.code === "plugin_version_conflict" && - preflight.installedVersion === current.version; - const failedPackageMutationPreflight = - requiresPackageMutation && !preflight?.ok && !expectedOwnedPluginUpgradeConflict; + const failedPackageMutationPreflight = requiresPackageMutation && !preflight?.ok; const conflictingPluginPin = target.kind === "plugin" && allPackages.some( @@ -430,7 +396,7 @@ export async function buildClawUpdatePlan(params: { ? "add" : current.state === "missing" ? "change" - : current.version === target.version + : current.version === target.version && !extensionChanged ? "unchanged" : "change"; actions.push({ @@ -451,14 +417,18 @@ export async function buildClawUpdatePlan(params: { : action === "add" ? "Target manifest adds a package reference." : action === "unchanged" - ? "Recorded package reference already matches the exact target version." - : "Target manifest changes the exact package version.", + ? "Recorded package reference already matches the exact target version and extension mapping." + : current?.version === target.version + ? "Target profile changes extension provenance without reinstalling the package." + : "Target manifest changes the exact package version.", ...(current ? { currentDigest: digestClawPackageRef(current) } : {}), desiredDigest: digest({ package: target, integrity: preflight?.integrity, installId: preflight?.installId, riskWarning: preflight?.warning, + prerequisites: preflight?.requirements, + extension: targetAction?.details?.extension, }), }); const capabilityChange = packageCapabilityChange({ @@ -469,19 +439,28 @@ export async function buildClawUpdatePlan(params: { integrity: preflight?.integrity, installId: preflight?.installId, riskWarning: preflight?.warning, + currentExtension: current?.extension, + desiredExtension: targetAction?.details?.extension, }); if (capabilityChange) { capabilityChanges.push(capabilityChange); } if (failedPackageMutationPreflight) { - const index = params.targetManifest.packages.findIndex((pkg) => packageKey(pkg) === key); - blockers.push( - diagnostic( - preflight?.code ?? "package_install_unavailable", - `$.packages[${index}]`, - preflight?.message ?? "Package preflight failed.", - ), + const packageIndex = params.targetManifest.packages.findIndex( + (pkg) => clawPackageKey(pkg) === key, ); + const extensionIndex = + params.targetOpenClawProfile?.extensions?.findIndex( + (extension) => clawPackageKey(extension) === key, + ) ?? -1; + const path = + packageIndex >= 0 + ? `$.packages[${packageIndex}]` + : `$.profiles.openclaw.extensions[${extensionIndex}]`; + const code = preflight?.code ?? "package_install_unavailable"; + if (!blockers.some((entry) => entry.code === code && entry.path === path)) { + blockers.push(diagnostic(code, path, preflight?.message ?? "Package preflight failed.")); + } } } for (const [key, current] of currentPackages) { @@ -696,6 +675,7 @@ export async function buildClawUpdatePlan(params: { summary: summarizeClawUpdatePlan(actions, capabilityChanges), actions, capabilityChanges, + readiness: targetPlan.readiness, blockers, diagnostics: params.diagnostics ?? [], }; diff --git a/src/cli/claws-cli-update-output.test.ts b/src/cli/claws-cli-update-output.test.ts new file mode 100644 index 000000000000..1665d5af7685 --- /dev/null +++ b/src/cli/claws-cli-update-output.test.ts @@ -0,0 +1,39 @@ +import { describe, expect, it } from "vitest"; +import { makeEmptyClawUpdatePlan } from "../claws/update-plan-empty.js"; +import type { RuntimeEnv } from "../runtime.js"; +import { logClawUpdatePlanSummary } from "./claws-cli-update-output.js"; + +describe("logClawUpdatePlanSummary", () => { + it("prints plugin setup prerequisites", () => { + const logs: string[] = []; + const runtime = { + log: (value: unknown) => logs.push(String(value)), + error: () => undefined, + exit: () => undefined, + } as RuntimeEnv; + const plan = { + ...makeEmptyClawUpdatePlan({ + agentId: "worker", + blockers: [], + digest: () => "sha256:plan", + }), + readiness: { + ready: false, + requirements: [ + { + kind: "plugin-setup" as const, + plugin: "market-data", + provider: "market-data", + envVars: ["MARKET_DATA_TOKEN"], + authMethods: ["token"], + }, + ], + }, + }; + + logClawUpdatePlanSummary(plan, runtime); + + expect(logs.join("\n")).toContain("Setup requirements (1)"); + expect(logs.join("\n")).toContain("MARKET_DATA_TOKEN"); + }); +}); diff --git a/src/cli/claws-cli-update-output.ts b/src/cli/claws-cli-update-output.ts index 987723e8cbc2..7d18d5efb895 100644 --- a/src/cli/claws-cli-update-output.ts +++ b/src/cli/claws-cli-update-output.ts @@ -25,6 +25,12 @@ export function logClawUpdatePlanSummary(plan: ClawUpdatePlan, runtime: RuntimeE ); runtime.log(redactSensitiveText(` effect: ${JSON.stringify(change.effect)}`)); } + if (plan.readiness.requirements.length > 0) { + runtime.log(`Setup requirements (${plan.readiness.requirements.length}):`); + for (const requirement of plan.readiness.requirements) { + runtime.log(redactSensitiveText(` - ${JSON.stringify(requirement)}`)); + } + } if (plan.blockers.length > 0) { runtime.error( plan.blockers diff --git a/src/cli/claws-cli.inspect.test.ts b/src/cli/claws-cli.inspect.test.ts new file mode 100644 index 000000000000..5c8690b5478f --- /dev/null +++ b/src/cli/claws-cli.inspect.test.ts @@ -0,0 +1,168 @@ +import { mkdir, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { useAutoCleanupTempDirTracker } from "../../test/helpers/temp-dir.js"; + +const mocks = vi.hoisted(() => ({ + preflightClawPackage: vi.fn(), +})); + +vi.mock("../claws/packages.js", async () => ({ + ...(await vi.importActual("../claws/packages.js")), + preflightClawPackage: mocks.preflightClawPackage, +})); + +const { runClawsInspectCommand } = await import("./claws-cli.runtime.js"); +const tempDirs = useAutoCleanupTempDirTracker(afterEach); + +describe("claws inspect extensions", () => { + beforeEach(() => { + vi.stubEnv("OPENCLAW_EXPERIMENTAL_CLAWS", "1"); + mocks.preflightClawPackage.mockReset(); + }); + + it("reports canonical profile extension mappings", async () => { + const root = tempDirs.make("openclaw-claws-inspect-extension-"); + await mkdir(join(root, "profiles")); + await writeFile( + join(root, "package.json"), + JSON.stringify({ + name: "@acme/demo-agent", + version: "1.2.3", + openclaw: { claw: "openclaw.claw.json" }, + }), + "utf8", + ); + await writeFile( + join(root, "openclaw.claw.json"), + JSON.stringify({ schemaVersion: 1, agent: { id: "demo-agent" } }), + "utf8", + ); + await writeFile( + join(root, "profiles", "openclaw.yml"), + [ + "schemaVersion: 1", + "agent: {}", + "extensions:", + " - id: audit-tools", + " kind: plugin", + " format: claude", + " source: clawhub", + " ref: '@owner/audit'", + " version: 2.0.1", + "", + ].join("\n"), + "utf8", + ); + mocks.preflightClawPackage.mockResolvedValue({ + ok: true, + action: "install", + integrity: `sha256:${"a".repeat(64)}`, + installId: "audit", + detectedFormat: "claude", + mapped: ["commands", "skills"], + unavailable: ["agents"], + adapterIdentity: "openclaw/test", + }); + const values: unknown[] = []; + const runtime = { + log: vi.fn(), + error: vi.fn(), + writeJson: vi.fn((value: unknown) => values.push(value)), + writeStdout: vi.fn(), + exit: vi.fn(), + }; + + await runClawsInspectCommand(root, { json: true }, runtime); + + expect(values[0]).toMatchObject({ + valid: true, + extensions: [ + { + id: "audit-tools", + detectedFormat: "claude", + mapped: ["commands", "skills"], + unavailable: ["agents"], + adapterIdentity: "openclaw/test", + }, + ], + }); + expect(mocks.preflightClawPackage).toHaveBeenCalledWith( + expect.objectContaining({ kind: "plugin", ref: "@owner/audit" }), + root, + ); + expect(runtime.exit).not.toHaveBeenCalled(); + }); + + it("rejects a plugin declared by both the portable manifest and OpenClaw profile", async () => { + const root = tempDirs.make("openclaw-claws-inspect-extension-collision-"); + await mkdir(join(root, "profiles")); + await writeFile( + join(root, "package.json"), + JSON.stringify({ + name: "@acme/demo-agent", + version: "1.2.3", + openclaw: { claw: "openclaw.claw.json" }, + }), + "utf8", + ); + await writeFile( + join(root, "openclaw.claw.json"), + JSON.stringify({ + schemaVersion: 1, + agent: { id: "demo-agent" }, + packages: [ + { + kind: "plugin", + source: "clawhub", + ref: "@owner/audit", + version: "2.0.1", + }, + ], + }), + "utf8", + ); + await writeFile( + join(root, "profiles", "openclaw.yml"), + [ + "schemaVersion: 1", + "agent: {}", + "extensions:", + " - id: audit-tools", + " kind: plugin", + " format: openclaw", + " source: clawhub", + " ref: '@owner/audit'", + " version: 2.0.1", + "", + ].join("\n"), + "utf8", + ); + mocks.preflightClawPackage.mockResolvedValue({ + ok: true, + action: "install", + integrity: `sha256:${"a".repeat(64)}`, + installId: "audit", + detectedFormat: "openclaw", + mapped: ["skills"], + unavailable: [], + adapterIdentity: "openclaw/test", + }); + const values: unknown[] = []; + const runtime = { + log: vi.fn(), + error: vi.fn(), + writeJson: vi.fn((value: unknown) => values.push(value)), + writeStdout: vi.fn(), + exit: vi.fn(), + }; + + await runClawsInspectCommand(root, { json: true }, runtime); + + expect(values[0]).toMatchObject({ + valid: false, + diagnostics: [expect.objectContaining({ code: "extension_package_collision" })], + }); + expect(runtime.exit).toHaveBeenCalledWith(1); + }); +}); diff --git a/src/cli/claws-cli.runtime.ts b/src/cli/claws-cli.runtime.ts index e70dc8aedce2..d1ab8adccfd0 100644 --- a/src/cli/claws-cli.runtime.ts +++ b/src/cli/claws-cli.runtime.ts @@ -10,6 +10,10 @@ import { CLAW_ADD_RESULT_SCHEMA_VERSION, ClawAddMutationError, } from "../claws/add.js"; +import { + findClawExtensionPackageCollisions, + planClawExtensions, +} from "../claws/application-plan.js"; import { assertExperimentalClawsEnabled } from "../claws/experimental.js"; import { CLAW_EXPORT_RESULT_SCHEMA_VERSION, @@ -25,8 +29,16 @@ import { readClawStatus, } from "../claws/lifecycle-state.js"; import { buildClawAddPlan } from "../claws/lifecycle.js"; +import { + findResumableIntroducedPluginRequirement, + readClawResumeStateReadOnly, +} from "../claws/package-resume.js"; import { preflightClawPackage } from "../claws/packages.js"; -import { readClawInstallRecord } from "../claws/provenance.js"; +import { + clawInstallRecordMatchesPlan, + readClawInstallRecord, + readClawPackageRefs, +} from "../claws/provenance.js"; import { readClawManifestFile } from "../claws/reader.js"; import { CLAW_INSPECT_RESULT_SCHEMA_VERSION, @@ -74,6 +86,15 @@ function logClawAddPlanSummary(plan: ClawAddPlan, runtime: RuntimeEnv): void { runtime.log(`Workspace: ${plan.agent.workspace}`); runtime.log(`Actions: ${plan.summary.totalActions}`); runtime.log(`Packages: ${plan.summary.packageActions}`); + for (const action of plan.actions.filter((candidate) => candidate.kind === "package")) { + const requirementState = + typeof action.details?.requirementState === "string" + ? action.details.requirementState + : "unresolved"; + runtime.log( + ` Requirement ${action.target}: ${requirementState}${action.action === "install" ? " (installation requires this exact plan consent)" : ""}`, + ); + } runtime.log(`MCP servers: ${plan.summary.mcpServerActions}`); for (const action of plan.actions.filter((candidate) => candidate.kind === "mcpServer")) { const server = action.details as Record | undefined; @@ -105,15 +126,14 @@ function logClawAddPlanSummary(plan: ClawAddPlan, runtime: RuntimeEnv): void { } } -function matchingResumeRecord(plan: ClawAddPlan, opts: ClawsAddOptions) { - if (opts.dryRun || !opts.yes || !opts.planIntegrity) { - return undefined; - } - const record = readClawInstallRecord(plan.agent.finalId); +async function matchingResumeState(plan: ClawAddPlan, opts: ClawsAddOptions) { + const readOnlyState = opts.dryRun + ? await readClawResumeStateReadOnly(plan.agent.finalId) + : undefined; + const record = opts.dryRun ? readOnlyState?.record : readClawInstallRecord(plan.agent.finalId); if ( !record || record.status === "complete" || - record.planIntegrity !== opts.planIntegrity || record.workspace !== plan.agent.workspace || record.claw.kind !== plan.claw.kind || record.claw.name !== plan.claw.name || @@ -122,7 +142,10 @@ function matchingResumeRecord(plan: ClawAddPlan, opts: ClawsAddOptions) { ) { return undefined; } - return record; + return { + record, + packageRefs: readOnlyState?.packageRefs ?? readClawPackageRefs({ agentId: plan.agent.finalId }), + }; } function failNonDryRun(opts: ClawsAddOptions, runtime: RuntimeEnv): boolean { @@ -195,25 +218,54 @@ export async function runClawsInspectCommand( return; } + const extensionPlan = await planClawExtensions({ + extensions: result.openClawProfile?.extensions ?? [], + workspace: result.source.packageRoot, + packagePreflight: preflightClawPackage, + }); + const extensionCollisions = findClawExtensionPackageCollisions({ + packages: result.manifest.packages, + extensions: result.openClawProfile?.extensions ?? [], + }); + const diagnostics = [ + ...result.diagnostics, + ...extensionPlan.blockers, + ...extensionCollisions.map(({ diagnostic }) => diagnostic), + ]; + const valid = diagnostics.every((diagnostic) => diagnostic.level !== "error"); const payload = { schemaVersion: CLAW_INSPECT_RESULT_SCHEMA_VERSION, stability: CLAW_OUTPUT_STABILITY, - valid: true, + valid, source: result.source, manifest: result.manifest, ...(result.openClawProfile ? { openClawProfile: result.openClawProfile } : {}), - diagnostics: result.diagnostics, + extensions: extensionPlan.extensions, + diagnostics, }; if (opts.json) { writeRuntimeJson(runtime, payload); + if (!valid) { + runtime.exit(1); + } return; } logExperimentalWarning(runtime); runtime.log(`Claw: ${result.source.name}@${result.source.version}`); runtime.log(`Agent: ${result.manifest.agent.name ?? result.manifest.agent.id}`); runtime.log(`Packages: ${result.manifest.packages.length}`); + runtime.log(`Extension requirements: ${extensionPlan.extensions.length}`); + for (const extension of extensionPlan.extensions) { + runtime.log( + ` ${extension.id}: ${extension.requirementState}; ${extension.detectedFormat ?? "unresolved"} -> ${(extension.mapped ?? []).join(", ") || "no mapped capabilities"}`, + ); + } runtime.log(`MCP servers: ${Object.keys(result.manifest.mcpServers).length}`); runtime.log(`Cron jobs: ${result.manifest.cronJobs.length}`); + if (!valid) { + runtime.error(formatDiagnostics(diagnostics)); + runtime.exit(1); + } } export async function runClawsAddCommand( @@ -271,8 +323,23 @@ export async function runClawsAddCommand( diagnostics: result.diagnostics, context: basePlanContext, }); - const resumeRecord = matchingResumeRecord(plan, opts); - if (resumeRecord && plan.blockers.length > 0) { + const resumeState = await matchingResumeState(plan, opts); + if (resumeState) { + const { record: resumeRecord, packageRefs: resumePackageRefs } = resumeState; + const packagePreflight = async ( + pkg: Parameters[0], + workspace: string, + ) => { + const preflight = await preflightClawPackage(pkg, workspace); + return findResumableIntroducedPluginRequirement({ + agentId: resumeRecord.agentId, + pkg, + preflight, + refs: resumePackageRefs, + }) + ? { ...preflight, action: "install" as const } + : preflight; + }; const canResumeWorkspace = resumeRecord.status === "workspace_ready" || resumeRecord.status === "config_committed"; const committedAgent = listAgentEntries(config).find( @@ -290,6 +357,7 @@ export async function runClawsAddCommand( diagnostics: result.diagnostics, context: { ...basePlanContext, + packagePreflight, existingAgentIds: canResumeAgent ? existingAgentIds.filter((agentId) => agentId !== resumeRecord.agentId) : existingAgentIds, @@ -301,6 +369,22 @@ export async function runClawsAddCommand( ...(canResumeWorkspace ? { resumableWorkspace: resumeRecord.workspace } : {}), }, }); + if (plan.blockers.length === 0 && !clawInstallRecordMatchesPlan(resumeRecord, plan)) { + plan = { + ...plan, + blockers: [ + ...plan.blockers, + { + level: "error", + code: "claw_resume_plan_mismatch", + phase: "plan", + path: "$", + message: + "The incomplete Claw add no longer matches the current plan; remove its partial state before retrying.", + }, + ], + }; + } } if (plan.blockers.length > 0) { diff --git a/src/cli/claws-cli.test-helpers.ts b/src/cli/claws-cli.test-helpers.ts new file mode 100644 index 000000000000..656dde62c9ed --- /dev/null +++ b/src/cli/claws-cli.test-helpers.ts @@ -0,0 +1,34 @@ +import { realpath, writeFile } from "node:fs/promises"; +import { basename, dirname, join } from "node:path"; + +export const minimalManifest = { + schemaVersion: 1, + agent: { id: "demo-agent", name: "Demo Agent" }, +}; + +export const pluginSetupReadiness = { + ready: false, + requirements: [ + { + kind: "plugin-setup" as const, + plugin: "market-data", + provider: "market-data", + envVars: ["MARKET_DATA_TOKEN"], + authMethods: ["token"], + }, + ], +}; + +export async function canonicalFuturePath(target: string): Promise { + return join(await realpath(dirname(target)), basename(target)); +} + +export async function writeManifestFile( + tempDirs: { make(prefix: string): string }, + value: unknown = minimalManifest, +): Promise { + const dir = tempDirs.make("openclaw-claws-cli-"); + const path = join(dir, "openclaw.claw.json"); + await writeFile(path, JSON.stringify(value), "utf8"); + return path; +} diff --git a/src/cli/claws-cli.test.ts b/src/cli/claws-cli.test.ts index 64fceb76f4d0..393068d849d6 100644 --- a/src/cli/claws-cli.test.ts +++ b/src/cli/claws-cli.test.ts @@ -1,11 +1,11 @@ -// Tests for the experimental grouped Claws CLI. -import { mkdir, realpath, writeFile } from "node:fs/promises"; -import { basename, dirname, join } from "node:path"; +import { mkdir, writeFile } from "node:fs/promises"; +import { join } from "node:path"; import { Command } from "commander"; import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import { useAutoCleanupTempDirTracker } from "../../test/helpers/temp-dir.js"; import { persistClawInstallRecord } from "../claws/provenance.js"; import { closeOpenClawStateDatabaseForTest } from "../state/openclaw-state-db.js"; +import * as cliTestHelpers from "./claws-cli.test-helpers.js"; const mocks = vi.hoisted(() => { const logs: string[] = []; @@ -110,13 +110,8 @@ const { runClawsAddCommand } = await import("./claws-cli.runtime.js"); const { ClawUpdateMutationError } = await import("../claws/update-apply.js"); const tempDirs = useAutoCleanupTempDirTracker(afterEach); -const minimalManifest = { schemaVersion: 1, agent: { id: "demo-agent", name: "Demo Agent" } }; - -async function writeManifest(value: unknown = minimalManifest): Promise { - const dir = tempDirs.make("openclaw-claws-cli-"); - const path = join(dir, "openclaw.claw.json"); - await writeFile(path, JSON.stringify(value), "utf8"); - return path; +async function writeManifest(value: unknown = cliTestHelpers.minimalManifest): Promise { + return await cliTestHelpers.writeManifestFile(tempDirs, value); } async function writePackage(): Promise<{ root: string; workspace: string }> { @@ -154,10 +149,6 @@ async function writePackage(): Promise<{ root: string; workspace: string }> { return { root, workspace: join(root, "target-workspace") }; } -async function canonicalFuturePath(target: string): Promise { - return join(await realpath(dirname(target)), basename(target)); -} - async function runCli(args: string[]) { const program = new Command(); program.exitOverride(); @@ -197,7 +188,15 @@ describe("claws cli", () => { mocks.stateTableGet.mockReturnValue({ 1: 1 }); mocks.openExistingOpenClawStateDatabaseReadOnly.mockReset(); mocks.openExistingOpenClawStateDatabaseReadOnly.mockReturnValue({ - db: { prepare: () => ({ get: mocks.stateTableGet }) }, + db: { + prepare: (sql: string) => ({ + get: sql.includes("sqlite_master") ? mocks.stateTableGet : vi.fn(() => undefined), + all: vi.fn(() => [ + { name: "bootstrap_source_path" }, + { name: "bootstrap_content_digest" }, + ]), + }), + }, path: "state.sqlite", walMaintenance: { checkpoint: () => false, close: mocks.closeReadOnlyDatabase }, }); @@ -291,6 +290,7 @@ describe("claws cli", () => { desired: { summary: "all", digest: "sha256:desired" }, }, ], + readiness: cliTestHelpers.pluginSetupReadiness, blockers: [], diagnostics: [], }); @@ -413,7 +413,7 @@ describe("claws cli", () => { it("takes identity from package.json and plans one new agent", async () => { const { root, workspace } = await writePackage(); - const expectedWorkspace = await canonicalFuturePath(workspace); + const expectedWorkspace = await cliTestHelpers.canonicalFuturePath(workspace); await runCli(["claws", "add", root, "--dry-run", "--workspace", workspace, "--json"]); @@ -522,7 +522,7 @@ describe("claws cli", () => { it("applies a minimal Claw only after explicit consent", async () => { const manifestPath = await writeManifest(); const workspace = join(tempDirs.make("openclaw-claws-add-"), "workspace"); - const expectedWorkspace = await canonicalFuturePath(workspace); + const expectedWorkspace = await cliTestHelpers.canonicalFuturePath(workspace); await runCli(["claws", "add", manifestPath, "--dry-run", "--workspace", workspace, "--json"]); const plan = JSON.parse(mocks.logs[0] ?? "{}"); mocks.logs.length = 0; @@ -825,7 +825,7 @@ describe("claws cli", () => { const output = mocks.logs.join("\n"); expect(output).toContain("Capability changes: 1; escalations requiring explicit review: 1"); - expect(output).toContain("Plan integrity: sha256:update-plan"); + expect(output).toContain("MARKET_DATA_TOKEN"); expect(output).toContain( "Capability consent: the exact plan-integrity token binds every ! change disclosed below.", ); @@ -858,6 +858,7 @@ describe("claws cli", () => { capabilityEscalations: 0, }, capabilityChanges: [], + readiness: { ready: true, requirements: [] }, actions: [ { kind: "workspaceFile", diff --git a/src/plugins/bundle-capability-support.ts b/src/plugins/bundle-capability-support.ts new file mode 100644 index 000000000000..bdd69d3aa85c --- /dev/null +++ b/src/plugins/bundle-capability-support.ts @@ -0,0 +1,22 @@ +import type { PluginBundleFormat } from "./manifest-types.js"; + +export function isBundleCapabilitySupported( + format: PluginBundleFormat, + capability: string, +): boolean { + if (capability === "skills" || capability === "mcpServers" || capability === "settings") { + return true; + } + if ( + (capability === "commands" || capability === "outputStyles" || capability === "lspServers") && + (format === "claude" || format === "cursor") + ) { + return true; + } + // Only the Claude reader merges agent directories into the runtime skill roots + // (`resolveClaudeSkillDirs`); Cursor detects `.cursor/agents` but never loads it. + if (capability === "agents") { + return format === "claude"; + } + return capability === "hooks" && (format === "codex" || format === "claude"); +} diff --git a/src/plugins/install-artifact-inspection.test.ts b/src/plugins/install-artifact-inspection.test.ts new file mode 100644 index 000000000000..dccfe4bb14fe --- /dev/null +++ b/src/plugins/install-artifact-inspection.test.ts @@ -0,0 +1,103 @@ +import { mkdir, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { afterEach, describe, expect, it } from "vitest"; +import { useAutoCleanupTempDirTracker } from "../../test/helpers/temp-dir.js"; +import { + inspectBundlePluginArtifact, + inspectNativePluginArtifact, +} from "./install-artifact-inspection.js"; +import { installPluginFromPath } from "./install-package.js"; + +const tempDirs = useAutoCleanupTempDirTracker(afterEach); + +describe("plugin install artifact inspection", () => { + it("classifies native plugins as canonically mapped", () => { + expect(inspectNativePluginArtifact()).toEqual({ + format: "openclaw", + mapped: ["plugin"], + unavailable: [], + }); + }); + + it("separates mapped and detect-only bundle capabilities deterministically", () => { + expect( + inspectBundlePluginArtifact({ + format: "claude", + capabilities: ["outputStyles", "skills", "agents", "mcpServers", "skills"], + }), + ).toEqual({ + format: "claude", + mapped: ["agents", "mcpServers", "outputStyles", "skills"], + unavailable: [], + }); + }); + + it("reports cursor agent directories as unavailable because the runtime never loads them", () => { + expect( + inspectBundlePluginArtifact({ + format: "cursor", + capabilities: ["agents", "commands", "skills"], + }), + ).toEqual({ + format: "cursor", + mapped: ["commands", "skills"], + unavailable: ["agents"], + }); + }); + + it("returns canonical inspection from the verified bundle install path", async () => { + const root = tempDirs.make("openclaw-plugin-artifact-inspection-"); + const bundle = join(root, "bundle"); + await mkdir(join(bundle, ".claude-plugin"), { recursive: true }); + await mkdir(join(bundle, "skills", "triage"), { recursive: true }); + await mkdir(join(bundle, "agents", "reviewer"), { recursive: true }); + await writeFile( + join(bundle, ".claude-plugin", "plugin.json"), + JSON.stringify({ name: "inspection-bundle", version: "1.0.0" }), + "utf8", + ); + + const result = await installPluginFromPath({ + path: bundle, + extensionsDir: join(root, "extensions"), + dryRun: true, + }); + + expect(result).toMatchObject({ + ok: true, + artifactInspection: { + format: "claude", + mapped: ["agents", "skills"], + unavailable: [], + }, + }); + }); + + it("preflights a cursor bundle with detected but unmapped agents", async () => { + const root = tempDirs.make("openclaw-plugin-artifact-inspection-cursor-"); + const bundle = join(root, "bundle"); + await mkdir(join(bundle, ".cursor-plugin"), { recursive: true }); + await mkdir(join(bundle, "skills", "triage"), { recursive: true }); + await mkdir(join(bundle, ".cursor", "agents"), { recursive: true }); + await writeFile( + join(bundle, ".cursor-plugin", "plugin.json"), + JSON.stringify({ name: "cursor-inspection-bundle", version: "1.0.0" }), + "utf8", + ); + + const result = await installPluginFromPath({ + path: bundle, + extensionsDir: join(root, "extensions"), + dryRun: true, + }); + + expect(result).toMatchObject({ + ok: true, + artifactInspection: { + format: "cursor", + mapped: ["skills"], + unavailable: ["agents"], + }, + }); + }); +}); diff --git a/src/plugins/install-artifact-inspection.ts b/src/plugins/install-artifact-inspection.ts new file mode 100644 index 000000000000..3a2fabfc966d --- /dev/null +++ b/src/plugins/install-artifact-inspection.ts @@ -0,0 +1,32 @@ +import { isBundleCapabilitySupported } from "./bundle-capability-support.js"; +import type { PluginBundleFormat } from "./manifest-types.js"; + +type PluginArtifactFormat = "openclaw" | PluginBundleFormat; + +export const PLUGIN_ARTIFACT_ADAPTER_IDENTITY = "openclaw/v1" as const; + +export type PluginInstallArtifactInspection = { + format: PluginArtifactFormat; + mapped: string[]; + unavailable: string[]; +}; + +export function inspectNativePluginArtifact(): PluginInstallArtifactInspection { + return { format: "openclaw", mapped: ["plugin"], unavailable: [] }; +} + +export function inspectBundlePluginArtifact(params: { + format: PluginBundleFormat; + capabilities: Iterable; +}): PluginInstallArtifactInspection { + const capabilities = [...new Set(params.capabilities)].toSorted(); + return { + format: params.format, + mapped: capabilities.filter((capability) => + isBundleCapabilitySupported(params.format, capability), + ), + unavailable: capabilities.filter( + (capability) => !isBundleCapabilitySupported(params.format, capability), + ), + }; +} diff --git a/src/plugins/install-package.ts b/src/plugins/install-package.ts index 78e3b6b8aab6..58122f05596c 100644 --- a/src/plugins/install-package.ts +++ b/src/plugins/install-package.ts @@ -1,5 +1,10 @@ import fs from "node:fs/promises"; import { resolveUserPath } from "../utils.js"; +import { + inspectBundlePluginArtifact, + inspectNativePluginArtifact, + type PluginInstallArtifactInspection, +} from "./install-artifact-inspection.js"; import { scanAndLinkInstalledPackage, validatePackagePluginInstallSource, @@ -171,7 +176,7 @@ async function installBundleFromSourceDir( return scanResult; } - return await installPluginDirectoryIntoExtensions({ + const installed = await installPluginDirectoryIntoExtensions({ sourceDir: params.sourceDir, pluginId, manifestName: manifestRes.manifest.name, @@ -187,6 +192,22 @@ async function installBundleFromSourceDir( hasDeps: false, depsLogMessage: "", }); + return installed.ok + ? { + ...installed, + artifactInspection: inspectBundlePluginArtifact({ + format: manifestRes.manifest.bundleFormat, + capabilities: manifestRes.manifest.capabilities, + }), + } + : installed; +} + +function withArtifactInspection( + result: InstallPluginResult, + artifactInspection: PluginInstallArtifactInspection, +): InstallPluginResult { + return result.ok ? { ...result, artifactInspection } : result; } async function installPluginFromSourceDir( @@ -196,11 +217,14 @@ async function installPluginFromSourceDir( ): Promise { const nativePackageManifest = await detectNativePackageInstallSource(params.sourceDir); if (nativePackageManifest) { - return await installPluginFromPackageDir({ - packageDir: params.sourceDir, - packageManifest: nativePackageManifest, - ...pickPackageInstallCommonParams(params), - }); + return withArtifactInspection( + await installPluginFromPackageDir({ + packageDir: params.sourceDir, + packageManifest: nativePackageManifest, + ...pickPackageInstallCommonParams(params), + }), + inspectNativePluginArtifact(), + ); } const bundleResult = await installBundleFromSourceDir({ sourceDir: params.sourceDir, @@ -209,10 +233,13 @@ async function installPluginFromSourceDir( if (bundleResult) { return bundleResult; } - return await installPluginFromPackageDir({ - packageDir: params.sourceDir, - ...pickPackageInstallCommonParams(params), - }); + return withArtifactInspection( + await installPluginFromPackageDir({ + packageDir: params.sourceDir, + ...pickPackageInstallCommonParams(params), + }), + inspectNativePluginArtifact(), + ); } async function detectNativePackageInstallSource( diff --git a/src/plugins/install-types.ts b/src/plugins/install-types.ts index 6df43b8f33dd..70dc8081c44b 100644 --- a/src/plugins/install-types.ts +++ b/src/plugins/install-types.ts @@ -1,5 +1,6 @@ import type { NpmIntegrityDrift, NpmSpecResolution } from "../infra/install-source-utils.js"; import type { InstallPolicySource } from "../security/install-policy.js"; +import type { PluginInstallArtifactInspection } from "./install-artifact-inspection.js"; import type { InstallSafetyOverrides } from "./install-security-scan.js"; import type { PackageManifest as PluginPackageManifest, PluginManifestSetup } from "./manifest.js"; @@ -45,6 +46,7 @@ export type InstallPluginResult = version?: string; extensions: string[]; setup?: PluginManifestSetup; + artifactInspection?: PluginInstallArtifactInspection; npmResolution?: NpmSpecResolution; integrityDrift?: NpmIntegrityDrift; } diff --git a/src/plugins/loader-runtime-candidate.ts b/src/plugins/loader-runtime-candidate.ts index d8e2848d439b..fcf6b1a9f6e9 100644 --- a/src/plugins/loader-runtime-candidate.ts +++ b/src/plugins/loader-runtime-candidate.ts @@ -1,5 +1,6 @@ import fs from "node:fs"; import { describeRootFileOpenFailure, openRootFileSync } from "../infra/boundary-file-read.js"; +import { isBundleCapabilitySupported } from "./bundle-capability-support.js"; import { inspectBundleMcpRuntimeSupport } from "./bundle-mcp.js"; import { resolveEffectiveEnableState, @@ -577,20 +578,8 @@ function recordBundleDiagnostics(params: { }): void { const unsupportedCapabilities = (params.record.bundleCapabilities ?? []).filter( (capability) => - capability !== "skills" && - capability !== "mcpServers" && - capability !== "settings" && - !( - (capability === "commands" || - capability === "agents" || - capability === "outputStyles" || - capability === "lspServers") && - (params.record.bundleFormat === "claude" || params.record.bundleFormat === "cursor") - ) && - !( - capability === "hooks" && - (params.record.bundleFormat === "codex" || params.record.bundleFormat === "claude") - ), + !params.record.bundleFormat || + !isBundleCapabilitySupported(params.record.bundleFormat, capability), ); for (const capability of unsupportedCapabilities) { params.registry.diagnostics.push({ diff --git a/src/state/openclaw-state-db-maintenance.ts b/src/state/openclaw-state-db-maintenance.ts index 60f8ae176fdb..89f7a6f10271 100644 --- a/src/state/openclaw-state-db-maintenance.ts +++ b/src/state/openclaw-state-db-maintenance.ts @@ -18,8 +18,25 @@ import { import { resolveOpenClawStateSqlitePath } from "./openclaw-state-db.paths.js"; import { OPENCLAW_STATE_SCHEMA_SQL } from "./openclaw-state-schema.js"; +/** + * Additive Claw provenance columns that only a writable open can ensure. A + * same-version database written before them stays readable so read-only + * planning surfaces are not refused before they can report anything. + */ +export const CLAW_LAZY_ADDITIVE_STATE_COLUMNS = [ + "claw_installs.bootstrap_content_digest", + "claw_installs.bootstrap_source_path", + "claw_package_refs.extension_adapter_identity", + "claw_package_refs.extension_detected_format", + "claw_package_refs.extension_format", + "claw_package_refs.extension_id", + "claw_package_refs.extension_mapped_json", + "claw_package_refs.extension_unavailable_json", +] as const; + const OPENCLAW_STATE_MAINTENANCE_SCHEMA_COMPATIBILITY = { allowedMissingTables: LAZY_ADDITIVE_STATE_TABLES, + allowedMissingColumns: CLAW_LAZY_ADDITIVE_STATE_COLUMNS, allowedColumnDefinitions: { "diagnostic_events.sequence": ["sequence INTEGER NOT NULL DEFAULT 0"], "commitments.attempts": ["attempts INTEGER NOT NULL DEFAULT 0"], diff --git a/src/state/openclaw-state-db-schema-additive.ts b/src/state/openclaw-state-db-schema-additive.ts index 55b219c47d80..b8dc4294f36b 100644 --- a/src/state/openclaw-state-db-schema-additive.ts +++ b/src/state/openclaw-state-db-schema-additive.ts @@ -104,6 +104,12 @@ export function ensureAdditiveStateColumns(db: DatabaseSync): void { "claw_package_refs", "package_integrity TEXT NOT NULL DEFAULT 'sha256:0000000000000000000000000000000000000000000000000000000000000000'", ); + ensureColumn(db, "claw_package_refs", "extension_id TEXT"); + ensureColumn(db, "claw_package_refs", "extension_format TEXT"); + ensureColumn(db, "claw_package_refs", "extension_detected_format TEXT"); + ensureColumn(db, "claw_package_refs", "extension_mapped_json TEXT"); + ensureColumn(db, "claw_package_refs", "extension_unavailable_json TEXT"); + ensureColumn(db, "claw_package_refs", "extension_adapter_identity TEXT"); const addedDiagnosticEventSequence = ensureColumn( db, "diagnostic_events", diff --git a/src/state/openclaw-state-db.generated.d.ts b/src/state/openclaw-state-db.generated.d.ts index 1bf88461aa49..1331a6866d3c 100644 --- a/src/state/openclaw-state-db.generated.d.ts +++ b/src/state/openclaw-state-db.generated.d.ts @@ -309,6 +309,12 @@ export interface ClawMcpServerRefs { export interface ClawPackageRefs { agent_id: string; claw_name: string; + extension_adapter_identity: string | null; + extension_detected_format: string | null; + extension_format: string | null; + extension_id: string | null; + extension_mapped_json: string | null; + extension_unavailable_json: string | null; independent_owner: number; installed_at_ms: number; origin: string; diff --git a/src/state/openclaw-state-db.test.ts b/src/state/openclaw-state-db.test.ts index 2b0fa2fa9830..44377c5ee121 100644 --- a/src/state/openclaw-state-db.test.ts +++ b/src/state/openclaw-state-db.test.ts @@ -3032,6 +3032,41 @@ INSERT INTO macos_port_guardian_records VALUES (4242, 18789, '/usr/bin/ssh', 're ).toEqual({ installed_at_ms: 1234, updated_at_ms: 1234 }); }); + it("adds optional Claw application provenance columns to existing state databases", () => { + const stateDir = createTempStateDir(); + const database = openOpenClawStateDatabase({ env: { OPENCLAW_STATE_DIR: stateDir } }); + const databasePath = database.path; + closeOpenClawStateDatabaseForTest(); + + const { DatabaseSync } = requireNodeSqlite(); + const legacyDb = new DatabaseSync(databasePath); + legacyDb.exec(` + ALTER TABLE claw_package_refs DROP COLUMN extension_id; + ALTER TABLE claw_package_refs DROP COLUMN extension_format; + ALTER TABLE claw_package_refs DROP COLUMN extension_detected_format; + ALTER TABLE claw_package_refs DROP COLUMN extension_mapped_json; + ALTER TABLE claw_package_refs DROP COLUMN extension_unavailable_json; + ALTER TABLE claw_package_refs DROP COLUMN extension_adapter_identity; + `); + markStateDatabaseAsV5(legacyDb); + legacyDb.close(); + + const reopened = openOpenClawStateDatabase({ env: { OPENCLAW_STATE_DIR: stateDir } }); + const packageColumns = reopened.db + .prepare("PRAGMA table_info(claw_package_refs)") + .all() as Array<{ name?: string }>; + expect(packageColumns.map((column) => column.name)).toEqual( + expect.arrayContaining([ + "extension_id", + "extension_format", + "extension_detected_format", + "extension_mapped_json", + "extension_unavailable_json", + "extension_adapter_identity", + ]), + ); + }); + it("adds worker bootstrap lifecycle columns to existing state databases", () => { const stateDir = createTempStateDir(); const databasePath = materializeCurrentStateDatabase(stateDir); diff --git a/src/state/openclaw-state-db.ts b/src/state/openclaw-state-db.ts index 26847bfbf3d7..58f53d3980b9 100644 --- a/src/state/openclaw-state-db.ts +++ b/src/state/openclaw-state-db.ts @@ -60,6 +60,7 @@ import { assertOpenClawStateDatabaseForMaintenance, assertOpenClawStateDatabaseV5ForMigration, assertSupportedSchemaVersion, + CLAW_LAZY_ADDITIVE_STATE_COLUMNS, createOpenClawDatabaseVerificationError, resolveDatabasePath, } from "./openclaw-state-db-maintenance.js"; @@ -84,10 +85,7 @@ export { OPENCLAW_SQLITE_BUSY_TIMEOUT_MS, OPENCLAW_STATE_SCHEMA_VERSION, }; -export const STATE_READ_ONLY_COMPATIBLE_MISSING_COLUMNS = [ - "claw_installs.bootstrap_source_path", - "claw_installs.bootstrap_content_digest", -] as const; +export const STATE_READ_ONLY_COMPATIBLE_MISSING_COLUMNS = CLAW_LAZY_ADDITIVE_STATE_COLUMNS; export type { OpenClawStateDatabase, OpenClawStateDatabaseOptions, diff --git a/src/state/openclaw-state-schema.sql b/src/state/openclaw-state-schema.sql index 96716352bde7..06721bf97590 100644 --- a/src/state/openclaw-state-schema.sql +++ b/src/state/openclaw-state-schema.sql @@ -2153,6 +2153,12 @@ CREATE TABLE IF NOT EXISTS claw_package_refs ( relationship TEXT NOT NULL CHECK (relationship IN ('managed', 'referenced')), origin TEXT NOT NULL CHECK (origin IN ('claw-introduced', 'pre-existing')), independent_owner INTEGER NOT NULL CHECK (independent_owner IN (0, 1)), + extension_id TEXT, + extension_format TEXT, + extension_detected_format TEXT, + extension_mapped_json TEXT, + extension_unavailable_json TEXT, + extension_adapter_identity TEXT, installed_at_ms INTEGER NOT NULL, updated_at_ms INTEGER NOT NULL, PRIMARY KEY (agent_id, package_kind, package_source, package_ref, package_version)