Files
openclaw/test/e2e/qa-lab/runtime/managed-worktrees-cli-product-proof.e2e.test.ts
Peter Steinberger 73bdb4b924 feat(agents): record run-end worktree cleanup outcome; prove Workboard dirty retention (#120434)
* feat(agents): record run-end worktree cleanup outcome

Persist removed, retained, and failed run-end cleanup outcomes on managed worktree records. Operators and QA can inspect the durable fact through worktrees.list and openclaw worktrees list --json.

Release note: Managed worktree run-end cleanup now records why a checkout was removed or retained in worktree list JSON.

* test(qa): prove dirty worktree retention outcome

* chore(protocol): regenerate swift gateway models

* fix(agents): harden worktree cleanup recovery

Register run_end_cleanup_json as a lazy compatible column so same-version v6 index repair and read-only doctor migration can recover databases created before the column existed.

Type removal contention at the registry boundary; unexpected claim failures now best-effort record a bounded failed outcome and rethrow the original error.

* fix(ci): clear repo-wide lint debt blocking merge gates

The red-main landing rule requires this PR to repair repository-wide merge-gate debt instead of bypassing it. Apply the current lint contracts mechanically and split turn-transition coverage into a concept-named sibling with per-file-safe test state.

Exact line delta: +676/-574 (net +102) across 44 test/support files.

* fix(ci): preserve cached health refresh proof

Require the public refresh call to exist before accepting that sensitive fields were omitted, so the boundary proof cannot pass on a missing call.

* fix(ci): correct test typing left by the lint sweep

Literal-widened totalTokensVersion fixtures, a WebSocket RawData overload
mismatch, and the protocol schema document cast broke check-test-types
after the repo-wide lint repair. Aligns the fixtures with SessionEntry,
narrows Buffer handling per RawData, and keeps the JSON-shaped undefined
omission under structuredClone.

* test(agents): reuse upstream resource-loader test support

The session-loop split and #120463's helper extraction landed the same
createResourceLoader/createCompactionHandlers twice; the rebase kept both,
orphaning main's agent-session-loop-resource-loader.test-support.ts and
failing the dead-code gate. Import the upstream helpers and delete the
duplicates.

* fix(agents): reject finalized rows at the worktree removal claim

Address the accepted ClawSweeper late-claim finding by rereading and rejecting missing or finalized worktree rows inside the synchronous removal-claim transaction.

Preserve the authoritative cleanup invariant: finalized contenders record nothing, while retained-busy is written only while the row remains live.

* refactor(agents): reuse registry update for busy outcomes

Keep the live-row conditional write in the canonical registry update path so the finalized-claim repair stays below the registry max-lines ratchet without weakening the authoritative-outcome invariant.

* test(agents): drop session test duplicates after rebase

Keep current main as the canonical owner of next-turn lifecycle coverage and correctness test support after replaying the older lint-debt split.

* fix(agents): guard post-abort cleanup outcomes against finalization

After abortWorktreeRemoval releases a stale remover's claim, its retained or
failed write raced a finalizing remover and could overwrite the authoritative
removed-lossless fact. Route every retained/failed write through the live-row
condition; only the finalizing remover's own removed-lossless write stays
unconditional.

* fix(agents): persist the removal outcome atomically with finalization

A delayed removed-lossless write after remove() finalized could race a
restore plus newer cleanup and overwrite the newer operator-visible fact.
The run-end outcome now rides remove()'s finalization update; every other
cleanup write stays live-row conditional, so no post-finalize write path
remains.

* test(qa): restore strict cached-health contract assertions

The lint sweep's Boolean() coercions let truthy non-booleans satisfy the
wire-typed cached-meta contract. Assert the literal boolean for unknown-typed
fields and use nullish-coalesced strict equivalents for boolean chains.

* fix(agents): clear the stale cleanup outcome when restoring a worktree

A restored checkout begins a new lifecycle; leaving the removed-lossless
fact on the live row showed operators a stale result until the next
cleanup. Restore clears the recorded outcome and the regression asserts
the cleared state before the next cleanup records fresh truth.

* fix(agents): scope stale cleanup outcomes to their observed lifecycle

A stale remover's retained/failed write raced a concurrent remove-plus-
restore: the revived row is live again, so the live-row condition alone
could stamp a prior-lifecycle outcome. Condition those writes on the
activity stamp the remover observed; restore bumps lastActiveAt, making
any prior-lifecycle write a no-op.

* fix(agents): advance the restore activity stamp within one millisecond

Stale cleanup writes fence on the activity stamp they observed; a restore
completing in the same millisecond could revive the row with an identical
stamp and let the fence match. Restore now always advances past the
stored value, and the ABA regression pins the clock to prove the
same-millisecond case.
2026-08-08 20:32:11 -07:00

204 lines
8.6 KiB
TypeScript

// QA Lab product proof for the managed-worktree child CLI lifecycle.
import { execFile } from "node:child_process";
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { promisify } from "node:util";
import { afterEach, describe, expect, it } from "vitest";
import type {
ManagedWorktreeGcResult,
ManagedWorktreeRecord,
RemoveManagedWorktreeResult,
} from "../../../../src/agents/worktrees/types.js";
import { closeOpenClawStateDatabaseForTest } from "../../../../src/state/openclaw-state-db.js";
import {
createOpenClawTestInstance,
type OpenClawTestInstance,
} from "../../../helpers/openclaw-test-instance.js";
const execFileAsync = promisify(execFile);
const WORKTREE_NAME = "qa-managed-worktree";
type CommandResult = Awaited<ReturnType<OpenClawTestInstance["cli"]>>;
type WorktreeListJson = { worktrees: ManagedWorktreeRecord[] };
let instance: OpenClawTestInstance | undefined;
let tempRoot: string | undefined;
afterEach(async () => {
closeOpenClawStateDatabaseForTest();
await instance?.cleanup();
instance = undefined;
if (tempRoot) {
await fs.rm(tempRoot, { recursive: true, force: true });
tempRoot = undefined;
}
});
function parseCommandJson<T>(
label: string,
result: CommandResult,
parse: (value: unknown) => T = (value) => value as T,
): T {
if (result.code !== 0) {
throw new Error(
`${label} failed with exit ${String(result.code)}\nstdout:\n${result.stdout}\nstderr:\n${result.stderr}`,
);
}
return parse(JSON.parse(result.stdout) as unknown);
}
async function git(cwd: string, ...args: string[]): Promise<string> {
const { stdout } = await execFileAsync("git", ["-C", cwd, ...args], { encoding: "utf8" });
return stdout.trimEnd();
}
async function initializeRepository(root: string): Promise<{ baseCommit: string; repo: string }> {
const repo = path.join(root, "source");
await fs.mkdir(path.join(repo, ".openclaw"), { recursive: true });
await fs.mkdir(path.join(repo, "generated"), { recursive: true });
await git(repo, "init", "-b", "main");
await git(repo, "config", "user.name", "OpenClaw Test");
await git(repo, "config", "user.email", "openclaw-test@example.invalid");
await fs.writeFile(path.join(repo, "README.md"), "base\n");
await fs.writeFile(path.join(repo, ".gitignore"), ".env.local\ngenerated/\n");
await fs.writeFile(path.join(repo, ".worktreeinclude"), ".env.local\ngenerated/**\n");
const setupScript = path.join(repo, ".openclaw", "worktree-setup.sh");
await fs.writeFile(
setupScript,
'#!/bin/sh\nset -eu\nprintf "%s\\n%s\\n" "$OPENCLAW_SOURCE_TREE_PATH" "$OPENCLAW_WORKTREE_PATH" > "$OPENCLAW_WORKTREE_PATH/setup-marker.txt"\n',
);
await fs.chmod(setupScript, 0o755);
await git(repo, "add", "README.md", ".gitignore", ".worktreeinclude", setupScript);
await git(repo, "commit", "-m", "initialize managed worktree fixture");
await fs.writeFile(path.join(repo, ".env.local"), "TOKEN=fixture\n");
await fs.chmod(path.join(repo, ".env.local"), 0o600);
await fs.writeFile(path.join(repo, "generated", "tool.sh"), "#!/bin/sh\necho generated\n");
await fs.chmod(path.join(repo, "generated", "tool.sh"), 0o755);
return {
baseCommit: await git(repo, "rev-parse", "HEAD"),
repo: await fs.realpath(repo),
};
}
describe("managed worktrees child CLI product proof", () => {
it(
"provisions, snapshots, restores, and preserves a manual worktree through gc",
{ timeout: 180_000 },
async () => {
const canonicalTmp = await fs.realpath(os.tmpdir());
tempRoot = await fs.mkdtemp(path.join(canonicalTmp, "openclaw-managed-worktree-cli-"));
const { baseCommit, repo } = await initializeRepository(tempRoot);
instance = await createOpenClawTestInstance({ name: "qa-managed-worktree-cli" });
const stateDir = await fs.realpath(instance.stateDir);
const created = parseCommandJson<ManagedWorktreeRecord>(
"worktrees create",
await instance.cli(["worktrees", "create", repo, "--name", WORKTREE_NAME, "--json"]),
);
expect(created).toMatchObject({
name: WORKTREE_NAME,
repoRoot: repo,
branch: `openclaw/${WORKTREE_NAME}`,
baseRef: "HEAD",
ownerKind: "manual",
});
expect(created.id).toEqual(expect.any(String));
expect(created.repoFingerprint).toEqual(expect.any(String));
expect(created.createdAt).toEqual(expect.any(Number));
expect(created.lastActiveAt).toEqual(expect.any(Number));
expect(await fs.realpath(created.path)).toBe(
path.join(stateDir, "worktrees", created.repoFingerprint, WORKTREE_NAME),
);
expect(await git(repo, "rev-parse", `refs/heads/${created.branch}`)).toBe(baseCommit);
const provisionedEnv = path.join(created.path, ".env.local");
const provisionedTool = path.join(created.path, "generated", "tool.sh");
await expect(fs.readFile(provisionedEnv, "utf8")).resolves.toBe("TOKEN=fixture\n");
await expect(fs.readFile(provisionedTool, "utf8")).resolves.toContain("echo generated");
expect((await fs.stat(provisionedEnv)).mode & 0o777).toBe(0o600);
expect((await fs.stat(provisionedTool)).mode & 0o777).toBe(0o755);
const setupPaths = (await fs.readFile(path.join(created.path, "setup-marker.txt"), "utf8"))
.trim()
.split("\n");
expect(setupPaths).toHaveLength(2);
expect(await fs.realpath(setupPaths[0]!)).toBe(repo);
expect(await fs.realpath(setupPaths[1]!)).toBe(await fs.realpath(created.path));
await fs.writeFile(path.join(created.path, "README.md"), "dirty tracked change\n");
await fs.writeFile(path.join(created.path, "notes.txt"), "restored note\n");
const removed = parseCommandJson<RemoveManagedWorktreeResult>(
"worktrees remove",
await instance.cli(["worktrees", "remove", created.id, "--json"]),
);
const expectedSnapshotRef = `refs/openclaw/snapshots/${created.id}`;
expect(removed).toEqual({ removed: true, snapshotRef: expectedSnapshotRef });
const snapshotCommit = await git(repo, "rev-parse", expectedSnapshotRef);
expect(await git(repo, "show-ref", "--verify", expectedSnapshotRef)).toContain(
expectedSnapshotRef,
);
await expect(fs.access(created.path)).rejects.toMatchObject({ code: "ENOENT" });
const removedList = parseCommandJson<WorktreeListJson>(
"worktrees list after remove",
await instance.cli(["worktrees", "list", "--json"]),
);
expect(removedList.worktrees).toContainEqual(
expect.objectContaining({ id: created.id, removedAt: expect.any(Number) }),
);
const restored = parseCommandJson<ManagedWorktreeRecord>(
"worktrees restore",
await instance.cli(["worktrees", "restore", created.id, "--json"]),
);
expect(restored).toMatchObject({
id: created.id,
branch: created.branch,
path: created.path,
});
expect(restored.removedAt).toBeUndefined();
const status = await git(restored.path, "status", "--porcelain");
expect(status.split("\n")).toEqual(expect.arrayContaining([" M README.md", "?? notes.txt"]));
await expect(fs.readFile(path.join(restored.path, "README.md"), "utf8")).resolves.toBe(
"dirty tracked change\n",
);
await expect(fs.readFile(path.join(restored.path, "notes.txt"), "utf8")).resolves.toBe(
"restored note\n",
);
await expect(fs.readFile(provisionedEnv, "utf8")).resolves.toBe("TOKEN=fixture\n");
await expect(fs.readFile(provisionedTool, "utf8")).resolves.toContain("echo generated");
expect((await fs.stat(provisionedEnv)).mode & 0o777).toBe(0o600);
expect((await fs.stat(provisionedTool)).mode & 0o777).toBe(0o755);
expect(await git(repo, "rev-parse", `refs/heads/${created.branch}`)).toBe(baseCommit);
expect((await git(repo, "log", "--format=%H", created.branch)).split("\n")).not.toContain(
snapshotCommit,
);
const gc = parseCommandJson<ManagedWorktreeGcResult>(
"worktrees gc",
await instance.cli(["worktrees", "gc", "--json"]),
);
expect(gc).toEqual({
removed: [],
orphansDeleted: expect.any(Number),
snapshotsPruned: expect.any(Number),
});
const activeList = parseCommandJson<WorktreeListJson>(
"worktrees list after gc",
await instance.cli(["worktrees", "list", "--json"]),
);
expect(activeList.worktrees).toContainEqual(
expect.objectContaining({ id: created.id, ownerKind: "manual" }),
);
expect(
activeList.worktrees.find((record) => record.id === created.id)?.removedAt,
).toBeUndefined();
await expect(fs.access(created.path)).resolves.toBeUndefined();
},
);
});