mirror of
https://github.com/openclaw/openclaw.git
synced 2026-08-25 03:45:46 -06:00
0e8faacd71
* fix(scripts): size the tsdown heap from the build's own cgroup budget The build heap probe only read the cgroup root (/sys/fs/cgroup/memory.max and the v1 equivalent). Those files exist only when the process runs in a namespaced container cgroup; under systemd the budget lives on the process's own slice, and the v2 root carries no limit at all. So every systemd-managed build found no limit, fell back to /proc/meminfo MemTotal, and took the full 12288 MB default heap regardless of its actual budget. Observed on a 15.4 GiB host: openclaw-main-update.service ran tsdown with NODE_OPTIONS=--max-old-space-size=12288 while its user@999.service slice was bounded at 5 GiB, reaching 3.2 GB RSS and 6.25 GB peak before the host began OOM-killing unrelated services. Resolve the limit from /proc/self/cgroup and walk that chain instead, reading memory.high alongside memory.max (memory.high throttles reclaim rather than failing allocation, so a heap above it stalls the build instead of OOM-ing), and take the tightest bound found. Root paths stay as the container fallback, and an explicitly injected path list still disables detection. Co-authored-by: jesse-merhi <79823012+jesse-merhi@users.noreply.github.com> * fix(scripts): resolve the build heap budget from the v1 memory controller too The slice walk only accepted the unified 0:: record, so a legacy or hybrid systemd host fell back to the root probe and kept taking host memory. One resolver now walks both hierarchies leaf-to-root, which makes the static root list its own depth-0 case and removes it. Co-authored-by: jesse-merhi <79823012+jesse-merhi@users.noreply.github.com> * fix(scripts): read cgroup controller mounts instead of assuming their paths v1 controllers can be co-mounted at the cgroup root, where memory.limit_in_bytes sits under the slice with no per-controller directory, so the hardcoded /sys/fs/cgroup/memory probe missed the budget and the build took the full 12288MB default. Mount points now come from mountinfo. Co-authored-by: jesse-merhi <79823012+jesse-merhi@users.noreply.github.com> * fix(scripts): translate cgroup records through the mount root mountinfo field 4 is the subtree a cgroupfs mount exposes. Under a container mount the /proc/self/cgroup record stays host-absolute, so walking it verbatim probed paths below the visible mount and the build fell back to host memory. Records now translate through the mount root before the walk. Co-authored-by: jesse-merhi <79823012+jesse-merhi@users.noreply.github.com> * fix(scripts): skip cgroup mounts that cannot represent this process Falling back to the mount root for a record outside the mount's subtree sized the build from an unrelated cgroup: an inherited namespace clamped the heap to the 2048MB floor from a foreign 1GiB limit. Non-representable mounts are now skipped, and the blind root probe only runs when no memory record exists. Co-authored-by: jesse-merhi <79823012+jesse-merhi@users.noreply.github.com> * fix(scripts): keep every cgroup mount view, not just the last one seen One hierarchy can be visible through several mounts and only some expose a subtree containing this process. Retaining only the last view dropped the budget whenever a non-representable bind view came later, sending the build back to host MemTotal. Co-authored-by: jesse-merhi <79823012+jesse-merhi@users.noreply.github.com> * fix(scripts): decode octal-escaped mountinfo paths before matching cgroups ClawSweeper P2 on7e64ad61f7: the cgroup resolver compared mountinfo's mount root and mount point verbatim. The kernel escapes space, tab, newline, and backslash in those two fields, so any cgroup mounted under such a path never matched, the bounded slice was missed, and heap sizing silently fell back to host memory. Decode both fields before matching. The decoder lives in scripts/lib beside the other shared script helpers rather than inline, so the scripts program has one copy rather than a new ad hoc one. Regression test fails pre-fix: a v2 mount at "/sys/fs/cgroup\040dir" with a 5 GiB memory.high yields --max-old-space-size=12288 (host fallback) before the fix and 4352 after. Follow-up, deliberately not bundled here: src/infra/sqlite-wal.ts, src/commands/doctor-state-integrity.ts, and src/plugins/bundled-source-overlays.ts each carry their own private copy of this same decoder. Consolidating all four into @openclaw/normalization-core is the right end state, but it touches a shared package plus three core modules and belongs in its own reviewable change. Co-authored-by: jesse-merhi <79823012+jesse-merhi@users.noreply.github.com> * fix(scripts): resolve cgroup-namespace-relative records to their mount ClawSweeper P1 ond6fe49dd3f: inside a cgroup namespace /proc/self/cgroup reports the namespace root ("0::/") while mountinfo field 4 stays the host subtree the cgroupfs was mounted from ("/docker/<id>"). relativeCgroupPath then found no prefix match and returned null; because a memory record had already been seen, the root probe was skipped and the build fell back to host MemTotal. A constrained container therefore missed its own budget entirely. That namespace root is exactly what the mount exposes at its mount point, so it resolves to "/" rather than failing closed. Regression test fails pre-fix: a "0::/" record against a /docker/2f1a9c mount root with a 5 GiB memory.max yields --max-old-space-size=12288 before the fix and 4352 after. Co-authored-by: jesse-merhi <79823012+jesse-merhi@users.noreply.github.com> * fix(scripts): reject inherited cgroup mount views instead of guessing ClawSweeper P1 onb4d200c5d2: the previous commit resolved a namespace-relative record against any mount root, including the inherited views cgroup_namespaces(7) documents, whose field-4 root reads "/..". Which cgroup such a view exposes is not derivable from mountinfo, so probing it can size the build from an unrelated cgroup's limit. Reject non-canonical mount roots outright. An undecidable view now falls back to host sizing, which is current main's behavior, rather than silently adopting the wrong budget. Regression test covers the "/.." inherited mount: it must yield host MemTotal sizing, not the 5 GiB limit sitting behind that mount. Co-authored-by: jesse-merhi <79823012+jesse-merhi@users.noreply.github.com> * fix(scripts): fail closed on namespace-root records against non-root mounts ClawSweeper P1 on731d3bbc8e: a "0::/" record does not prove that a mount rooted at some other subtree exposes this process's cgroup. Resolving that pair could cap the build heap from an unrelated cgroup's limit. Return no mapping for it. An undecidable pair now falls back to host sizing, which is current main's behavior, so the failure mode is a missed optimisation rather than a wrong budget. The "/.." inherited-mount rejection stays; this covers the broader ambiguous mapping it did not. The namespace-relative test is repointed accordingly: an unrelated mounted subtree must yield host sizing, not that subtree's limit. Net production change: none (4 lines swapped). Co-authored-by: jesse-merhi <79823012+jesse-merhi@users.noreply.github.com> * fix(build): cap tsdown heap to the real budget and refuse hosts that cannot build The 2048MB floor was applied on top of a discovered cgroup limit, so a small container was handed a heap larger than it could honour. Measured in real cgroups, that does not OOM-kill, it thrashes: a 1500MiB container sat pinned at its ceiling for 10 minutes with oom_kill at 0, never finished the second of eleven invocations, and starved every other process on the host. Cap to the discovered budget, then refuse up front when that budget cannot hold the build. The threshold is the whole-build peak, not a single pass: a full eleven-invocation build peaks at 4730MiB, so a 5GiB slice completes while 4GiB and 2816MiB slices are both killed partway through the third invocation. The refusal runs before any output is cleaned, so a host that cannot rebuild does not also lose the build it has. * fix(build): harden tsdown heap admission * fix(build): guard the default tsdown plan * fix(build): preserve runtime-only Docker builds * fix(build): admit only declaration cache misses * fix(build): scope heap admission to real budgets * fix(build): guard direct unified declarations * fix(build): guard the canonical tsdown config * fix(build): satisfy cache planning lint * fix(gateway): release empty orphan leases * fix(build): cap cgroup budget by host memory * fix(build): serialize the canonical tsdown config * test(build): freeze host memory fixtures * fix(build): honor cgroup v1 soft limits * fix(build): respect cgroup v1 hierarchy mode * fix(build): admit unified runtime plans * fix(build): admit every unified runtime path * fix(build): collect repeated tsdown filters * fix(build): ignore cgroup v1 soft limits * fix(build): use explicit heap override as opt-in * refactor(build): simplify memory admission * fix(build): harden constrained build recovery * fix(ci): prebuild runtime before real CLI shards * fix(build): honor runtime-only runner environment * fix(ci): satisfy tooling shard lint
561 lines
23 KiB
TypeScript
561 lines
23 KiB
TypeScript
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
|
import { tmpdir } from "node:os";
|
|
import path from "node:path";
|
|
import { describe, expect, it } from "vitest";
|
|
import { listAvailableExtensionIds } from "../../scripts/lib/changed-extensions.mts";
|
|
import {
|
|
createChangedExtensionFallbackShards,
|
|
createChangedNodeTestShards,
|
|
hasBuildArtifactAffectingChange,
|
|
hasCoreExtensionImpact,
|
|
hasPromptSnapshotAffectingChange,
|
|
hasQaSmokeAffectingChange,
|
|
hasSqliteSessionLifecycleAffectingChange,
|
|
resolveChangedDockerSeedLanes,
|
|
} from "../../scripts/lib/ci-changed-node-test-plan.mts";
|
|
import {
|
|
listExtensionTestFilesForRoots,
|
|
resolveExtensionTestConfig,
|
|
} from "../../scripts/lib/extension-test-plan.mts";
|
|
import { hasImportGraphImpactOnTargets } from "../../scripts/test-projects.test-support.mts";
|
|
import { listGitTrackedFiles } from "../../src/test-utils/repo-files.js";
|
|
import { isGatewayServerTestFile } from "../vitest/vitest.gateway-server-paths.mjs";
|
|
|
|
const CODEX_TEST_PROCESS_FILE_LIMIT = 12;
|
|
|
|
function expectBoundedCodexFallback(
|
|
shards: ReturnType<typeof createChangedExtensionFallbackShards>,
|
|
) {
|
|
const targets = shards.flatMap((shard) => shard.includePatterns ?? []);
|
|
|
|
expect(shards.length).toBeGreaterThan(1);
|
|
expect(
|
|
shards.every(
|
|
(shard) =>
|
|
shard.configs[0] === "test/vitest/vitest.extension-codex.config.ts" &&
|
|
(shard.includePatterns?.length ?? 0) > 0 &&
|
|
(shard.includePatterns?.length ?? 0) <= CODEX_TEST_PROCESS_FILE_LIMIT,
|
|
),
|
|
).toBe(true);
|
|
expect(targets).toEqual(listExtensionTestFilesForRoots(["extensions/codex"]));
|
|
expect(new Set(targets).size).toBe(targets.length);
|
|
}
|
|
|
|
function expectAllExtensionConfigs(
|
|
shards: ReturnType<typeof createChangedExtensionFallbackShards>,
|
|
) {
|
|
const configs = new Set(shards.flatMap((shard) => shard.configs));
|
|
const expectedConfigs = new Set(
|
|
listAvailableExtensionIds().map((extensionId) =>
|
|
resolveExtensionTestConfig(`extensions/${extensionId}`),
|
|
),
|
|
);
|
|
|
|
expect(configs).toEqual(expectedConfigs);
|
|
expect(configs).toContain("test/vitest/vitest.extension-codex.config.ts");
|
|
}
|
|
|
|
const allDockerSeedLanes = ["mcp-channels", "cron-mcp-cleanup", "mcp-code-mode-gateway"];
|
|
it.each([
|
|
[["scripts/e2e/mcp-channels-seed.ts"], ["mcp-channels"]],
|
|
[["scripts/e2e/cron-mcp-cleanup-seed.ts"], ["cron-mcp-cleanup"]],
|
|
[["scripts/e2e/mcp-code-mode-gateway-seed.ts"], ["mcp-code-mode-gateway"]],
|
|
[["scripts/e2e/lib/mcp-code-mode-probe-server.ts"], ["mcp-code-mode-gateway"]],
|
|
[["scripts/e2e/docker-openai-seed.ts"], allDockerSeedLanes],
|
|
[
|
|
[
|
|
"scripts/e2e/mcp-code-mode-gateway-seed.ts",
|
|
"scripts/e2e/mcp-channels-seed.ts",
|
|
"scripts/e2e/lib/mcp-code-mode-probe-server.ts",
|
|
"scripts/e2e/cron-mcp-cleanup-seed.ts",
|
|
],
|
|
allDockerSeedLanes,
|
|
],
|
|
[[".github/workflows/ci.yml"], allDockerSeedLanes],
|
|
[["scripts/lib/ci-changed-node-test-plan.mts"], allDockerSeedLanes],
|
|
[["scripts\\e2e\\lib\\mcp-code-mode-probe-server.ts"], ["mcp-code-mode-gateway"]],
|
|
[["scripts/e2e/install-e2e.ts", "docs/ci.md"], []],
|
|
])("resolves Docker seed lanes for %j", (changedPaths, expected) => {
|
|
expect(resolveChangedDockerSeedLanes(changedPaths)).toEqual(expected);
|
|
});
|
|
|
|
describe("CI changed Node test plan", () => {
|
|
it("routes Control UI style changes through source-scanning policy tests", () => {
|
|
const shards = createChangedNodeTestShards(["ui/src/styles/chat/layout.css"]);
|
|
const targets = shards?.flatMap((shard) => shard.targets ?? []) ?? [];
|
|
|
|
expect(targets).toEqual([
|
|
"ui/src/styles/base-theme-tokens.node.test.ts",
|
|
"ui/src/styles/cursor-policy.node.test.ts",
|
|
]);
|
|
});
|
|
|
|
it("routes cron alert sanitization changes through alert policy suites", () => {
|
|
const shards = createChangedNodeTestShards(["src/cron/failure-notification-text.ts"]);
|
|
const targets = shards?.flatMap((shard) => shard.targets ?? []) ?? [];
|
|
|
|
expect(targets).toEqual([
|
|
"src/cron/service.stream-trigger.test.ts",
|
|
"src/cron/service.stream-validation.test.ts",
|
|
"src/cron/service/timer.timeout-watchdog.test.ts",
|
|
]);
|
|
});
|
|
|
|
it("routes a focused source change into one targeted job", () => {
|
|
expect(createChangedNodeTestShards(["src/agents/live-model-filter.ts"])).toEqual([
|
|
{
|
|
checkName: "checks-node-changed",
|
|
configs: [],
|
|
requiresDist: false,
|
|
runner: "blacksmith-8vcpu-ubuntu-2404",
|
|
shardName: "changed",
|
|
targets: [
|
|
"src/agents/live-model-filter.test.ts",
|
|
"src/agents/live-model-dynamic-candidates.test.ts",
|
|
"src/agents/model-compat.test.ts",
|
|
],
|
|
},
|
|
]);
|
|
});
|
|
|
|
it("keeps boundary coverage on test-only diffs without the build-artifacts lane", () => {
|
|
// Test-only diffs skip build-artifacts (which hosts the full boundary
|
|
// gate), so the plan carries its own nondist boundary shard instead.
|
|
expect(createChangedNodeTestShards(["test/extension-import-boundaries.test.ts"])).toEqual([
|
|
{
|
|
checkName: "checks-node-changed",
|
|
configs: [],
|
|
requiresDist: false,
|
|
runner: "blacksmith-8vcpu-ubuntu-2404",
|
|
shardName: "changed",
|
|
targets: ["test/extension-import-boundaries.test.ts"],
|
|
},
|
|
{
|
|
checkName: "checks-node-changed-boundary",
|
|
configs: ["test/vitest/vitest.boundary.config.ts"],
|
|
requiresDist: false,
|
|
runner: "blacksmith-8vcpu-ubuntu-2404",
|
|
shardName: "changed-boundary",
|
|
},
|
|
]);
|
|
});
|
|
|
|
it("classifies build-artifact and QA smoke impact by changed surface", () => {
|
|
expect(hasBuildArtifactAffectingChange(["src/agents/foo.test.ts", "test/helpers/x.ts"])).toBe(
|
|
false,
|
|
);
|
|
expect(
|
|
hasBuildArtifactAffectingChange([
|
|
"src/gateway/server.auth.control-ui.trusted-proxy.suite.ts",
|
|
]),
|
|
).toBe(false);
|
|
expect(hasBuildArtifactAffectingChange(["src/agents/foo.ts"])).toBe(true);
|
|
// Build-input classification: only sources and the build pipeline can
|
|
// change dist bytes; repo scripts, workflows, and qa scenarios cannot.
|
|
expect(hasBuildArtifactAffectingChange(["scripts/build-all.mts"])).toBe(true);
|
|
expect(hasBuildArtifactAffectingChange(["tsconfig.json"])).toBe(true);
|
|
expect(hasBuildArtifactAffectingChange(["scripts/run-vitest.mjs"])).toBe(false);
|
|
expect(hasBuildArtifactAffectingChange([".github/workflows/ci.yml"])).toBe(false);
|
|
expect(hasBuildArtifactAffectingChange(["qa/scenarios/index.yaml"])).toBe(false);
|
|
expect(hasBuildArtifactAffectingChange(["ui/src/app.ts"])).toBe(false);
|
|
expect(hasQaSmokeAffectingChange(["extensions/qa-lab/src/ci-smoke-plan.ts"])).toBe(true);
|
|
expect(hasQaSmokeAffectingChange(["qa/scenarios/index.yaml"])).toBe(true);
|
|
// Smoke drives matrix + telegram; other channel plugins are invisible to it.
|
|
expect(hasQaSmokeAffectingChange(["extensions/telegram/src/index.ts"])).toBe(true);
|
|
expect(hasQaSmokeAffectingChange(["extensions/discord/src/index.ts"])).toBe(false);
|
|
// Broad runtime changes ride the main-push smoke run instead of taxing
|
|
// every PR with the six-part matrix; only QA-owned surfaces select it.
|
|
expect(hasQaSmokeAffectingChange(["ui/src/app.ts"])).toBe(false);
|
|
expect(hasQaSmokeAffectingChange(["src/infra/retry.ts"])).toBe(false);
|
|
expect(hasQaSmokeAffectingChange(["packages/llm-core/src/index.ts"])).toBe(false);
|
|
expect(hasQaSmokeAffectingChange(["pnpm-lock.yaml"])).toBe(false);
|
|
expect(hasQaSmokeAffectingChange(["scripts/run-vitest.mjs"])).toBe(false);
|
|
expect(hasQaSmokeAffectingChange(["test/scripts/ci-node-test-plan.test.ts"])).toBe(false);
|
|
// The QA lane's own orchestration must not be able to skip the lane.
|
|
expect(hasQaSmokeAffectingChange([".github/workflows/ci.yml"])).toBe(true);
|
|
expect(hasQaSmokeAffectingChange([".github/actions/setup-node-env/action.yml"])).toBe(true);
|
|
expect(hasQaSmokeAffectingChange(["scripts/lib/ci-changed-node-test-plan.mts"])).toBe(true);
|
|
expect(hasQaSmokeAffectingChange([".github/workflows/labeler.yml"])).toBe(false);
|
|
});
|
|
|
|
it("classifies prompt-snapshot impact by surface and generator import graph", () => {
|
|
// Inside the generator's import graph -> regenerated output can change.
|
|
expect(hasPromptSnapshotAffectingChange(["src/auto-reply/reply/prompt-prelude.ts"])).toBe(true);
|
|
// The codex extension loads through a dynamic bundled-plugin module id the
|
|
// graph walk cannot see; it stays on the always-run surface.
|
|
expect(hasPromptSnapshotAffectingChange(["extensions/codex/src/index.ts"])).toBe(true);
|
|
expect(
|
|
hasPromptSnapshotAffectingChange([
|
|
"test/fixtures/agents/prompt-snapshots/codex-runtime-happy-path/README.md",
|
|
]),
|
|
).toBe(true);
|
|
expect(hasPromptSnapshotAffectingChange(["scripts/generate-prompt-snapshots.ts"])).toBe(true);
|
|
// Workspace packages feed the generator through package-specifier imports
|
|
// the relative graph walk cannot see.
|
|
expect(hasPromptSnapshotAffectingChange(["packages/llm-core/src/index.ts"])).toBe(true);
|
|
// The gate's own orchestration must not be able to skip the gated lane.
|
|
expect(hasPromptSnapshotAffectingChange([".github/workflows/ci.yml"])).toBe(true);
|
|
expect(hasPromptSnapshotAffectingChange(["scripts/lib/ci-changed-node-test-plan.mts"])).toBe(
|
|
true,
|
|
);
|
|
// Outside the surface and the generator graph -> the lane may skip.
|
|
expect(hasPromptSnapshotAffectingChange(["ui/src/app.ts"])).toBe(false);
|
|
expect(hasPromptSnapshotAffectingChange(["extensions/discord/src/index.ts"])).toBe(false);
|
|
expect(hasPromptSnapshotAffectingChange(["docs/index.md"])).toBe(false);
|
|
expect(hasPromptSnapshotAffectingChange(["test/scripts/ci-node-test-plan.test.ts"])).toBe(
|
|
false,
|
|
);
|
|
// Deleted source files cannot be graphed; fail safe to running the check.
|
|
expect(hasPromptSnapshotAffectingChange(["src/infra/definitely-deleted-module.ts"])).toBe(true);
|
|
});
|
|
|
|
it("classifies SQLite session lifecycle impact by owner and import graph", () => {
|
|
expect(
|
|
hasSqliteSessionLifecycleAffectingChange([
|
|
"src/agents/embedded-agent-runner/run/attempt-session-runtime-prepare.ts",
|
|
]),
|
|
).toBe(true);
|
|
expect(
|
|
hasSqliteSessionLifecycleAffectingChange(["src/gateway/server-methods/sessions.ts"]),
|
|
).toBe(true);
|
|
expect(
|
|
hasSqliteSessionLifecycleAffectingChange(["src/sessions/session-lifecycle-admission.ts"]),
|
|
).toBe(true);
|
|
expect(hasSqliteSessionLifecycleAffectingChange(["src/config/sessions.ts"])).toBe(true);
|
|
expect(
|
|
hasSqliteSessionLifecycleAffectingChange([
|
|
"test/scripts/sqlite-sessions-transcripts-flip-proof.built-cli.e2e.test.ts",
|
|
]),
|
|
).toBe(true);
|
|
expect(
|
|
hasSqliteSessionLifecycleAffectingChange([
|
|
"packages/media-understanding-common/src/provider-id.ts",
|
|
]),
|
|
).toBe(false);
|
|
expect(hasSqliteSessionLifecycleAffectingChange(["src/agents/model-auth.ts"])).toBe(false);
|
|
expect(hasSqliteSessionLifecycleAffectingChange(["extensions/discord/src/index.ts"])).toBe(
|
|
false,
|
|
);
|
|
expect(
|
|
hasSqliteSessionLifecycleAffectingChange([
|
|
"src/config/sessions/session-registry-maintenance.test.ts",
|
|
]),
|
|
).toBe(false);
|
|
expect(
|
|
hasSqliteSessionLifecycleAffectingChange(["src/infra/definitely-deleted-module.ts"]),
|
|
).toBe(false);
|
|
expect(
|
|
hasSqliteSessionLifecycleAffectingChange([
|
|
"src/agents/embedded-agent-runner/run/deleted-session-runtime.ts",
|
|
]),
|
|
).toBe(true);
|
|
});
|
|
|
|
it("fails safe to the full plan for broad changes", () => {
|
|
expect(createChangedNodeTestShards(["package.json"])).toBeNull();
|
|
});
|
|
|
|
it("keeps minimal-gateway boot coverage reachable from gateway startup changes", () => {
|
|
// A gateway startup stall must fail in the gateway lane; the boot smoke is
|
|
// selected purely through the import graph, so a rename or an import shape
|
|
// the graph walker cannot see would silently drop it from targeted plans
|
|
// and the stall would first surface on unrelated ui-e2e PRs again.
|
|
const bootSmoke = "src/gateway/server-startup-minimal-boot.test.ts";
|
|
expect(isGatewayServerTestFile(bootSmoke)).toBe(true);
|
|
expect(
|
|
hasImportGraphImpactOnTargets(
|
|
["src/gateway/server-startup-bootstrap.ts"],
|
|
[bootSmoke],
|
|
process.cwd(),
|
|
),
|
|
).toBe(true);
|
|
});
|
|
|
|
it("fails safe whenever a diff deletes source files", () => {
|
|
expect(createChangedNodeTestShards(["src/infra/format-time/deleted-helper.ts"])).toBeNull();
|
|
expect(
|
|
createChangedNodeTestShards([
|
|
"src/infra/format-time/deleted-helper.ts",
|
|
"src/agents/live-model-filter.ts",
|
|
]),
|
|
).toBeNull();
|
|
});
|
|
|
|
it("keeps targeting when a diff only deletes test files alongside live source", () => {
|
|
const shards = createChangedNodeTestShards([
|
|
"src/agents/deleted-obsolete.test.ts",
|
|
"src/agents/live-model-filter.ts",
|
|
]);
|
|
expect(shards).not.toBeNull();
|
|
const targets = shards?.flatMap((shard) => shard.targets ?? []) ?? [];
|
|
expect(targets).toContain("src/agents/live-model-filter.test.ts");
|
|
});
|
|
|
|
it("runs only the boundary shard when a diff deletes test files", () => {
|
|
const cwd = mkdtempSync(path.join(tmpdir(), "openclaw-ci-deleted-test-"));
|
|
try {
|
|
expect(createChangedNodeTestShards(["src/gone.test.ts"], { cwd })).toEqual([
|
|
{
|
|
checkName: "checks-node-changed-boundary",
|
|
configs: ["test/vitest/vitest.boundary.config.ts"],
|
|
requiresDist: false,
|
|
runner: "blacksmith-8vcpu-ubuntu-2404",
|
|
shardName: "changed-boundary",
|
|
},
|
|
]);
|
|
} finally {
|
|
rmSync(cwd, { force: true, recursive: true });
|
|
}
|
|
});
|
|
|
|
it("fails safe when an unresolved path is mixed with a precise source change", () => {
|
|
expect(
|
|
createChangedNodeTestShards(["src/agents/live-model-filter.ts", "tsconfig.json"]),
|
|
).toBeNull();
|
|
});
|
|
|
|
it("fails safe when public SDK changes affect extension imports", () => {
|
|
expect(createChangedNodeTestShards(["src/plugin-sdk/core.ts"])).toBeNull();
|
|
});
|
|
|
|
it("fails safe when a core change reaches package consumers through the public SDK", () => {
|
|
expect(createChangedNodeTestShards(["src/shared/text/strip-markdown.ts"])).toBeNull();
|
|
});
|
|
|
|
it("fails safe when a core change reaches a public SDK wrapper through an import", () => {
|
|
expect(createChangedNodeTestShards(["src/channels/chat-meta-shared.ts"])).toBeNull();
|
|
});
|
|
|
|
it("fails safe when workspace package consumers use package imports", () => {
|
|
expect(
|
|
createChangedNodeTestShards(["packages/gateway-protocol/src/frame-guards.ts"]),
|
|
).toBeNull();
|
|
});
|
|
|
|
it("supplements mixed package diffs with the affected extension config", () => {
|
|
const changedPaths = [
|
|
"packages/gateway-protocol/src/frame-guards.ts",
|
|
"extensions/codex/src/session-upstream-marker.ts",
|
|
];
|
|
|
|
expect(createChangedNodeTestShards(changedPaths)).toBeNull();
|
|
expectBoundedCodexFallback(createChangedExtensionFallbackShards(changedPaths));
|
|
});
|
|
|
|
it("covers every extension config when core changes can impact extension consumers", () => {
|
|
const shards = createChangedExtensionFallbackShards([
|
|
"src/gateway/tool-resolution.ts",
|
|
"src/agents/openclaw-tools.ts",
|
|
"extensions/discord/src/channel.ts",
|
|
]);
|
|
|
|
expectAllExtensionConfigs(shards);
|
|
});
|
|
|
|
it("covers every extension config when the fallback planner itself changes", () => {
|
|
expectAllExtensionConfigs(
|
|
createChangedExtensionFallbackShards(["scripts/lib/ci-changed-node-test-plan.mts"]),
|
|
);
|
|
});
|
|
|
|
it("covers every extension config when the extension inventory changes", () => {
|
|
expectAllExtensionConfigs(
|
|
createChangedExtensionFallbackShards(["scripts/lib/changed-extensions.mts"]),
|
|
);
|
|
});
|
|
|
|
it("classifies core and fallback-gate extension impact", () => {
|
|
expect(hasCoreExtensionImpact(["src/agents/openclaw-tools.ts"])).toBe(true);
|
|
expect(hasCoreExtensionImpact(["scripts/lib/changed-extensions.mts"])).toBe(true);
|
|
expect(hasCoreExtensionImpact(["scripts/lib/ci-changed-node-test-plan.mts"])).toBe(true);
|
|
expect(hasCoreExtensionImpact(["scripts/lib/extension-test-plan.mts"])).toBe(true);
|
|
expect(hasCoreExtensionImpact(["extensions/discord/src/channel.ts"])).toBe(false);
|
|
expect(hasCoreExtensionImpact(["docs/ci.md"])).toBe(false);
|
|
});
|
|
|
|
it("keeps extension-only fallbacks scoped to the changed extension config", () => {
|
|
expect(createChangedExtensionFallbackShards(["extensions/discord/src/channel.ts"])).toEqual([
|
|
{
|
|
checkName: "checks-node-changed-extensions-config",
|
|
configs: ["test/vitest/vitest.extension-discord.config.ts"],
|
|
requiresDist: false,
|
|
runner: "blacksmith-8vcpu-ubuntu-2404",
|
|
shardName: "changed-extensions-config",
|
|
},
|
|
]);
|
|
});
|
|
|
|
it("does not create extension fallback shards for docs-only diffs", () => {
|
|
expect(createChangedExtensionFallbackShards(["docs/ci.md"])).toEqual([]);
|
|
});
|
|
|
|
it.each([
|
|
{
|
|
changedPath: "extensions/browser/src/browser/cdp.helpers.test.ts",
|
|
config: "test/vitest/vitest.extension-browser.config.ts",
|
|
},
|
|
{
|
|
changedPath: "extensions/codex/src/session-upstream-marker.ts",
|
|
config: "test/vitest/vitest.extension-codex.config.ts",
|
|
},
|
|
])("runs the whole owning extension config for $changedPath", ({ changedPath, config }) => {
|
|
const shards = createChangedNodeTestShards([changedPath]);
|
|
|
|
expect(shards).not.toBeNull();
|
|
expect(shards?.flatMap((shard) => shard.configs)).toContain(config);
|
|
});
|
|
|
|
it("packs Telegram process lifetimes into bounded changed-extension jobs", () => {
|
|
const shards = createChangedExtensionFallbackShards(["extensions/telegram/src/channel.ts"]);
|
|
const targets = shards.flatMap((shard) => shard.includePatterns ?? []);
|
|
|
|
expect(shards.length).toBeGreaterThan(1);
|
|
expect(
|
|
shards.every(
|
|
(shard) =>
|
|
shard.configs[0] === "test/vitest/vitest.extension-telegram.config.ts" &&
|
|
(shard.includePatterns?.length ?? 0) > 0 &&
|
|
(shard.includePatterns?.length ?? 0) <= 10,
|
|
),
|
|
).toBe(true);
|
|
expect(targets.length).toBeGreaterThan(10);
|
|
expect(new Set(targets).size).toBe(targets.length);
|
|
expect(shards).toHaveLength(Math.ceil(targets.length / 10));
|
|
});
|
|
|
|
it("preserves Matrix process bounds in mixed package fallbacks", () => {
|
|
const shards = createChangedExtensionFallbackShards([
|
|
"packages/gateway-protocol/src/frame-guards.ts",
|
|
"extensions/matrix/src/channel.ts",
|
|
]);
|
|
const targets = shards.flatMap((shard) => shard.includePatterns ?? []);
|
|
|
|
expect(shards.length).toBeGreaterThan(1);
|
|
expect(
|
|
shards.every(
|
|
(shard) =>
|
|
shard.configs[0] === "test/vitest/vitest.extension-matrix.config.ts" &&
|
|
(shard.includePatterns?.length ?? 0) > 0 &&
|
|
(shard.includePatterns?.length ?? 0) <= 40,
|
|
),
|
|
).toBe(true);
|
|
expect(targets.length).toBeGreaterThan(40);
|
|
expect(new Set(targets).size).toBe(targets.length);
|
|
});
|
|
|
|
it("skips extension fallback when the core-impact predicate does not fire", () => {
|
|
expect(createChangedExtensionFallbackShards(["src/agents/live-model-filter.ts"])).toEqual([]);
|
|
});
|
|
|
|
it("falls back to bounded Codex config shards for deleted sources", () => {
|
|
const cwd = mkdtempSync(path.join(tmpdir(), "openclaw-ci-extension-fallback-"));
|
|
try {
|
|
expectBoundedCodexFallback(
|
|
createChangedExtensionFallbackShards(["extensions/codex/src/deleted-session-runtime.ts"], {
|
|
cwd,
|
|
}),
|
|
);
|
|
expect(
|
|
createChangedExtensionFallbackShards(
|
|
["extensions/codex/src/deleted-session-runtime.test.ts"],
|
|
{ cwd },
|
|
),
|
|
).toEqual([]);
|
|
} finally {
|
|
rmSync(cwd, { force: true, recursive: true });
|
|
}
|
|
});
|
|
|
|
it("serializes the Memory Core extension fallback config", () => {
|
|
expect(
|
|
createChangedExtensionFallbackShards(["extensions/memory-core/src/memory/mmr.ts"]),
|
|
).toEqual([
|
|
{
|
|
checkName: "checks-node-changed-extensions-config",
|
|
configs: ["test/vitest/vitest.extension-memory.config.ts"],
|
|
planConcurrency: 1,
|
|
requiresDist: false,
|
|
runner: "blacksmith-8vcpu-ubuntu-2404",
|
|
shardName: "changed-extensions-config",
|
|
},
|
|
]);
|
|
});
|
|
|
|
it("prebuilds private QA dist before the QA Lab extension fallback", () => {
|
|
expect(createChangedExtensionFallbackShards(["extensions/qa-lab/src/cli.runtime.ts"])).toEqual([
|
|
expect.objectContaining({
|
|
configs: ["test/vitest/vitest.extension-qa.config.ts"],
|
|
pretestBuildMode: "private-qa",
|
|
}),
|
|
]);
|
|
});
|
|
|
|
it("fails safe when a targeted config needs special shard setup", () => {
|
|
expect(createChangedNodeTestShards(["scripts/docs-i18n/main.go"])).toBeNull();
|
|
expect(createChangedNodeTestShards(["src/tui/tui-pty-harness.e2e.test.ts"])).toBeNull();
|
|
});
|
|
|
|
it("fails safe when an unresolved source only finds an unrelated directory test", () => {
|
|
const cwd = mkdtempSync(path.join(tmpdir(), "openclaw-ci-target-"));
|
|
try {
|
|
mkdirSync(path.join(cwd, "src"));
|
|
writeFileSync(path.join(cwd, "src/value.ts"), "export const value = 1;\n");
|
|
writeFileSync(path.join(cwd, "src/unrelated.test.ts"), "export const unrelated = true;\n");
|
|
expect(createChangedNodeTestShards(["src/value.ts"], { cwd })).toBeNull();
|
|
} finally {
|
|
rmSync(cwd, { force: true, recursive: true });
|
|
}
|
|
});
|
|
|
|
it("fails safe for aggregate full-suite configs", () => {
|
|
expect(
|
|
createChangedNodeTestShards(["test/vitest/vitest.full-core-support-boundary.config.ts"]),
|
|
).toBeNull();
|
|
});
|
|
|
|
it("fails safe for leaf configs split across full-suite processes", () => {
|
|
expect(createChangedNodeTestShards(["test/vitest/vitest.commands.config.ts"])).toBeNull();
|
|
});
|
|
|
|
it("fails safe when source targets expand to a whole config", () => {
|
|
expect(
|
|
createChangedNodeTestShards(["ui/src/app-routes.ts", "ui/src/app-navigation.ts"]),
|
|
).toBeNull();
|
|
});
|
|
|
|
it("chunks many targets into bounded parallel jobs", () => {
|
|
// A wide test-file diff exercises the multi-chunk path against the real
|
|
// tree; the cron suite has well over one chunk's worth of test files.
|
|
const changedTests = listGitTrackedFiles({ pathspecs: "src/cron" })
|
|
?.filter((file) => file.endsWith(".test.ts") && !/\.(?:e2e|live)\.test\.ts$/u.test(file))
|
|
.slice(0, 15);
|
|
expect(changedTests?.length).toBe(15);
|
|
const shards = createChangedNodeTestShards(changedTests ?? []);
|
|
expect(shards).not.toBeNull();
|
|
const targetShards = shards?.filter((shard) => shard.targets) ?? [];
|
|
expect(targetShards.length).toBeGreaterThan(1);
|
|
expect(
|
|
targetShards.every((shard, index) => shard.checkName === `checks-node-changed-${index + 1}`),
|
|
).toBe(true);
|
|
expect(targetShards.every((shard) => (shard.targets?.length ?? 0) <= 12)).toBe(true);
|
|
const targets = targetShards.flatMap((shard) => shard.targets ?? []);
|
|
expect(new Set(targets).size).toBe(targets.length);
|
|
});
|
|
|
|
it("serializes the owning Memory Core extension config for direct changes", () => {
|
|
const shards = createChangedNodeTestShards([
|
|
"extensions/memory-core/src/memory/mmr.ts",
|
|
"extensions/memory-core/src/memory/mmr.test.ts",
|
|
]);
|
|
expect(shards).not.toBeNull();
|
|
expect(shards).toContainEqual({
|
|
checkName: "checks-node-changed-extensions-config",
|
|
configs: ["test/vitest/vitest.extension-memory.config.ts"],
|
|
planConcurrency: 1,
|
|
requiresDist: false,
|
|
runner: "blacksmith-8vcpu-ubuntu-2404",
|
|
shardName: "changed-extensions-config",
|
|
});
|
|
});
|
|
});
|