diff --git a/.agents/skills/openclaw-live-updater/SKILL.md b/.agents/skills/openclaw-live-updater/SKILL.md index 1ff01276ded5..abd0e239b3fd 100644 --- a/.agents/skills/openclaw-live-updater/SKILL.md +++ b/.agents/skills/openclaw-live-updater/SKILL.md @@ -57,10 +57,12 @@ Load `$release-openclaw-ci` and `$openclaw-testing`. This is validation only, ne 2. Only when the cadence is due, confirm no full/all run is active, then snapshot exact current `origin/main` after checking mirror invariants. Run the provider-secret preflight without printing secrets and dispatch the trusted workflow once: ```bash + MAIN_SHA="" gh workflow run full-release-validation.yml \ --repo openclaw/openclaw \ --ref main \ - -f ref= \ + -f ref="$MAIN_SHA" \ + -f expected_sha="$MAIN_SHA" \ -f provider=openai \ -f mode=both \ -f release_profile=full \ diff --git a/.agents/skills/openclaw-parallels-smoke/SKILL.md b/.agents/skills/openclaw-parallels-smoke/SKILL.md index 1fcceec86d75..cabc6c174646 100644 --- a/.agents/skills/openclaw-parallels-smoke/SKILL.md +++ b/.agents/skills/openclaw-parallels-smoke/SKILL.md @@ -9,6 +9,17 @@ Use this skill for Parallels guest workflows and smoke interpretation. Do not lo ## Global rules +- Parallels is postpublish-confidence coverage by default, not a prerequisite + for the bounded beta-publish phase. Run it against one exact published + package version and record the Validation SHA + Tooling SHA run tuple that + authorized that package. Validation SHA maps to its Code SHA or Release SHA; + it is not a third release identity. +- A failed postpublish Parallels lane does not retroactively unpublish a beta. + Classify it as product, harness/tooling/provenance, infrastructure/credential, + or wrapper failure. Only a confirmed product defect enters the next beta. +- Use one coordinator and one lane per independent guest family. Diagnose and + retry one failed family once, then reassess; do not restart the aggregate + matrix automatically. - Inventory existing VMs and snapshots before provisioning anything. When a preconfigured pristine snapshot matches the requested baseline, switch to it and reuse its user, tools, and base setup. Do not create a new VM, reinstall macOS, or rebuild the guest baseline for a "fresh" run. diff --git a/.agents/skills/openclaw-testing/SKILL.md b/.agents/skills/openclaw-testing/SKILL.md index dbb1da773c8d..05565be14b86 100644 --- a/.agents/skills/openclaw-testing/SKILL.md +++ b/.agents/skills/openclaw-testing/SKILL.md @@ -288,21 +288,25 @@ rerun after a focused patch. ### Full Release Validation `Full Release Validation` (`.github/workflows/full-release-validation.yml`) is -the manual product-validation umbrella. Run the full child matrix on the -product-complete pre-changelog **Code SHA**. It resolves a target ref, then +the manual product-validation umbrella. Bind each run to the immutable +**Validation SHA + Tooling SHA** tuple. Validation SHA maps to the Code SHA for +product validation or the Release SHA for changelog-only validation; it is not +a third release identity. The workflow resolves it before child dispatch, then dispatches: - manual `CI` for the full normal CI graph, with Android enabled via `include_android=true` - `Plugin Prerelease` for release-only plugin static checks, extension shards, the release-only `agentic-plugins` shard, and plugin product Docker lanes -- `OpenClaw Release Checks` for install smoke, cross-OS release checks, live and - E2E checks, Docker release-path suites, OpenWebUI, QA Lab, fast Matrix, and - Telegram release lanes +- `OpenClaw Release Checks` for install smoke, cross-OS release checks, package + acceptance, and QA parity; broad live/E2E and QA-live lanes join `all` only + when release soak is enabled - optional post-publish Telegram E2E when a package spec is supplied -Run the full matrix only when validating an actual Code SHA, after broad shared -CI or release orchestration changes, or when explicitly asked: +For beta-publish, use `release_profile=beta` with +`run_release_soak=false`. Postpublish-confidence uses the exact published +package with `run_release_soak=true` or explicit focused groups. +Stable-publish uses `release_profile=stable`. ```bash node scripts/full-release-validation-at-sha.mjs \ @@ -316,14 +320,12 @@ Validation directly from and against `extended-stable/YYYY.M.33` with replaced by a `release-ci/*` run. Use `$release-openclaw-ci` for its failure classification and run-identity rules. -The helper pins the trusted workflow revision on current `main` while targeting -the historical release SHA and recording the canonical release branch as -context. It infers `beta` for alpha/beta package versions and `stable` for +The helper pins the Tooling SHA on trusted `main`, passes the resolved Code SHA +as `expected_sha`, and records the canonical release branch as context. It +infers `beta` for alpha/beta package versions and `stable` for stable/correction versions. Pass `-f release_profile=full` only for the broad advisory provider/media sweep. Do not make `full` faster by silently dropping -suites; optimize setup, artifact reuse, and sharding instead. The parent -verifier job appends a child overview plus slowest-job tables for child runs; -rerun only that verifier after a child rerun turns green. +suites; use the bounded phase that matches the release decision. Standalone manual `CI` dispatches do not run the plugin prerelease suite, the extension batch sweep, or the release-only `agentic-plugins` Vitest shard. Those @@ -331,15 +333,16 @@ lanes are intentionally reserved for the separate `Plugin Prerelease` child so PRs, main pushes, and ad hoc broad CI checks do not spend Docker/package time or all-plugin runtime time on release-only product coverage. -If a full run is already active on a newer `origin/main`, prefer watching that -run over dispatching a duplicate. Do not cancel release, release-check, or child -workflow runs unless Peter explicitly asks for cancellation. +Use one operator, one transition-only watcher, and at most one investigator for +the current failed surface. Parent timeout or cancellation leaves adopted exact +children running; cancel an exact child only by explicit operator action or the +workflow's identity-mismatch/fail-fast path. -The child-dispatch jobs record the child run ids. The final -`Verify full validation` job re-queries those child runs and is the canonical -parent gate. If a child workflow failed but was later rerun successfully, rerun -only the failed parent verifier job; do not dispatch a new full umbrella unless -the release evidence is stale. +The child-dispatch jobs record child run ids, and `Verify full validation` +re-queries them during that parent attempt. A later narrow green run is useful +recovery evidence but is not publish authorization by itself and there is no +standalone finalizer. The release owner must reassess the recorded evidence and +current publish gate. Once the Code SHA is green, generate and commit only `CHANGELOG.md`. The new **Release SHA** is eligible for product-evidence reuse only when GitHub proves @@ -350,14 +353,16 @@ SHA children. Package, install/update, and release-note proof still runs on the Release SHA because its tarball bytes changed. Any non-changelog path invalidates reuse and requires a new Code SHA full matrix. -For bounded recovery after a focused fix, pass `-f rerun_group=`. +For bounded recovery, classify the failure as product, +harness/tooling/provenance, infrastructure/credential, or wrapper before +editing. Only a confirmed product failure changes the Code SHA. Use one +diagnosis, one fix when needed, and one narrow retry with +`-f rerun_group=`, then reassess. Supported umbrella groups are `all`, `ci`, `plugin-prerelease`, `release-checks`, `install-smoke`, `cross-os`, `live-e2e`, `package`, `qa`, `qa-parity`, `qa-live`, and `npm-telegram`. Use the narrowest group that covers -the failed box. After a targeted release-check fix, do not restart the full -umbrella by habit: dispatch the matching `rerun_group` and rerun only the parent -verifier/evidence step after the child is green unless the release evidence is -stale. For a single failed live/E2E shard, use +the failed box. Do not automatically dispatch `all` after a narrow retry. For a +single failed live/E2E shard, use `-f rerun_group=live-e2e -f live_suite_filter=` so the Blacksmith workflow only spends setup and queue time on that suite. diff --git a/.agents/skills/release-openclaw-ci/SKILL.md b/.agents/skills/release-openclaw-ci/SKILL.md index 42d30d721bd6..6e26879561ab 100644 --- a/.agents/skills/release-openclaw-ci/SKILL.md +++ b/.agents/skills/release-openclaw-ci/SKILL.md @@ -10,8 +10,19 @@ Use this with `$release-openclaw-maintainer` and `$openclaw-testing` when a rele ## Guardrails - No version bump, tag, npm publish, GitHub release, or release promotion without explicit operator approval. +- After compaction, resume, or new steering, rewrite the effective goal and + current phase from the latest explicit operator instruction. Do not merge old + scope back into the active release. - Hold the release scope once a release branch or Code SHA exists. Validate and ship that exact release; do not turn moving `main` into a second work queue. +- Record every active validation run as the immutable tuple **Validation SHA + + Tooling SHA**. Validation SHA maps to the Code SHA for product validation or + the Release SHA for changelog-only validation; it is not a third release + identity. A branch or temporary ref is context and transport. +- Apply a release firebreak after the Code SHA is frozen. Admit only confirmed + product defects, package/provenance defects in the bytes to publish, security + defects, or failures that make publication impossible. Queue other findings + for postpublish confidence or the next beta. - Use trusted `main` workflow revisions as immutable dispatch sources. Do not adopt newer main code, repair unrelated main CI, wait for broad main health, or expand a release fix because the workflow source lives on `main`. @@ -33,6 +44,9 @@ Use this with `$release-openclaw-maintainer` and `$openclaw-testing` when a rele - Full Release Validation collects independent child failures to terminal completion by default. Pass `fail_fast=true` only when the shorter first-failure cancellation path is preferable. +- Use one release operator, one transition-only watcher, and at most one + investigator for the current failed surface. Do not build audit-review-plan + trees around a single workflow transition. - For regular beta/stable releases, treat the product-complete pre-changelog commit as the Code SHA. Full product validation and performance evidence bind to that SHA. The later Release SHA may reuse those results only when it is a @@ -65,27 +79,29 @@ Use this with `$release-openclaw-maintainer` and `$openclaw-testing` when a rele ## Run identity and retry budget -Record the target SHA, target ref, parent run id, attempt, and effective -workflow SHA before watching or recovering Full Release Validation. +Record Validation SHA, Tooling SHA, target context ref, parent run id, attempt, +and phase before watching or recovering Full Release Validation. Keep Code SHA +and Release SHA separately in the lifecycle ledger. -- One target SHA has one active canonical `rerun_group=all` parent run by - default. -- Use GitHub's failed-job rerun on the same parent when its original inputs - still select the correct work. -- A new parent for the same target SHA is allowed only when no usable parent - exists, the existing run cannot consume a required trusted-workflow fix, its - evidence identity is invalid, or the operator explicitly requests a fresh - run. The replacement must also use `rerun_group=all`; record which terminal - parent it supersedes and why. -- A narrow `rerun_group` dispatch is supplemental diagnostic proof only. It - never supersedes the canonical parent and cannot satisfy publish evidence. - Run it only after the canonical parent is terminal, then obtain green - `rerun_group=all` evidence before publish. -- Never keep two parents active for the same SHA. Cancel only the superseded - task-owned run after the replacement is identified. -- After two unchanged retries of the same failure, stop repeating it. Recheck - classification and report one precise blocker or the missing evidence needed - for a different action. +- Conceptual phases map to current inputs as follows: + - `beta-publish`: `release_profile=beta`, `run_release_soak=false` + - `postpublish-confidence`: published package inputs with + `run_release_soak=true` or explicit focused groups + - `stable-publish`: `release_profile=stable` +- Keep at most one active parent for the same Validation SHA + Tooling SHA + rerun + group. Concurrency does not cancel an older exact child automatically. +- Parent cancellation or timeout leaves an adopted identity-checked child + running. The operator must cancel that exact child explicitly when it is no + longer useful. +- Recover one failed surface with one diagnosis, one fix when needed, and one + narrow retry. Then reassess the release decision. Do not automatically + dispatch `rerun_group=all`. +- A new all-group parent is justified only when shared orchestration changed, + earlier evidence is invalid for the selected tuple, or the operator explicitly + requests it. Record the invalidating event. +- Narrow child or group evidence does not by itself become publish + authorization. Keep it in the evidence ledger for the release owner to judge + against the current publish gate. ## Preflight @@ -163,10 +179,11 @@ dispatching child lanes. Npm preflight and package/install acceptance still run against the exact Release SHA and its new tarball bytes. The SHA-pinned helper infers `beta` for alpha/beta package versions and `stable` -for stable/correction versions. Pass `release_profile=full` only when the -operator explicitly asks for the broad advisory provider/media matrix. Stable -and full profiles force the release soak; the beta profile may opt in with -`run_release_soak=true`. Use narrow `rerun_group` after focused fixes. +for stable/correction versions and passes the Validation SHA + Tooling SHA run +identity. `beta` without soak is the bounded beta-publish gate. Run broad live +QA and E2E as postpublish confidence with `run_release_soak=true` or explicit +groups. Stable and full profiles force the release soak. Use a narrow +`rerun_group` after focused fixes; never widen automatically. Publish with `openclaw-release-publish.yml` using `release_profile=from-validation` unless a maintainer intentionally wants to cross-check a specific profile; the publish workflow reads the effective profile from the full-validation manifest. @@ -177,9 +194,11 @@ For `.33+`, dispatch from and target the canonical branch; the regular SHA-pinned helper would produce a rejected `release-ci/*` identity: ```bash +RELEASE_SHA="$(git rev-parse HEAD)" gh workflow run full-release-validation.yml \ --ref extended-stable/YYYY.M.33 \ -f ref=extended-stable/YYYY.M.33 \ + -f expected_sha="$RELEASE_SHA" \ -f release_profile=stable ``` @@ -225,18 +244,21 @@ Stop watchers before ending the turn or switching strategy. Anthropic API-key lane. 5. For live-cache failures, inspect whether it is missing/invalid key, empty text, provider refusal, timeout, or baseline miss. Do not weaken release gates without clear provider evidence. 6. Classify before editing: - - product/code failure: fix the release branch, freeze a new Code SHA, run - focused proof, then obtain green full validation for that new SHA - - workflow/harness/infrastructure/credential failure: fix the smallest - owning surface and rerun failed jobs on the current parent when its inputs - still select the correct work; otherwise supersede the terminal parent once - with a new `rerun_group=all` parent on the required trusted-workflow - revision. Keep the same Code SHA; touching `main` still requires the active - release scope lock + - confirmed product/code failure: fix the release branch, freeze a new Code + SHA, and invalidate product evidence + - harness/tooling/provenance failure: keep the Code SHA, fix the smallest + owning surface, and retry only the failed surface with the required Tooling + SHA + - infrastructure/credential failure: keep both SHAs, repair the external + prerequisite, and retry only the failed surface + - wrapper/monitor failure: keep the child and candidate identities; record + the wrapper result separately from the child result - changelog/release-note failure: change only `CHANGELOG.md`, keep Code SHA evidence, and repeat Release SHA proof - publish child/registry selector failure: keep Release SHA and resume the failed child; never rebuild an immutable version that already published + Only the first class changes the Code SHA. After one diagnosis/fix/narrow + retry, reassess instead of starting another all-group cycle. 7. If a required PR CI run is capacity-stalled with queued jobs and no active jobs, do not cancel unrelated work or accept a generic manual dispatch. First verify the PR head carries the current fallback schema: @@ -261,8 +283,8 @@ include_android=true -f release_gate=true`. Record: -- release identity: Code/Release SHAs for regular releases; canonical branch, - exact SHA, and immutable tag for extended-stable +- release lifecycle ledger: Code SHA, Release SHA, and Tooling SHA for regular + releases; canonical branch, exact SHA, and immutable tag for extended-stable - evidence-reuse policy and complete changed-path set - active full parent run URL, attempt, workflow SHA, and any superseded parent with the exact replacement reason diff --git a/.agents/skills/release-openclaw-ci/references/release-ci-notes.md b/.agents/skills/release-openclaw-ci/references/release-ci-notes.md index 69a248808c95..ffd8716a26dd 100644 --- a/.agents/skills/release-openclaw-ci/references/release-ci-notes.md +++ b/.agents/skills/release-openclaw-ci/references/release-ci-notes.md @@ -18,14 +18,16 @@ - Fetch failed-job logs only after a job reaches a terminal failing state. - Prefer same-parent failed-job reruns when the original inputs still select the right work. -- Keep one active parent per target SHA. Create a replacement only when the - existing parent cannot consume a required workflow fix or its evidence - identity is invalid; record the superseded run and reason. The replacement - remains `rerun_group=all`. -- Treat narrow `rerun_group` dispatches as supplemental diagnostics only; they - do not satisfy publish evidence. -- Stop after two unchanged retries and report the exact blocker instead of - creating another verification loop. +- Keep one active parent per exact Validation SHA + Tooling SHA + rerun group. Create + a replacement only when the current evidence is invalid or cannot consume a + required workflow fix; record the invalidating event and do not widen + automatically to `rerun_group=all`. +- Classify one failed surface, make one fix when needed, and retry the narrowest + failed group once. Then reassess whether to ship, explicitly waive, or block + instead of creating another verification loop. +- Preserve successful exact-tuple evidence when the documented finalization + rules allow reuse. Narrow evidence does not become publish authorization by + itself, and there is no standalone rerunnable finalizer today. - Leave bad secrets unset. A 401 candidate from 1Password should not overwrite GitHub. - Make the final release evidence note durable: parent URL, child run URLs, SHA, command proof, and gaps. diff --git a/.agents/skills/release-openclaw-maintainer/SKILL.md b/.agents/skills/release-openclaw-maintainer/SKILL.md index 1202796a87a4..4eb6cd4e4840 100644 --- a/.agents/skills/release-openclaw-maintainer/SKILL.md +++ b/.agents/skills/release-openclaw-maintainer/SKILL.md @@ -17,7 +17,7 @@ Before validation or publication, write one compact state record and keep it current: - goal and terminal success criteria -- release version, tag, branch, cut SHA, Code SHA, and Release SHA +- release version, tag, branch, cut SHA, Code SHA, Tooling SHA, and Release SHA - active Full Release Validation parent run id and attempt - npm preflight and publish parent run ids - completed phases and immutable child artifacts @@ -26,6 +26,9 @@ current: Use `references/release-handoff-template.md` when starting a release session, recovering after compaction, or handing the release to another operator. +After compaction, resume, or new steering, the latest explicit operator +instruction replaces the effective goal and phase; do not merge superseded +scope back into the release. Completed phases stay complete. Reopen one only when a named event invalidates its evidence, such as a Code SHA change, a non-changelog Release SHA change, or a workflow fix that the existing parent run cannot consume. @@ -94,6 +97,14 @@ a workflow fix that the existing parent run cannot consume. infrastructure, repair the smallest owning surface and rerun against the same Code SHA. Touch `main` only under the active release scope lock above. Never mutate the release candidate to satisfy newer tooling or heal unrelated main. +- Apply a release firebreak after the Code SHA is frozen. Admit only confirmed + product defects, package/provenance defects in the bytes to publish, security + defects, or failures that make publication impossible. Queue adjacent + improvements and broad confidence findings for the next beta or postpublish + work. +- Operate with one release owner, one transition-only watcher, and at most one + investigator for the current failed surface. One diagnosis, one fix when + needed, and one narrow retry consume the failure budget; then reassess. - Generate `CHANGELOG.md` only after the Code SHA is green. The resulting **Release SHA** must be a descendant whose complete diff from the Code SHA is exactly `CHANGELOG.md`. Release-note checks, npm preflight/package bytes, @@ -743,10 +754,10 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts - Parallels validation and any local live model QA for this train must use both `OPENAI_API_KEY` and `ANTHROPIC_API_KEY`. If either cannot be injected, stop before starting those local long lanes and report the missing key. -- Live credentialed channel QA is the GitHub Actions workflow +- Postpublish credentialed channel QA is the GitHub Actions workflow `QA-Lab - All Lanes` (`.github/workflows/qa-live-transports-convex.yml`), not a - local substitute. Dispatch it from Actions against the release tag and wait - for it to pass before npm preflight/publish readiness. Use a SHA only when it + local substitute. Dispatch it from Actions against the published release tag + during postpublish confidence. Use a SHA only when it satisfies the workflow's secret-bearing trust gate: main ancestor or open PR head. It runs the QA Lab mock parity gate plus live Matrix and live Telegram lanes using the `qa-live-shared` environment; Telegram uses Convex CI @@ -759,21 +770,15 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts - `pnpm ui:build` - `pnpm release:check` - `OPENCLAW_INSTALL_SMOKE_SKIP_NONROOT=1 pnpm test:install:smoke` -- Full pre-npm beta test roster: - - default release checks above - - all Docker tests: `pnpm test:docker:all`, plus standalone Docker live lanes - not covered by the aggregate when operator says "all docker tests": - `pnpm test:docker:live-acp-bind`, `pnpm test:docker:live-cli-backend`, and - `pnpm test:docker:live-codex-harness` - - all Parallels install/update tests: - `pnpm test:parallels:npm-update -- --json` plus any needed individual - rerun lanes from `openclaw-parallels-smoke` - - all QA release validation: dispatch GitHub Actions > `QA-Lab - All Lanes` - against the release tag and require success. This is the release gate for - live credentialed Matrix/Telegram channel coverage. Use a SHA only when it - satisfies the workflow trust gate. Run local OpenAI/Anthropic suites or - repo-backed character evals only when the operator asks for extra model - coverage or a failure needs local debugging. +- Release validation phases: + - `beta-publish`: `release_profile=beta`, `run_release_soak=false`. This is + the bounded prepublish gate; it excludes broad live/E2E, QA-live, and + Parallels confidence work. + - `postpublish-confidence`: run against the exact published beta package with + `run_release_soak=true` or explicit focused groups. This is the default home + for QA-live, broad Docker/live E2E, mobile, and Parallels. + - `stable-publish`: `release_profile=stable`; require the stable publish + roster and accepted confidence evidence. - Post-published beta verification roster: - `node --import tsx scripts/openclaw-npm-postpublish-verify.ts ` - install/update smoke against the published beta channel @@ -794,8 +799,8 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts operator explicitly scopes a harness-only isolation check; a lane that disables bundled plugin installs is not valid plugin/dependency release evidence. - - targeted QA reruns only for areas touched by fixes after the full pre-npm - roster, unless the operator requests the full QA roster again. If the fix + - targeted QA reruns only for areas touched by fixes after the beta-publish + gate, unless the operator requests the full QA roster again. If the fix touches live channel QA, credential plumbing, Matrix, Telegram, or the QA harness, rerun Actions > `QA-Lab - All Lanes`. - Check all release-related build surfaces touched by the release, not only the npm package. @@ -837,7 +842,10 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts Docker/Parallels matrix unless the beta evidence is stale, the stable build differs materially from beta, or the operator explicitly asks for full retesting. -- If any required build, packaging step, or release workflow is red, do not say the release is ready. +- A red required surface blocks readiness only after classification shows that + it invalidates the current phase or publish gate. Report harness, provenance, + infrastructure, credential, and wrapper failures separately from confirmed + product failures. ## Use the right auth flow @@ -1018,9 +1026,12 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts 8. Run the deterministic source preflight, then Full Release Validation against the exact Code SHA with `node scripts/full-release-validation-at-sha.mjs --sha --target-ref release/YYYY.M.PATCH`. - Use one transition watcher. Product failures return to step 6 with a new - Code SHA; tooling/harness failures are fixed separately and rerun against the - same Code SHA. + For beta-publish, keep `release_profile=beta` and + `run_release_soak=false`. Record the Validation SHA + Tooling SHA tuple + (Validation SHA is the Code SHA in this phase) and use one transition + watcher. Product failures return to step 6 with a new Code SHA; + tooling/harness/provenance, infrastructure/credential, and wrapper failures + keep the Code SHA and rerun only the failed surface. 9. After the Code SHA is green, run `/changelog` once for the stable-base target version using current `origin/main` for canonical PR provenance. Keep the heading as `## YYYY.M.PATCH`, not `## YYYY.M.PATCH-beta.N`. @@ -1046,7 +1057,10 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts --npm-preflight-run --skip-dispatch` to consume the existing reused full evidence and exact Release SHA preflight instead of dispatching either again. It completes - package/install proof and prints the publish command. + package/install proof and prints the publish command. Beta and alpha + candidates defer Parallels to postpublish `pnpm release:beta-smoke` by + default; stable/full candidates run it prepublish. Use `--run-parallels` or + `--skip-parallels` only for an explicit operator override. 18. Start publication only after the candidate bundle is green. Reuse successful immutable child runs/artifacts on retry; do not rebuild or republish versions that already succeeded. @@ -1063,12 +1077,15 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts `openclaw/releases/.github/workflows/openclaw-macos-publish.yml` with `preflight_only=true` and wait for it to pass. Save that run id because the real publish requires it to reuse the notarized mac artifacts. -23. Classify every failure before changing git state. Product defects return to +23. Classify every failure before changing git state. Confirmed product defects return to step 6 and invalidate downstream Code/Release SHA evidence. Changelog or release-note defects change only the Release SHA and reuse the green Code - SHA evidence after the exact delta is reverified. Tooling, credential, - approval, registry selector, or publication-child failures keep the - candidate unchanged and resume the smallest failed surface. + SHA evidence after the exact delta is reverified. Tooling, provenance, + credential, infrastructure, wrapper, approval, registry selector, or + publication-child failures keep the candidate unchanged and resume the + smallest failed surface. Only confirmed product failure changes the Code + SHA. After one diagnosis/fix/narrow retry, reassess; do not automatically + rerun `all`. 24. Start `.github/workflows/openclaw-release-publish.yml` from the exact pinned trusted workflow source with the same tag for the real beta or stable publish, choose `npm_dist_tag` (`beta` default, @@ -1099,17 +1116,23 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts 27. Run the post-published beta verification roster. Do not scan current `main` for extra fixes unless the operator explicitly requests a backport audit. Apply only operator-selected backports, and increment to the next beta if a - selected fix must change the already-published package. If any - lane fails after the beta package is published, fix, commit/push/pull, - increment to the next beta tag, and rerun the affected beta evidence. Once + selected fix must change the already-published package. A failed confidence + lane does not retroactively unpublish the beta. Classify it first; only a + confirmed product defect admitted by the release owner creates a fix, + increments the next beta tag, and reruns affected evidence. Once the beta is live, start remote/manual rosters where they can overlap safely, but keep local Docker and Parallels load controlled. Ensure the full expensive roster has passed at least once before stable/latest promotion. The roster includes the manual Actions > `NPM Telegram Beta E2E` workflow against the exact published beta package. - If a pre-npm lane fails before any tag/package leaves the machine, fix and - rerun the same intended beta attempt. Repeat up to the operator's - authorized beta-attempt limit, normally 4. + If a pre-npm lane fails before any tag/package leaves the machine, classify + it first and use one diagnosis, one fix when needed, and one narrow retry. + Only an operator-admitted confirmed product defect may create a new intended + beta attempt. Harness/tooling/provenance, infrastructure/credential, and + wrapper failures keep the candidate identity and do not authorize another + all-group cycle. The operator's separate beta-attempt cap, normally 4, + remains a ceiling for admitted product attempts, not an automatic retry + budget. 28. Announce the beta/stable release on Discord best-effort using the configured secret workflow. 29. If the operator requested beta only, stop after beta verification and the announcement. diff --git a/.agents/skills/release-openclaw-maintainer/references/release-handoff-template.md b/.agents/skills/release-openclaw-maintainer/references/release-handoff-template.md index 8fff78eb9cd9..0e66311256c9 100644 --- a/.agents/skills/release-openclaw-maintainer/references/release-handoff-template.md +++ b/.agents/skills/release-openclaw-maintainer/references/release-handoff-template.md @@ -7,6 +7,8 @@ captured by a durable run or artifact URL. ## Goal Ship `` on `` and stop when ``. +After compaction or resume, replace this section with the latest explicit +operator steering. Do not preserve superseded scope. ## Immutable state @@ -14,12 +16,14 @@ Ship `` on `` and stop when ``. - branch: `` - cut SHA: `` - Code SHA: `` +- Tooling SHA: `` - Release SHA: `` - tag: `v` - workflow ref: `` - publication inventory: `` - approved backports: `` - approved main changes: `` +- admitted release blockers: `` - frozen-target compatibility repairs: `` ## Active evidence @@ -34,27 +38,39 @@ Ship `` on `` and stop when ``. ## Phase +- conceptual phase: `` +- current input mapping: `` - completed: `` - current: `` - next action: `` +- roles: `` +- retry budget: `` ## Failure policy -- regular product/code failure: fix the release branch, freeze a new Code SHA, - and invalidate downstream product evidence +- confirmed product/code failure: fix the release branch, freeze a new Code + SHA, and invalidate downstream product evidence - regular changelog-only failure: change only `CHANGELOG.md`, freeze a new Release SHA, and reuse green Code SHA evidence after delta proof - extended-stable branch change: land the approved product/changelog change or smallest frozen-target repair by PR, record its source/invariant, and replace all exact-head evidence -- workflow/tooling/credential failure: keep the candidate frozen and recover - the smallest owning surface +- harness/tooling/provenance failure: keep the Code SHA, change the Tooling SHA + only when needed, and recover the smallest owning surface +- infrastructure/credential failure: keep both SHAs and repair the external + prerequisite +- wrapper/monitor failure: record parent and child conclusions separately; + parent cancellation leaves adopted children running until the operator + cancels them explicitly +- postpublish-confidence failure: do not retroactively unpublish the beta; + admit a confirmed product fix to the next beta - external approval or permission blocker: stop with the exact job, URL, missing permission, and required operator action Do not scan moving `main`, add optional backports, dispatch a replacement -validation parent, or repeat completed phases unless a named invalidating event -requires it. +validation parent, automatically rerun `all`, or repeat completed phases unless +a named invalidating event requires it. Narrow evidence informs the release +decision but is not publish authorization by itself. ## Stop conditions diff --git a/.agents/skills/release-openclaw-nightly/SKILL.md b/.agents/skills/release-openclaw-nightly/SKILL.md index 2d6bfcb383ee..755ebeba017a 100644 --- a/.agents/skills/release-openclaw-nightly/SKILL.md +++ b/.agents/skills/release-openclaw-nightly/SKILL.md @@ -226,6 +226,7 @@ BRANCH="$(git branch --show-current)" "$GH" workflow run full-release-validation.yml --repo openclaw/openclaw --ref "$BRANCH" \ -f ref="$BRANCH" \ + -f expected_sha="$SHA" \ -f release_profile=beta \ -f rerun_group=all diff --git a/.github/workflows/full-release-validation.yml b/.github/workflows/full-release-validation.yml index 0392b49747a5..64d4a5b27217 100644 --- a/.github/workflows/full-release-validation.yml +++ b/.github/workflows/full-release-validation.yml @@ -8,6 +8,11 @@ on: required: true default: main type: string + expected_sha: + description: Optional full Validation SHA that ref must resolve to + required: false + default: "" + type: string target_context_ref: description: Optional canonical release branch or tag context for an exact-SHA target required: false @@ -143,8 +148,8 @@ permissions: contents: read concurrency: - group: full-release-validation-${{ inputs.ref }}-${{ inputs.rerun_group }} - cancel-in-progress: ${{ (inputs.ref == 'main' && inputs.rerun_group == 'all') || startsWith(inputs.ref, 'tideclaw/alpha/') || startsWith(inputs.ref, 'release/') }} + group: full-release-validation-${{ inputs.expected_sha || inputs.ref }}-${{ github.sha }}-${{ inputs.rerun_group }} + cancel-in-progress: false env: FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true" @@ -175,9 +180,11 @@ jobs: env: TARGET_REF: ${{ inputs.ref }} TARGET_CONTEXT_REF: ${{ inputs.target_context_ref }} + EXPECTED_SHA: ${{ inputs.expected_sha }} run: | bash workflow/scripts/github/resolve-openclaw-ref.sh \ --ref "$TARGET_REF" \ + --expected-sha "$EXPECTED_SHA" \ --github-output "$GITHUB_OUTPUT" - name: Validate release inputs @@ -195,7 +202,7 @@ jobs: env: TARGET_REF: ${{ inputs.ref }} TARGET_SHA: ${{ steps.resolve.outputs.sha }} - CHILD_WORKFLOW_REF: ${{ github.ref_name }} + TOOLING_SHA: ${{ github.sha }} NPM_TELEGRAM_PACKAGE_SPEC: ${{ inputs.npm_telegram_package_spec }} RELEASE_PACKAGE_SPEC: ${{ inputs.release_package_spec }} EVIDENCE_PACKAGE_SPEC: ${{ inputs.evidence_package_spec }} @@ -214,8 +221,8 @@ jobs: echo "## Full release validation" echo echo "- Target ref: \`${TARGET_REF}\`" - echo "- Target SHA: \`${TARGET_SHA}\`" - echo "- Child workflow ref: \`${CHILD_WORKFLOW_REF}\`" + echo "- Validation SHA: \`${TARGET_SHA}\`" + echo "- Tooling SHA: \`${TOOLING_SHA}\`" echo "- Release soak lanes: \`${RUN_RELEASE_SOAK}\`" echo "- Fail fast: \`${FAIL_FAST}\`" echo "- Package Acceptance Telegram E2E deferred: \`${SKIP_PACKAGE_TELEGRAM_E2E}\`" @@ -456,7 +463,8 @@ jobs: needs: [resolve_target, evidence_reuse] if: ${{ always() && needs.resolve_target.result == 'success' && contains(fromJSON('["all","ci"]'), inputs.rerun_group) && needs.evidence_reuse.outputs.reuse != 'true' }} runs-on: ubuntu-24.04 - timeout-minutes: ${{ inputs.release_profile != 'beta' && 240 || 60 }} + # The child owns lane timeouts; this monitor also covers queue delay and iOS. + timeout-minutes: 240 outputs: run_id: ${{ steps.dispatch.outputs.run_id }} url: ${{ steps.dispatch.outputs.url }} @@ -678,23 +686,23 @@ jobs: fi cancel_child() { - if [[ -n "${active_child_run_id:-}" ]]; then - echo "Cancelling child workflow ${active_child_workflow}: ${active_child_run_id}" >&2 - gh run cancel "$active_child_run_id" >/dev/null 2>&1 || true + if [[ -n "${run_id:-}" ]]; then + echo "Cancelling child workflow ${workflow}: ${run_id}" >&2 + gh run cancel "$run_id" >/dev/null 2>&1 || true fi } run_json="$(validate_child_run "$run_id")" - # Identity fields prove cancellation ownership before workflow-SHA validation. - # EXIT traps run after function locals unwind; preserve only adopted child identity. - active_child_workflow="$workflow" - active_child_run_id="$run_id" - trap cancel_child EXIT INT TERM + # Generic monitor failures and parent cancellation leave an + # identity-checked child running for explicit operator recovery. + { + echo "- Adopted child: \`${workflow}\` run \`${run_id}\`" + echo "- Parent cancellation leaves this child running; cancel it explicitly if no longer needed." + } >> "$GITHUB_STEP_SUMMARY" child_head_sha="$(jq -r '.head_sha // ""' <<< "$run_json")" if [[ "$child_head_sha" != "$PARENT_WORKFLOW_SHA" ]]; then echo "::error::${workflow} child run used workflow SHA ${child_head_sha}, expected parent workflow SHA ${PARENT_WORKFLOW_SHA}." >&2 cancel_child - trap - EXIT INT TERM exit 1 fi @@ -760,7 +768,6 @@ jobs: fi jq '.[] | {name, conclusion, url: (.url // .html_url)}' <<< "$failed_jobs_json" cancel_child - trap - EXIT INT TERM exit 1 fi } @@ -781,7 +788,6 @@ jobs: fi sleep 60 done - trap - EXIT INT TERM if [[ "$workflow" == "openclaw-release-checks.yml" ]]; then jobs_json="$(fetch_child_jobs | jq -s '{jobs: [.[] | {name, conclusion, url: .html_url}]}')" @@ -983,7 +989,8 @@ jobs: needs: [resolve_target, evidence_reuse, prepare_release_candidate] if: ${{ always() && needs.resolve_target.result == 'success' && (needs.prepare_release_candidate.result == 'success' || needs.prepare_release_candidate.result == 'skipped') && contains(fromJSON('["all","plugin-prerelease"]'), inputs.rerun_group) && needs.evidence_reuse.outputs.reuse != 'true' }} runs-on: ubuntu-24.04 - timeout-minutes: ${{ inputs.release_profile == 'full' && 300 || inputs.release_profile == 'stable' && 240 || 60 }} + # The child owns lane timeouts; this monitor also covers queue delay. + timeout-minutes: ${{ inputs.release_profile == 'full' && 300 || 240 }} outputs: run_id: ${{ steps.dispatch.outputs.run_id }} url: ${{ steps.dispatch.outputs.url }} @@ -1006,10 +1013,10 @@ jobs: name: Run release/live/Docker/QA validation needs: [resolve_target, evidence_reuse, prepare_release_candidate] if: ${{ always() && needs.resolve_target.result == 'success' && (needs.prepare_release_candidate.result == 'success' || needs.prepare_release_candidate.result == 'skipped') && contains(fromJSON('["all","release-checks","install-smoke","cross-os","live-e2e","package","qa","qa-parity","qa-live"]'), inputs.rerun_group) && needs.evidence_reuse.outputs.reuse != 'true' }} - runs-on: ubuntu-24.04 - # The child owns lane timeouts; this monitor must also tolerate queue delay - # so it does not cancel healthy release checks before their final verifier. - timeout-minutes: 240 + runs-on: blacksmith-4vcpu-ubuntu-2404 + # The bounded package critical path tops out at 310 minutes; 420 leaves + # queue/API margin. Parent timeout preserves the adopted child for exact cancellation. + timeout-minutes: 420 outputs: run_id: ${{ steps.dispatch.outputs.run_id }} url: ${{ steps.dispatch.outputs.url }} @@ -1047,7 +1054,8 @@ jobs: if: ${{ always() && needs.resolve_target.result == 'success' && contains(fromJSON('["all","npm-telegram"]'), inputs.rerun_group) && (inputs.npm_telegram_package_spec != '' || inputs.release_package_spec != '') && needs.evidence_reuse.outputs.reuse != 'true' }} continue-on-error: ${{ startsWith(github.ref, 'refs/heads/tideclaw/alpha/') }} runs-on: ubuntu-24.04 - timeout-minutes: ${{ inputs.release_profile == 'full' && 360 || 60 }} + # The child owns lane timeouts; this monitor also covers queue delay. + timeout-minutes: ${{ inputs.release_profile == 'full' && 360 || 120 }} outputs: run_id: ${{ steps.dispatch.outputs.run_id }} url: ${{ steps.dispatch.outputs.url }} @@ -1074,7 +1082,9 @@ jobs: # Keep this monitor off the four-slot GitHub-hosted pool so performance starts # with the other child workflows instead of extending the critical path. runs-on: blacksmith-4vcpu-ubuntu-2404 - timeout-minutes: ${{ inputs.release_profile == 'full' && 360 || 120 }} + # Artifact-only and publish paths top out at 255 and 280 minutes; 360 leaves + # queue/API margin. Parent timeout preserves the adopted child for exact cancellation. + timeout-minutes: 360 outputs: run_id: ${{ steps.dispatch.outputs.run_id }} url: ${{ steps.dispatch.outputs.url }} diff --git a/.github/workflows/install-smoke-reusable.yml b/.github/workflows/install-smoke-reusable.yml index 1fb64ecac643..891295f20fdc 100644 --- a/.github/workflows/install-smoke-reusable.yml +++ b/.github/workflows/install-smoke-reusable.yml @@ -38,6 +38,7 @@ env: jobs: preflight: runs-on: ubuntu-24.04 + timeout-minutes: 15 outputs: docs_only: ${{ steps.manifest.outputs.docs_only }} run_install_smoke: ${{ steps.manifest.outputs.run_install_smoke }} @@ -118,6 +119,7 @@ jobs: needs: [preflight] if: needs.preflight.outputs.run_fast_install_smoke == 'true' && needs.preflight.outputs.run_full_install_smoke != 'true' runs-on: ubuntu-24.04 + timeout-minutes: 120 env: DOCKER_BUILD_SUMMARY: "false" DOCKER_BUILD_RECORD_UPLOAD: "false" @@ -228,6 +230,7 @@ jobs: needs: [preflight] if: needs.preflight.outputs.run_full_install_smoke == 'true' runs-on: ubuntu-24.04 + timeout-minutes: 60 permissions: contents: read packages: read @@ -332,6 +335,7 @@ jobs: needs: [preflight, root_dockerfile_image] if: always() && needs.preflight.result == 'success' && needs.preflight.outputs.run_full_install_smoke == 'true' runs-on: ubuntu-24.04 + timeout-minutes: 5 permissions: contents: read steps: @@ -349,6 +353,7 @@ jobs: needs: [preflight] if: needs.preflight.outputs.run_full_install_smoke == 'true' runs-on: ubuntu-24.04 + timeout-minutes: 30 steps: - name: Checkout CLI uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 @@ -365,6 +370,7 @@ jobs: needs: [preflight, root_dockerfile_image, root_dockerfile_image_ready] if: needs.preflight.outputs.run_full_install_smoke == 'true' runs-on: ubuntu-24.04 + timeout-minutes: 90 env: OPENCLAW_DOCKER_E2E_REQUIRE_LOCAL_IMAGE: "1" steps: @@ -534,6 +540,7 @@ jobs: needs: [preflight, root_dockerfile_image, root_dockerfile_image_ready] if: needs.preflight.outputs.run_full_install_smoke == 'true' runs-on: ubuntu-24.04 + timeout-minutes: 150 env: DOCKER_BUILD_SUMMARY: "false" DOCKER_BUILD_RECORD_UPLOAD: "false" @@ -700,6 +707,7 @@ jobs: needs: [preflight, root_dockerfile_image, root_dockerfile_image_ready] if: needs.preflight.outputs.run_full_install_smoke == 'true' && needs.preflight.outputs.run_bun_global_install_smoke == 'true' runs-on: ubuntu-24.04 + timeout-minutes: 60 env: OPENCLAW_DOCKER_E2E_REQUIRE_LOCAL_IMAGE: "1" steps: diff --git a/.github/workflows/openclaw-cross-os-release-checks-reusable.yml b/.github/workflows/openclaw-cross-os-release-checks-reusable.yml index 8705d788ba33..71fc746328da 100644 --- a/.github/workflows/openclaw-cross-os-release-checks-reusable.yml +++ b/.github/workflows/openclaw-cross-os-release-checks-reusable.yml @@ -249,6 +249,7 @@ env: jobs: prepare: runs-on: ubuntu-24.04 + timeout-minutes: 90 continue-on-error: ${{ inputs.advisory }} outputs: baseline_artifact_digest: ${{ steps.upload_baseline.outputs.artifact-digest }} diff --git a/.github/workflows/openclaw-live-and-e2e-checks-reusable.yml b/.github/workflows/openclaw-live-and-e2e-checks-reusable.yml index 36313bfcb293..eea162bffeb1 100644 --- a/.github/workflows/openclaw-live-and-e2e-checks-reusable.yml +++ b/.github/workflows/openclaw-live-and-e2e-checks-reusable.yml @@ -542,6 +542,7 @@ env: jobs: validate_selected_ref: runs-on: ubuntu-24.04 + timeout-minutes: 30 outputs: selected_sha: ${{ steps.validate.outputs.selected_sha }} trusted_reason: ${{ steps.validate.outputs.trusted_reason }} @@ -776,6 +777,7 @@ jobs: validate_live_suite_filter: runs-on: ubuntu-24.04 if: inputs.live_suite_filter != '' + timeout-minutes: 10 steps: - name: Validate focused live suite filter shell: bash @@ -892,6 +894,7 @@ jobs: plan_release_workflow_matrices: needs: validate_selected_ref runs-on: ubuntu-24.04 + timeout-minutes: 10 outputs: docker_e2e_count: ${{ steps.plan.outputs.docker_e2e_count }} docker_e2e_matrix: ${{ steps.plan.outputs.docker_e2e_matrix }} diff --git a/.github/workflows/openclaw-release-checks.yml b/.github/workflows/openclaw-release-checks.yml index 99a095229a3c..baa577ccad5e 100644 --- a/.github/workflows/openclaw-release-checks.yml +++ b/.github/workflows/openclaw-release-checks.yml @@ -143,6 +143,7 @@ jobs: mode: ${{ steps.inputs.outputs.mode }} release_profile: ${{ steps.inputs.outputs.release_profile }} run_release_soak: ${{ steps.inputs.outputs.run_release_soak }} + qa_live_scheduled: ${{ steps.inputs.outputs.qa_live_scheduled }} fail_fast: ${{ steps.inputs.outputs.fail_fast }} run_maturity_scorecard: ${{ steps.inputs.outputs.run_maturity_scorecard }} allow_unreleased_changelog: ${{ steps.inputs.outputs.allow_unreleased_changelog }} @@ -430,10 +431,10 @@ jobs: codex_plugin_spec="npm:@openclaw/codex@${BASH_REMATCH[1]}" fi + qa_filter_seen=false filter="$(printf '%s' "$RELEASE_LIVE_SUITE_FILTER_INPUT" | tr '[:upper:]' '[:lower:]')" repo_live_suite_filter="$filter" if [[ -n "${filter// }" ]]; then - qa_filter_seen=false repo_filter_tokens=() matrix_selected=false buzz_selected=false @@ -525,6 +526,12 @@ jobs: fi fi + qa_live_scheduled=false + if [[ "$RELEASE_RERUN_GROUP_INPUT" == "qa" || "$RELEASE_RERUN_GROUP_INPUT" == "qa-live" ]] || + [[ "$RELEASE_RERUN_GROUP_INPUT" == "all" && ( "$run_release_soak" == "true" || "$qa_filter_seen" == "true" ) ]]; then + qa_live_scheduled=true + fi + cross_os_scheduled=false if [[ "$RELEASE_RERUN_GROUP_INPUT" == "all" || "$RELEASE_RERUN_GROUP_INPUT" == "cross-os" ]]; then cross_os_scheduled=true @@ -542,6 +549,7 @@ jobs: printf 'mode=%s\n' "$RELEASE_MODE_INPUT" printf 'release_profile=%s\n' "$release_profile" printf 'run_release_soak=%s\n' "$run_release_soak" + printf 'qa_live_scheduled=%s\n' "$qa_live_scheduled" printf 'fail_fast=%s\n' "$fail_fast" printf 'run_maturity_scorecard=%s\n' "$run_maturity_scorecard" printf 'allow_unreleased_changelog=%s\n' "$allow_unreleased_changelog" @@ -572,6 +580,7 @@ jobs: RELEASE_MODE: ${{ inputs.mode }} RELEASE_PROFILE: ${{ steps.inputs.outputs.release_profile }} RUN_RELEASE_SOAK: ${{ steps.inputs.outputs.run_release_soak }} + QA_LIVE_SCHEDULED: ${{ steps.inputs.outputs.qa_live_scheduled }} FAIL_FAST: ${{ steps.inputs.outputs.fail_fast }} RUN_MATURITY_SCORECARD: ${{ steps.inputs.outputs.run_maturity_scorecard }} SKIP_PACKAGE_TELEGRAM_E2E: ${{ steps.inputs.outputs.skip_package_telegram_e2e }} @@ -593,6 +602,7 @@ jobs: echo "- Cross-OS mode: \`${RELEASE_MODE}\`" echo "- Release profile: \`${RELEASE_PROFILE}\`" echo "- Release soak lanes: \`${RUN_RELEASE_SOAK}\`" + echo "- QA-live scheduled: \`${QA_LIVE_SCHEDULED}\`" echo "- Matrix QA fail fast: \`${FAIL_FAST}\`" echo "- Maturity scorecard docs: \`${RUN_MATURITY_SCORECARD}\`" echo "- Package Acceptance Telegram E2E deferred: \`${SKIP_PACKAGE_TELEGRAM_E2E}\`" @@ -604,7 +614,7 @@ jobs: if [[ -n "${RELEASE_CROSS_OS_SUITE_FILTER// }" ]]; then echo "- Cross-OS suite filter: \`${RELEASE_CROSS_OS_SUITE_FILTER}\`" fi - echo "- QA live lanes: Matrix \`${{ steps.inputs.outputs.qa_live_matrix_enabled }}\`, Buzz \`${{ steps.inputs.outputs.qa_live_buzz_enabled }}\`, Telegram \`${{ steps.inputs.outputs.qa_live_telegram_enabled }}\`, Discord \`${{ steps.inputs.outputs.qa_live_discord_enabled }}\`, WhatsApp \`${{ steps.inputs.outputs.qa_live_whatsapp_enabled }}\`, Slack \`${{ steps.inputs.outputs.qa_live_slack_enabled }}\`" + echo "- QA-live lane eligibility: Matrix \`${{ steps.inputs.outputs.qa_live_matrix_enabled }}\`, Buzz \`${{ steps.inputs.outputs.qa_live_buzz_enabled }}\`, Telegram \`${{ steps.inputs.outputs.qa_live_telegram_enabled }}\`, Discord \`${{ steps.inputs.outputs.qa_live_discord_enabled }}\`, WhatsApp \`${{ steps.inputs.outputs.qa_live_whatsapp_enabled }}\`, Slack \`${{ steps.inputs.outputs.qa_live_slack_enabled }}\`" if [[ -n "${RELEASE_PACKAGE_SPEC// }" ]]; then echo "- Release package spec: \`${RELEASE_PACKAGE_SPEC}\`" fi @@ -1916,7 +1926,7 @@ jobs: qa_live_release_checks: name: Run QA Live Matrix catalog needs: [resolve_target] - if: contains(fromJSON('["all","qa","qa-live"]'), needs.resolve_target.outputs.rerun_group) && needs.resolve_target.outputs.qa_live_matrix_enabled == 'true' + if: needs.resolve_target.outputs.qa_live_scheduled == 'true' && needs.resolve_target.outputs.qa_live_matrix_enabled == 'true' permissions: contents: read pull-requests: read @@ -1934,7 +1944,7 @@ jobs: qa_live_buzz_release_checks: name: Run QA Lab live Buzz lane needs: [resolve_target] - if: contains(fromJSON('["all","qa","qa-live"]'), needs.resolve_target.outputs.rerun_group) && needs.resolve_target.outputs.qa_live_buzz_enabled == 'true' + if: needs.resolve_target.outputs.qa_live_scheduled == 'true' && needs.resolve_target.outputs.qa_live_buzz_enabled == 'true' permissions: contents: read pull-requests: read @@ -1954,7 +1964,7 @@ jobs: qa_live_telegram_release_checks: name: Run QA Lab live Telegram lane needs: [resolve_target] - if: contains(fromJSON('["all","qa","qa-live"]'), needs.resolve_target.outputs.rerun_group) && needs.resolve_target.outputs.qa_live_telegram_enabled == 'true' + if: needs.resolve_target.outputs.qa_live_scheduled == 'true' && needs.resolve_target.outputs.qa_live_telegram_enabled == 'true' runs-on: ubuntu-24.04 timeout-minutes: 210 permissions: @@ -2079,7 +2089,7 @@ jobs: qa_live_discord_release_checks: name: Run QA Lab live Discord lane needs: [resolve_target] - if: contains(fromJSON('["all","qa","qa-live"]'), needs.resolve_target.outputs.rerun_group) && needs.resolve_target.outputs.qa_live_discord_enabled == 'true' + if: needs.resolve_target.outputs.qa_live_scheduled == 'true' && needs.resolve_target.outputs.qa_live_discord_enabled == 'true' continue-on-error: true runs-on: ubuntu-24.04 timeout-minutes: 60 @@ -2194,7 +2204,7 @@ jobs: qa_live_whatsapp_release_checks: name: Run QA Lab live WhatsApp lane needs: [resolve_target] - if: contains(fromJSON('["all","qa","qa-live"]'), needs.resolve_target.outputs.rerun_group) && needs.resolve_target.outputs.qa_live_whatsapp_enabled == 'true' + if: needs.resolve_target.outputs.qa_live_scheduled == 'true' && needs.resolve_target.outputs.qa_live_whatsapp_enabled == 'true' continue-on-error: true runs-on: ubuntu-24.04 timeout-minutes: 60 @@ -2293,7 +2303,7 @@ jobs: qa_live_slack_release_checks: name: Run QA Lab live Slack lane needs: [resolve_target] - if: contains(fromJSON('["all","qa","qa-live"]'), needs.resolve_target.outputs.rerun_group) && needs.resolve_target.outputs.qa_live_slack_enabled == 'true' + if: needs.resolve_target.outputs.qa_live_scheduled == 'true' && needs.resolve_target.outputs.qa_live_slack_enabled == 'true' continue-on-error: true runs-on: ubuntu-24.04 timeout-minutes: 60 @@ -2528,7 +2538,7 @@ jobs: QA_LIVE_RELEASE_CHECKS_RESULT: ${{ needs.qa_live_release_checks.result }} QA_LIVE_BUZZ_RELEASE_CHECKS_RESULT: ${{ needs.qa_live_buzz_release_checks.result }} QA_LIVE_TELEGRAM_RELEASE_CHECKS_RESULT: ${{ needs.qa_live_telegram_release_checks.result }} - QA_LIVE_TELEGRAM_SELECTED: ${{ contains(fromJSON('["all","qa","qa-live"]'), needs.resolve_target.outputs.rerun_group) && needs.resolve_target.outputs.qa_live_telegram_enabled == 'true' }} + QA_LIVE_TELEGRAM_SELECTED: ${{ needs.resolve_target.outputs.qa_live_scheduled == 'true' && needs.resolve_target.outputs.qa_live_telegram_enabled == 'true' }} QA_LIVE_DISCORD_RELEASE_CHECKS_RESULT: ${{ needs.qa_live_discord_release_checks.result }} QA_LIVE_WHATSAPP_RELEASE_CHECKS_RESULT: ${{ needs.qa_live_whatsapp_release_checks.result }} QA_LIVE_SLACK_RELEASE_CHECKS_RESULT: ${{ needs.qa_live_slack_release_checks.result }} diff --git a/.github/workflows/qa-live-transports-convex.yml b/.github/workflows/qa-live-transports-convex.yml index 5cb1256cab0b..63b85e853c11 100644 --- a/.github/workflows/qa-live-transports-convex.yml +++ b/.github/workflows/qa-live-transports-convex.yml @@ -120,6 +120,7 @@ jobs: authorize_actor: name: Authorize workflow actor runs-on: blacksmith-8vcpu-ubuntu-2404 + timeout-minutes: 10 outputs: authorized: ${{ steps.permission.outputs.authorized }} steps: @@ -158,6 +159,7 @@ jobs: needs: authorize_actor if: needs.authorize_actor.outputs.authorized == 'true' runs-on: blacksmith-8vcpu-ubuntu-2404 + timeout-minutes: 30 outputs: selected_revision: ${{ steps.validate.outputs.selected_revision }} trusted_reason: ${{ steps.validate.outputs.trusted_reason }} diff --git a/CHANGELOG.md b/CHANGELOG.md index a890ec050630..87bf4c7b5494 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,7 @@ Docs: https://docs.openclaw.ai ### Changes +- **Release validation:** defer beta candidate Parallels smoke to postpublish `release:beta-smoke` by default, keep stable/full prepublish coverage, and bound nested release workflow monitors with explicit job timeouts. - **macOS app profiles:** isolate named app instances across state, preferences, Keychain, Gateway services, and duplicate-instance ownership while keeping host-global login and node services untouched. - **Developer workflow:** remove the obsolete scoped-commit helper and use standard Git commands in isolated worktrees. - **Plugin uninstall cleanup:** remove exact recorded install paths from `plugins.load.paths` for marketplace, npm, and other managed installs while preserving parent, child, prefix, and unrelated paths. diff --git a/docs/ci.md b/docs/ci.md index 13b57eb59be4..fce2b21038a2 100644 --- a/docs/ci.md +++ b/docs/ci.md @@ -164,7 +164,10 @@ PR max-lines checks derive the baseline from the checked-out synthetic merge tre ```bash gh workflow run ci.yml --ref release/YYYY.M.PATCH gh workflow run ci.yml --ref main -f target_ref= -f include_android=true -gh workflow run full-release-validation.yml --ref main -f ref= +VALIDATION_SHA="" +gh workflow run full-release-validation.yml --ref main \ + -f ref="$VALIDATION_SHA" \ + -f expected_sha="$VALIDATION_SHA" ``` Gateway extended-stable runs npm preflight, Full Release Validation, and plugin @@ -287,7 +290,16 @@ Every lane uploads its complete GitHub artifact, including CPU, heap, trace, and ## Full Release Validation -`Full Release Validation` is the manual umbrella workflow for "run everything before release." It accepts a branch, tag, or full commit SHA, dispatches the manual `CI` workflow with that target (including Android), dispatches `Plugin Prerelease` for release-only plugin/package/static/Docker proof, dispatches `OpenClaw Performance` against the target SHA, and dispatches `OpenClaw Release Checks` for install smoke, package acceptance, cross-OS package checks, QA Lab parity, Matrix, Telegram, and gated Discord, WhatsApp, and Slack lanes (advisory maturity scorecard rendering is opt-in via `run_maturity_scorecard`). Stable and full profiles always include exhaustive live/E2E and Docker release-path soak coverage; the beta profile can opt in with `run_release_soak=true`. The canonical package Telegram E2E runs inside Package Acceptance, so a full candidate does not start a duplicate live poller. After publishing, pass `release_package_spec` to reuse the shipped npm package across release checks, Package Acceptance, Docker, cross-OS, and Telegram without rebuilding. Use `npm_telegram_package_spec` only for a focused published-package Telegram rerun. The Codex plugin live package lane uses the same selected state by default: published `release_package_spec=openclaw@` derives `codex_plugin_spec=npm:@openclaw/codex@`, while SHA/artifact runs pack `extensions/codex` from the selected ref. Set `codex_plugin_spec` explicitly for custom plugin sources such as `npm:`, `npm-pack:`, or `git:` specs. Its live agent proof sends visible progress, continues through randomized workspace reads and an exact artifact write, then sends completion. +`Full Release Validation` is the manual release umbrella. Every run binds an +exact Validation SHA + Tooling SHA tuple and rejects an `expected_sha` mismatch +before child dispatch. Validation SHA maps to the Code SHA for product +validation or the Release SHA for changelog-only validation; it is not a third +release identity. Beta-publish maps to `release_profile=beta` with +`run_release_soak=false`; its `all` run includes normal CI, Plugin Prerelease, +package/install/cross-OS checks, performance, and QA parity, but excludes broad +live/E2E and QA-live. Postpublish-confidence uses the exact published package +with soak or explicit focused groups. Stable-publish maps to +`release_profile=stable`. See [Full release validation](/reference/full-release-validation) for the stage matrix, exact workflow job names, profile differences, artifacts, and @@ -331,13 +343,10 @@ pnpm ci:full-release --sha ``` GitHub workflow dispatch refs must be branches or tags, not raw commit SHAs. The -helper pushes a temporary `release-ci/-...` branch at a trusted `main` -workflow SHA, passes the requested target SHA through the workflow `ref` input, -reuses strict exact-target evidence when available, verifies every child -workflow `headSha` matches the trusted workflow SHA, and deletes the temporary -branch when the run completes. Pass `-f reuse_evidence=false` to force fresh -validation. The umbrella verifier also fails if any child workflow ran at a -different workflow SHA. +helper pushes a temporary `release-ci/-...` branch at a trusted Tooling +SHA, passes the requested Validation SHA through `ref` and `expected_sha`, reuses +strict exact-target evidence when available, and verifies every child workflow +`headSha` matches the Tooling SHA. `release_profile` controls live/provider breadth passed into release checks. The manual release workflows default to `stable`; use `full` only when you @@ -355,17 +364,22 @@ CLI's own first-scenario cancellation. - `stable` adds the stable provider/backend set. - `full` runs the broad advisory provider/media matrix. -The umbrella records the dispatched child run ids, and the final `Verify full validation` job re-checks current child run conclusions and appends slowest-job tables for each child run. If a child workflow is rerun and turns green, rerun only the parent verifier job to refresh the umbrella result and timing summary. +The umbrella records dispatched child run ids, and `Verify full validation` +checks them during that parent attempt. Parent cancellation or timeout leaves +adopted exact children running; cancel one explicitly when it is no longer +needed. -For recovery, both `Full Release Validation` and `OpenClaw Release Checks` accept `rerun_group`. Use `all` for a release candidate, `ci` for only the normal full CI child, `plugin-prerelease` for only the plugin prerelease child, `performance` for only the OpenClaw Performance child, `release-checks` for every release child, or a narrower group: `install-smoke`, `cross-os`, `live-e2e`, `package`, `qa`, `qa-parity`, `qa-live`, or `npm-telegram` on the umbrella. This keeps a failed release box rerun bounded after a focused fix. For one failed cross-OS lane, combine `rerun_group=cross-os` with `cross_os_suite_filter`, for example `windows/packaged-upgrade`; long cross-OS commands emit heartbeat lines and packaged-upgrade summaries include per-phase timings. Selected Matrix and Telegram QA lanes block normal release validation, as does the core runtime-pair tool coverage gate. QA parity, runtime parity, and the gated Discord, WhatsApp, and Slack live lanes are advisory. +For recovery, classify product, harness/tooling/provenance, +infrastructure/credential, and wrapper failures before editing. Only confirmed +product failure changes the Code SHA. Use one diagnosis, one fix when needed, +and one narrow `rerun_group` retry, then reassess; never widen automatically to +`all`. Narrow evidence is not publish authorization by itself. `OpenClaw Release Checks` uses the trusted workflow ref to resolve the selected ref once into a `release-package-under-test` tarball, then passes that artifact to cross-OS checks and Package Acceptance, plus the live/E2E release-path Docker workflow when soak coverage runs. That keeps the package bytes consistent across release boxes and avoids repacking the same candidate in multiple child jobs. For the Codex npm-plugin live lane, release checks either pass a matching published plugin spec derived from `release_package_spec`, pass the operator-supplied `codex_plugin_spec`, or leave the input blank so the Docker script packs the selected checkout's Codex plugin. -Duplicate `Full Release Validation` runs for `ref=main` and `rerun_group=all` -supersede the older umbrella. The parent monitor cancels any child workflow it -has already dispatched when the parent is cancelled, so newer main validation -does not sit behind a stale two-hour release-check run. Release branch/tag -validation and focused rerun groups keep `cancel-in-progress: false`. +Full Release Validation concurrency is keyed by Validation SHA, Tooling SHA, +and rerun group with `cancel-in-progress: false`. Parent cancellation does not +cancel adopted children. ## Live and E2E shards diff --git a/docs/help/testing.md b/docs/help/testing.md index 639f969d22e5..97d2c204c7e3 100644 --- a/docs/help/testing.md +++ b/docs/help/testing.md @@ -168,17 +168,17 @@ These commands sit beside the main test suites when you need QA-lab realism. CI runs QA Lab in dedicated workflows. Agentic parity is nested under `QA-Lab - All Lanes` and release validation, not a standalone PR workflow. Broad validation should use `Full Release Validation` with -`rerun_group=qa-parity` or the release-checks QA group. Stable/default release -checks keep exhaustive live/Docker soak behind `run_release_soak=true`; the -`full` profile forces soak on. `QA-Lab - All Lanes` runs nightly on `main` and -from manual dispatch with the mock parity lane, live Matrix lane, -Convex-managed live Telegram lane, and Convex-managed live Discord lane as -parallel jobs. Scheduled QA and release checks run the catalog-derived Matrix -selection through the shared live adapter. `OpenClaw Release Checks` runs parity plus the -reusable Matrix live-adapter lane and Telegram lane before release approval. Release +`rerun_group=qa-parity` or the release-checks QA group. Stable/full, +soak-enabled, and explicit `qa`/`qa-live` release checks include the QA-live +Matrix and Telegram lanes. Bounded beta-publish `all` without soak runs parity +but defers those live lanes to postpublish-confidence. `QA-Lab - All Lanes` runs +nightly on `main` and from manual dispatch with the mock parity lane, live +Matrix lane, Convex-managed live Telegram lane, and Convex-managed live Discord +lane as parallel jobs. Scheduled QA and selected release checks run the +catalog-derived Matrix selection through the shared live adapter. Release transport checks use `mock-openai/gpt-5.6-luna` so they stay deterministic and -avoid normal provider-plugin startup. These live transport gateways -disable memory search; memory behavior stays covered by the QA parity suites. +avoid normal provider-plugin startup. These live transport gateways disable +memory search; memory behavior stays covered by the QA parity suites. Full release live media shards use `ghcr.io/openclaw/openclaw-live-media-runner:ubuntu-24.04`, which already has diff --git a/docs/reference/RELEASING.md b/docs/reference/RELEASING.md index e43c74a90eef..95146ac147c2 100644 --- a/docs/reference/RELEASING.md +++ b/docs/reference/RELEASING.md @@ -86,6 +86,7 @@ gh workflow run openclaw-npm-release.yml \ gh workflow run full-release-validation.yml \ --ref extended-stable/YYYY.M.33 \ -f ref=extended-stable/YYYY.M.33 \ + -f expected_sha="$RELEASE_SHA" \ -f release_profile=stable ``` @@ -229,8 +230,8 @@ This checklist is the public shape of the release flow. Private credentials, sig 1. Start from current `main`: pull latest, confirm the target commit is pushed, and confirm `main` CI is green enough to branch from. 2. Create `release/YYYY.M.PATCH` from that commit. Backports are optional; apply only the operator-selected set. Bump every required version location, run `pnpm release:prep`, finish release fixes and required forward-ports, and review `src/plugins/compat/registry.ts` plus `src/commands/doctor/shared/deprecation-compat.ts`. -3. Freeze the product-complete pre-changelog commit as the **Code SHA**. Run the deterministic source preflight, then use `node scripts/full-release-validation-at-sha.mjs --sha --target-ref release/YYYY.M.PATCH`. This pins trusted workflow tooling while the full Vitest, Docker, QA, package, and performance matrix targets the exact Code SHA. -4. Classify failures before editing. A product/code failure creates a new Code SHA and requires green full validation for that SHA. A workflow, harness, credential, approval, or infrastructure failure is repaired in its owning surface and rerun against the same Code SHA. +3. Freeze the product-complete pre-changelog commit as the **Code SHA** and record the trusted **Tooling SHA**. Run the deterministic source preflight, then use `node scripts/full-release-validation-at-sha.mjs --sha --target-ref release/YYYY.M.PATCH`. Beta-publish uses `release_profile=beta` without soak; postpublish-confidence owns broad live, QA-live, mobile, and Parallels work. +4. Classify failures before editing as product, harness/tooling/provenance, infrastructure/credential, or wrapper. Only confirmed product failure creates a new Code SHA. Use one diagnosis, one fix when needed, and one narrow retry, then reassess. 5. Only after the Code SHA is green, generate the top `CHANGELOG.md` section from merged PRs and direct commits since the last reachable shipped tag. Keep entries user-facing and deduplicated. When a divergent shipped tag or later forward-port re-associates already-released PRs, pass it explicitly as `--shipped-ref`. 6. Commit only `CHANGELOG.md`. This commit is the **Release SHA**. The complete diff from Code SHA to Release SHA must be exactly `CHANGELOG.md`; any other changed path returns the release to step 2. 7. Run SHA-pinned Full Release Validation for the Release SHA with evidence reuse enabled. The lightweight parent must record `changelog-only-release-v1`, point at the green Code SHA, and dispatch no product child lanes. This reuses product evidence; it does not reuse package bytes. @@ -246,7 +247,7 @@ This checklist is the public shape of the release flow. Private credentials, sig --skip-dispatch ``` - For stable, also pass `--windows-node-tag vX.Y.Z`. The helper verifies release-note provenance, npm preflight bytes, Parallels install/update proof, Telegram package proof, and plugin publish plans, then prints the publish command. After it completes green, create and push the final signed tag at that same Release SHA, then run the printed publish command. + For stable, also pass `--windows-node-tag vX.Y.Z`. Beta and alpha candidates defer Parallels install/update proof to the postpublish `pnpm release:beta-smoke` roster by default; pass `--run-parallels` only when the operator explicitly wants that proof before publish. Stable and full candidates run Parallels by default. The helper still verifies release-note provenance, npm preflight bytes, Telegram package proof, and plugin publish plans, then prints the publish command. After it completes green, create and push the final signed tag at that same Release SHA, then run the printed publish command. `pnpm release:candidate` validates the current frozen branch tip by default (or the explicit `--target-sha`), and rejects a tag that already exists. It records evidence before the final signed tag is pushed. @@ -328,7 +329,7 @@ A legacy fallback correction tag may reuse base-package evidence only when the c Manually dispatch `Windows Node Release` only for recovery, and always pass an exact tag, never `latest`, plus the explicit `expected_installer_digests` JSON map from the approved source release. Website download links should target exact OpenClaw release asset URLs for the current stable release, or `releases/latest/download/...` only after verifying GitHub's latest redirect points at that same release; do not link only to the companion repo release page. -- Release checks now run in a separate manual workflow: `OpenClaw Release Checks`. It also runs the QA Lab mock parity lane plus the Matrix catalog and Telegram QA lane before release approval. The live lanes use the `qa-live-shared` environment; Telegram also uses Convex CI credential leases. +- Release checks now run in a separate manual workflow: `OpenClaw Release Checks`. It always runs the QA Lab mock parity lane. The Matrix catalog and Telegram QA-live lanes run for stable/full validation, soak-enabled validation, or an explicit `qa`/`qa-live` rerun group. Bounded beta-publish `all` without soak defers those live lanes to postpublish-confidence. The live lanes use the `qa-live-shared` environment; Telegram also uses Convex CI credential leases. - Cross-OS install and upgrade runtime validation is part of public `OpenClaw Release Checks` and `Full Release Validation`, which call the reusable workflow `.github/workflows/openclaw-cross-os-release-checks-reusable.yml` directly. This split is intentional: keep the real npm release path short, deterministic, and artifact-focused, while slower live checks stay in their own lane so they do not stall or block publish. - Secret-bearing release checks should be dispatched through `Full Release Validation` or from the `main`/release workflow ref so workflow logic and secrets stay controlled. - `OpenClaw Release Checks` accepts a branch, tag, or full commit SHA as long as the resolved commit is reachable from an OpenClaw branch or release tag. @@ -367,7 +368,7 @@ pnpm ci:full-release \ --target-ref release/YYYY.M.PATCH ``` -The helper fetches current `origin/main`, pushes `release-ci/-...` at that trusted workflow commit, infers `beta` from alpha/beta package versions and `stable` otherwise, dispatches `Full Release Validation` from the temporary branch with `ref=`, verifies every child workflow `headSha` matches the pinned parent workflow SHA, then deletes the temporary branch. Pass `-f reuse_evidence=false` to force a fresh run, `-f release_profile=full` for the broad advisory sweep, or `--workflow-sha ` to pin an older commit that is still reachable from current `origin/main`. The workflow itself never writes repository refs. This keeps main-only release tooling available without adding tooling commits to the candidate and avoids proving a newer `main` child run by accident. +The helper fetches current `origin/main`, pushes `release-ci/-...` at that trusted Tooling SHA, infers `beta` from alpha/beta package versions and `stable` otherwise, and dispatches `Full Release Validation` with the Validation SHA as `expected_sha`. Target resolution rejects a mismatch before child dispatch. Every child workflow `headSha` must match the Tooling SHA. Pass `-f reuse_evidence=false` to force a fresh run, `-f release_profile=full` for the broad advisory sweep, or `--workflow-sha ` to pin a compatible older commit still reachable from current `origin/main`. The helper rejects pinned tooling that lacks the `expected_sha` dispatch input and never silently selects a newer Tooling SHA. The workflow itself never writes repository refs. After the Code SHA is green, commit only `CHANGELOG.md` and run the same helper with the Release SHA: @@ -379,7 +380,7 @@ pnpm ci:full-release \ The second parent reuses product evidence only when GitHub proves the Release SHA descends from the Code SHA and the complete changed path set is exactly `CHANGELOG.md`. It records `changelog-only-release-v1` and dispatches no product children. Npm preflight and package/install acceptance still run on the Release SHA because its tarball bytes changed. -For a fresh Code SHA, the workflow resolves the target, dispatches manual `CI`, then dispatches `OpenClaw Release Checks`. `OpenClaw Release Checks` fans out install smoke, cross-OS release checks, live/E2E Docker release-path coverage when soak is enabled, Package Acceptance with the canonical Telegram package E2E, QA Lab parity, live Matrix, and live Telegram. A full/all run is only acceptable when the `Full Release Validation` summary shows `normal_ci`, `plugin_prerelease`, and `release_checks` as successful, unless a focused rerun intentionally skipped the separate `Plugin Prerelease` child. Use the standalone `npm-telegram` child only for a focused published-package rerun with `release_package_spec` or `npm_telegram_package_spec`. The final verifier summary includes slowest-job tables for each child run, so the release manager can see the current critical path without downloading logs. +For a fresh Code SHA, the workflow resolves the target, dispatches manual `CI`, then dispatches `OpenClaw Release Checks`. Beta-publish maps to `release_profile=beta` and `run_release_soak=false`; its `all` run excludes broad live/E2E and QA-live lanes. Postpublish-confidence uses the exact published package with soak or explicit focused groups. Stable-publish maps to `release_profile=stable`. The final verifier summary includes slowest-job tables for each child run. The product-performance child is artifact-only in this release path. The umbrella dispatches it with `publish_reports=false`, and validation is rejected @@ -421,10 +422,11 @@ pnpm ci:full-release \ --target-ref release/YYYY.M.PATCH \ -f release_package_spec=openclaw@YYYY.M.PATCH-beta.N \ -f evidence_package_spec=openclaw@YYYY.M.PATCH-beta.N \ + -f run_release_soak=true \ -f npm_telegram_provider_mode=mock-openai ``` -Do not use the full umbrella as the first rerun after a focused fix. If one box fails, use the failed child workflow, job, Docker lane, package profile, model provider, or QA lane for the next proof. Run the full umbrella again only when the fix changed shared release orchestration or made earlier all-box evidence stale. The umbrella's final verifier re-checks the recorded child workflow run ids, so after a child workflow is rerun successfully, rerun only the failed `Verify full validation` parent job. +Do not use the full umbrella as the first rerun after a focused fix. Classify the failure as product, harness/tooling/provenance, infrastructure/credential, or wrapper. Only confirmed product failure changes the Code SHA. Use one diagnosis, one fix when needed, and one narrow retry, then reassess. A narrow green run is evidence, not publish authorization by itself; there is no standalone parent finalizer. `rerun_group=all` may reuse a prior green umbrella run when the release profile, effective soak setting, and validation inputs match and either the target SHA @@ -435,9 +437,10 @@ is exactly `CHANGELOG.md`. Exact-target reuse records preflight, package bytes, release-note provenance, and install/update acceptance must still run against the Release SHA. Any version, source, generated, dependency, package, or workflow-owned target change requires a new Code SHA -and fresh full validation. Newer umbrella runs for the same `release/*` ref and -rerun group supersede in-progress ones automatically. Pass -`reuse_evidence=false` to force a fresh full run. +and fresh full validation. Concurrency is keyed by Validation SHA, Tooling SHA, +and rerun group and does not cancel prior runs. Parent cancellation leaves +adopted children running until the operator cancels the exact child. Pass +`reuse_evidence=false` only when a fresh full run is intentionally required. For bounded recovery, pass `rerun_group` to the umbrella. `all` is the real release-candidate run, `ci` runs only the normal CI child, `plugin-prerelease` runs only the release-only plugin child, `release-checks` runs every release box, and the narrower release groups are `install-smoke`, `cross-os`, `live-e2e`, `package`, `qa`, `qa-parity`, `qa-live`, and `npm-telegram`. Focused `npm-telegram` reruns require `release_package_spec` or `npm_telegram_package_spec`; full/all runs use the canonical package Telegram E2E inside Package Acceptance. Focused cross-OS reruns can add `cross_os_suite_filter=windows/packaged-upgrade` or another OS/suite filter. QA release-check failures block normal release validation, including OpenClaw dynamic tool drift in the core runtime-pair lane. Tideclaw alpha runs may still treat non-package-safety release-check lanes as advisory. With `release_profile=beta`, the `Run repo/live E2E validation` live-provider suites are advisory (warnings, not blockers); stable and full profiles keep them blocking. When `live_suite_filter` explicitly requests a gated QA live lane such as Discord, WhatsApp, or Slack, the matching `OPENCLAW_RELEASE_QA_*_LIVE_CI_ENABLED` repo variable must be enabled; otherwise input capture fails instead of silently skipping the lane. diff --git a/docs/reference/full-release-validation.md b/docs/reference/full-release-validation.md index 1979375eb5e8..98571a3061ee 100644 --- a/docs/reference/full-release-validation.md +++ b/docs/reference/full-release-validation.md @@ -14,7 +14,8 @@ whole release. Run release preparation before freezing the Code SHA; it refreshes Control UI locale output when the background bot has not landed it yet, then enforces the same strict zero-fallback check used by release CI. -Freeze the product-complete pre-changelog commit as the **Code SHA**, then run: +Freeze the product-complete pre-changelog commit as the **Code SHA** and select +one trusted workflow commit as the **Tooling SHA**, then run: ```bash pnpm ci:full-release \ @@ -30,13 +31,18 @@ package versions and `stable` otherwise. Pass alternate workflow inputs with independent failures together. Pass `-f fail_fast=true` when the shorter first-failure cancellation path is preferable. -The helper creates a temporary `release-ci/*` ref pinned to one trusted -`origin/main` workflow SHA, passes the target SHA only as the candidate `ref`, -and deletes the temporary ref after validation. Every dispatched child must -report that same workflow SHA. Pass +The helper creates a temporary `release-ci/*` ref pinned to the Tooling SHA, +passes the Validation SHA as both the candidate ref and `expected_sha`, and +deletes the temporary ref after validation. The Validation SHA equals the Code +SHA for product validation or the Release SHA for changelog-only validation; it +is not a third release identity. The workflow rejects malformed or mismatched +expected SHAs before child dispatch. Every child must report the same Tooling +SHA. Pass `-f reuse_evidence=false` to force a fresh run or -`--workflow-sha ` to select an older workflow commit still -reachable from current `origin/main`. The workflow never creates or updates +`--workflow-sha ` to select a compatible older workflow +commit still reachable from current `origin/main`. The helper rejects a pinned +Tooling SHA that does not declare the `expected_sha` dispatch input; it never +silently substitutes newer tooling. The workflow never creates or updates repository refs itself. ## Extended-stable exception @@ -45,9 +51,11 @@ Extended-stable publish requires a run whose workflow and target are both the canonical branch: ```bash +RELEASE_SHA="$(git rev-parse HEAD)" gh workflow run full-release-validation.yml \ --ref extended-stable/YYYY.M.33 \ -f ref=extended-stable/YYYY.M.33 \ + -f expected_sha="$RELEASE_SHA" \ -f release_profile=stable ``` @@ -67,10 +75,16 @@ SHA descends from the Code SHA and the complete changed path set is exactly `CHANGELOG.md`; npm preflight and package/install acceptance still run on the Release SHA. -`release_profile=stable` and `release_profile=full` always run the exhaustive -live/Docker soak. Pass `run_release_soak=true` to include the same soak lanes -with the `beta` profile. Stable publication rejects a validation manifest -without this soak and blocking product-performance evidence. +The conceptual phases map to current inputs: + +- `beta-publish`: `release_profile=beta`, `run_release_soak=false` +- `postpublish-confidence`: exact published package plus + `run_release_soak=true` or explicit focused groups +- `stable-publish`: `release_profile=stable` + +Beta-publish `all` excludes broad live/E2E soak and QA-live lanes. Stable and +full always run the soak. Stable publication rejects a validation manifest +without soak and blocking product-performance evidence. Package Acceptance normally builds the candidate tarball from the resolved `ref`, including full-SHA runs dispatched with `pnpm ci:full-release`. After a @@ -122,17 +136,17 @@ the `runtime-assets` Docker target with other stages and is enforced by the umbrella verifier; lanes no longer wait for it before dispatching. A narrower `rerun_group` skips this preflight. -| Stage | Details | -| ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Target resolution | **Job:** `Resolve target ref`
**Child workflow:** none
**Proves:** resolves the release branch, tag, or full commit SHA and records selected inputs.
**Rerun:** rerun the umbrella if this fails. | -| Shared candidate | **Job:** `Prepare shared release candidate`
**Child workflow:** `OpenClaw Live And E2E Checks (Reusable)`
**Proves:** packs and validates one exact-SHA package, builds one functional Docker image, and records immutable package and image artifact tuples for both package-facing child workflows.
**Rerun:** rerun the affected package, plugin-prerelease, cross-OS, or live/E2E group. | -| Docker assets preflight | **Job:** `Verify Docker runtime image assets`
**Child workflow:** none
**Proves:** the `runtime-assets` Docker build target still succeeds before any other stage dispatches. Runs only for `rerun_group=all`.
**Rerun:** rerun the umbrella with `rerun_group=all`. | -| Vitest and normal CI | **Job:** `Run normal full CI`
**Child workflow:** `CI`
**Proves:** manual full CI graph against the target ref, including Linux Node lanes, bundled plugin shards, plugin and channel contract shards, Node 22 compatibility, `check-*`, `check-additional-*`, built-artifact smoke checks, docs checks, Python skills, Windows, macOS, Control UI i18n, and Android via the umbrella.
**Rerun:** `rerun_group=ci`. | -| Plugin prerelease | **Job:** `Run plugin prerelease validation`
**Child workflow:** `Plugin Prerelease`
**Proves:** release-only plugin static checks, agentic plugin coverage, full plugin batch shards, plugin prerelease Docker lanes, and a non-blocking `plugin-inspector-advisory` artifact for compatibility triage.
**Rerun:** `rerun_group=plugin-prerelease`. | -| Release checks | **Job:** `Run release/live/Docker/QA validation`
**Child workflow:** `OpenClaw Release Checks`
**Proves:** install smoke, cross-OS package checks, Package Acceptance, QA Lab parity, live Matrix, Buzz, and Telegram, plus gated advisory Discord, WhatsApp, and Slack lanes. Stable and full profiles also run exhaustive live/E2E suites and Docker release-path chunks; beta can opt in with `run_release_soak=true`.
**Rerun:** `rerun_group=release-checks` or a narrower release-checks handle. | -| Package Telegram | **Job:** `Run package Telegram E2E`
**Child workflow:** `NPM Telegram Beta E2E`
**Proves:** a focused published-package Telegram E2E when `release_package_spec` or `npm_telegram_package_spec` is set. Full candidate validation uses the canonical Package Acceptance Telegram E2E instead.
**Rerun:** `rerun_group=npm-telegram` with `release_package_spec` or `npm_telegram_package_spec`. | -| Product performance | **Job:** `Run product performance evidence`
**Child workflow:** `OpenClaw Performance`
**Proves:** release-profile performance run (`profile=release`, `repeat=3`, `fail_on_regression=true`, `publish_reports=false`) against the target SHA. Kova output stays in workflow artifacts and the child must prove its report publisher was skipped. Required (blocking) only for `rerun_group=all` or `rerun_group=performance`; not required for narrower rerun groups.
**Rerun:** `rerun_group=performance`. | -| Umbrella verifier | **Job:** `Verify full validation`
**Child workflow:** none
**Proves:** re-checks recorded child run conclusions and appends slowest-job tables from child workflows.
**Rerun:** rerun only this job after rerunning a failed child to green. | +| Stage | Details | +| ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Target resolution | **Job:** `Resolve target ref`
**Child workflow:** none
**Proves:** resolves the release branch, tag, or full commit SHA and records selected inputs.
**Rerun:** rerun the umbrella if this fails. | +| Shared candidate | **Job:** `Prepare shared release candidate`
**Child workflow:** `OpenClaw Live And E2E Checks (Reusable)`
**Proves:** packs and validates one exact-SHA package, builds one functional Docker image, and records immutable package and image artifact tuples for both package-facing child workflows.
**Rerun:** rerun the affected package, plugin-prerelease, cross-OS, or live/E2E group. | +| Docker assets preflight | **Job:** `Verify Docker runtime image assets`
**Child workflow:** none
**Proves:** the `runtime-assets` Docker build target still succeeds before any other stage dispatches. Runs only for `rerun_group=all`.
**Rerun:** rerun the umbrella with `rerun_group=all`. | +| Vitest and normal CI | **Job:** `Run normal full CI`
**Child workflow:** `CI`
**Proves:** manual full CI graph against the target ref, including Linux Node lanes, bundled plugin shards, plugin and channel contract shards, Node 22 compatibility, `check-*`, `check-additional-*`, built-artifact smoke checks, docs checks, Python skills, Windows, macOS, Control UI i18n, and Android via the umbrella.
**Rerun:** `rerun_group=ci`. | +| Plugin prerelease | **Job:** `Run plugin prerelease validation`
**Child workflow:** `Plugin Prerelease`
**Proves:** release-only plugin static checks, agentic plugin coverage, full plugin batch shards, plugin prerelease Docker lanes, and a non-blocking `plugin-inspector-advisory` artifact for compatibility triage.
**Rerun:** `rerun_group=plugin-prerelease`. | +| Release checks | **Job:** `Run release/live/Docker/QA validation`
**Child workflow:** `OpenClaw Release Checks`
**Proves:** install smoke, cross-OS package checks, Package Acceptance, and QA Lab parity. QA-live Matrix, Buzz, and Telegram plus gated advisory Discord, WhatsApp, and Slack run for stable/full, beta with `run_release_soak=true`, or explicit `qa`/`qa-live` groups. Stable and full profiles also run exhaustive live/E2E suites and Docker release-path chunks.
**Rerun:** `rerun_group=release-checks` or a narrower release-checks handle. | +| Package Telegram | **Job:** `Run package Telegram E2E`
**Child workflow:** `NPM Telegram Beta E2E`
**Proves:** a focused published-package Telegram E2E when `release_package_spec` or `npm_telegram_package_spec` is set. Full candidate validation uses the canonical Package Acceptance Telegram E2E instead.
**Rerun:** `rerun_group=npm-telegram` with `release_package_spec` or `npm_telegram_package_spec`. | +| Product performance | **Job:** `Run product performance evidence`
**Child workflow:** `OpenClaw Performance`
**Proves:** release-profile performance run (`profile=release`, `repeat=3`, `fail_on_regression=true`, `publish_reports=false`) against the target SHA. Kova output stays in workflow artifacts and the child must prove its report publisher was skipped. Required (blocking) only for `rerun_group=all` or `rerun_group=performance`; not required for narrower rerun groups.
**Rerun:** `rerun_group=performance`. | +| Umbrella verifier | **Job:** `Verify full validation`
**Child workflow:** none
**Proves:** re-checks recorded child run conclusions and appends slowest-job tables from child workflows.
**Rerun:** rerun only this job after rerunning a failed child to green. | The umbrella always dispatches product performance in artifact-only mode. `OpenClaw Performance` permits report publication only for scheduled runs or a @@ -153,11 +167,10 @@ publish consumers. The verifier always prefers the attempt-qualified artifact; as a transition, it accepts the stable name only for an attempt-1 manifest v2 producer. It rejects that legacy name for later attempts and manifest v3. -For `ref=main` with `rerun_group=all`, for `release/*` refs, and for Tideclaw -alpha refs, a newer umbrella run supersedes an older one with the same ref and -rerun group. When the parent is cancelled, its monitor cancels any child -workflow it already dispatched. Tag and pinned-SHA validation runs do not -cancel each other. +Concurrency is keyed by Validation SHA, Tooling SHA, and rerun group and does +not cancel an older run. Parent cancellation or timeout leaves an adopted +identity-checked child running. Cancel that exact child explicitly when it is +no longer useful. ## Release checks stages @@ -168,7 +181,7 @@ artifact when package or Docker-facing stages need it. | Stage | Details | | ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Release target | **Job:** `Resolve target ref`
**Backing workflow:** none
**Tests:** selected ref, optional expected SHA, profile, rerun group, and focused live suite filter.
**Rerun:** `rerun_group=release-checks`. | +| Release target | **Job:** `Resolve target ref`
**Backing workflow:** none
**Tests:** selected ref, optional expected Validation SHA, profile, rerun group, and focused live suite filter.
**Rerun:** `rerun_group=release-checks`. | | Package artifact | **Job:** `Prepare release package artifact`
**Backing workflow:** none
**Tests:** validates the umbrella's immutable package tuple, or packs one candidate tarball for a direct/focused Release Checks dispatch, then exposes it to downstream package-facing checks.
**Rerun:** the affected package, cross-OS, or live/E2E group. | | Install smoke | **Job:** `Run install smoke`
**Backing workflow:** `Install Smoke`
**Tests:** full install path with root Dockerfile smoke image reuse, QR package install, root and gateway Docker smokes, installer Docker tests, and Bun global install image-provider smoke.
**Rerun:** `rerun_group=install-smoke`. | | Cross-OS | **Job:** `cross_os_release_checks`
**Backing workflow:** `OpenClaw Cross-OS Release Checks (Reusable)`
**Tests:** fresh and upgrade lanes on Linux, Windows, and macOS for the selected provider and mode, using the candidate tarball plus a baseline package.
**Rerun:** `rerun_group=cross-os`. | @@ -211,11 +224,11 @@ commands with package artifact and image reuse inputs when available. `release_profile` mostly controls live/provider breadth inside release checks. It does not remove normal full CI, Plugin Prerelease, install smoke, package -acceptance, or QA Lab. Stable and full profiles always run exhaustive repo/live -E2E and Docker release-path soak coverage. The beta profile can opt in with -`run_release_soak=true`. Package Acceptance supplies the canonical package -Telegram E2E for every full candidate, so the umbrella does not duplicate that -live poller. +acceptance, or QA parity. Stable and full profiles always run exhaustive +repo/live E2E, Docker release-path, and QA-live soak coverage. The beta profile +adds those lanes only with `run_release_soak=true` or an explicit `qa` or +`qa-live` rerun. Package Acceptance supplies the canonical package Telegram +E2E for every candidate, so the umbrella does not duplicate that live poller. | Profile | Intended use | Included live/provider coverage | | -------- | --------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -247,7 +260,7 @@ Use `rerun_group` to avoid repeating unrelated release boxes: | Handle | Scope | | ------------------- | ----------------------------------------------------------------------------------------------- | -| `all` | All Full Release Validation stages. | +| `all` | Phase-default stages; beta without soak excludes broad live/E2E and QA-live. | | `ci` | Manual full CI child only. | | `plugin-prerelease` | Plugin Prerelease child only. | | `release-checks` | All OpenClaw Release Checks stages. | @@ -301,8 +314,11 @@ need fresh QA evidence. ## Evidence to keep Keep the `Full Release Validation` summary as the release-level index. It links -child run ids and includes slowest-job tables. For failures, inspect the child -workflow first, then rerun the smallest matching handle above. +child run ids and includes slowest-job tables. Classify failures as product, +harness/tooling/provenance, infrastructure/credential, or wrapper. Only a +confirmed product failure changes the Code SHA. Use one diagnosis, one fix when +needed, and one narrow retry, then reassess; do not automatically rerun `all`. +Narrow evidence is not publish authorization by itself. For a regular release, record both Code SHA and Release SHA, the reuse policy and changed-path set, the green Code SHA parent run, and the lightweight Release diff --git a/scripts/full-release-validation-at-sha.mts b/scripts/full-release-validation-at-sha.mts index 2d737ff88596..76556af604a0 100644 --- a/scripts/full-release-validation-at-sha.mts +++ b/scripts/full-release-validation-at-sha.mts @@ -9,6 +9,7 @@ import { existsSync, mkdtempSync, rmSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { pathToFileURL } from "node:url"; +import { parse as parseYaml } from "yaml"; import { execGhRead } from "./lib/plain-gh.mjs"; const WORKFLOW = "full-release-validation.yml"; @@ -18,6 +19,8 @@ const RELEASE_EVIDENCE_VERIFIER_PATHS = [ ".agents/skills/release-openclaw-ci/scripts/release-ci-summary.mjs", ]; const GH_READ_TIMEOUT_MS = 60_000; +export const FULL_RELEASE_WAIT_TIMEOUT_MINUTES = 720; +export const FULL_RELEASE_WAIT_POLL_INTERVAL_MS = 45_000; const GH_READ_OPTIONS = { encoding: "utf8", killSignal: "SIGKILL", @@ -67,9 +70,9 @@ function displayValue(value: unknown): string { function usage() { console.error(`Usage: node scripts/full-release-validation-at-sha.mjs [--sha ] [--target-ref ] [--workflow-sha ] [--keep-branch] [--dry-run] [-- -f key=value ...] -Creates temporary remote branches pinned to trusted main release tooling and -the exact target commit, dispatches Full Release Validation with the target -branch as its ref input, +Creates temporary remote branches pinned to the exact Tooling SHA and Validation SHA, +dispatches Full Release Validation with the Validation SHA branch as its ref input +and expected_sha as its immutable identity, watches the parent run, verifies all child workflow head SHAs match the trusted workflow lineage through the release evidence manifest, then deletes both temporary branches by default. --keep-branch retains both branches. Exact-target and changelog-only Release SHA @@ -213,6 +216,9 @@ export function parseArgs(argv: string[]) { if (Object.hasOwn(args.inputs, "ref")) { throw new Error("SHA-pinned release validation reserves the ref input for --sha"); } + if (Object.hasOwn(args.inputs, "expected_sha")) { + throw new Error("SHA-pinned release validation reserves expected_sha for the resolved --sha"); + } if ( args.targetRef && !RELEASE_BRANCH_PATTERN.test(args.targetRef) && @@ -375,7 +381,8 @@ function readWorkflowRun(parentRunId: string, workflowSha: string) { function waitForWorkflowRun(parentRunId: string, workflowSha: string) { let lastSummary = ""; let consecutiveErrors = 0; - for (let attempt = 0; attempt < 480; attempt += 1) { + const deadline = Date.now() + FULL_RELEASE_WAIT_TIMEOUT_MINUTES * 60_000; + while (Date.now() < deadline) { let suite: Record | undefined; try { suite = readWorkflowRun(parentRunId, workflowSha); @@ -404,10 +411,19 @@ function waitForWorkflowRun(parentRunId: string, workflowSha: string) { `Full Release Validation concluded ${stringValue(suite.conclusion, "unknown").toLowerCase()}: https://github.com/openclaw/openclaw/actions/runs/${parentRunId}`, ); } - Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, 45_000); + const remainingMs = deadline - Date.now(); + if (remainingMs <= 0) { + break; + } + Atomics.wait( + new Int32Array(new SharedArrayBuffer(4)), + 0, + 0, + Math.min(FULL_RELEASE_WAIT_POLL_INTERVAL_MS, remainingMs), + ); } throw new Error( - `Timed out waiting for Full Release Validation: https://github.com/openclaw/openclaw/actions/runs/${parentRunId}`, + `Timed out after ${FULL_RELEASE_WAIT_TIMEOUT_MINUTES} minutes waiting for Full Release Validation: https://github.com/openclaw/openclaw/actions/runs/${parentRunId}`, ); } @@ -431,12 +447,33 @@ export function assertTrustedWorkflowHarness( runStatus("git", ["cat-file", "-e", `${workflowSha}:${relativePath}`], { stdio: ["ignore", "ignore", "ignore"], }).status === 0, + readPath: (relativePath: string) => string = (relativePath) => + run("git", ["show", `${workflowSha}:${relativePath}`]), ) { if (!pathExists(TRUSTED_WORKFLOW_PATH)) { throw new Error( `trusted workflow SHA ${workflowSha} does not contain ${TRUSTED_WORKFLOW_PATH}`, ); } + let workflow: unknown; + try { + workflow = parseYaml(readPath(TRUSTED_WORKFLOW_PATH)); + } catch (error) { + throw new Error( + `Tooling SHA ${workflowSha} contains invalid ${TRUSTED_WORKFLOW_PATH}: ${error instanceof Error ? error.message : String(error)}`, + ); + } + if ( + !isRecord(workflow) || + !isRecord(workflow.on) || + !isRecord(workflow.on.workflow_dispatch) || + !isRecord(workflow.on.workflow_dispatch.inputs) || + !Object.hasOwn(workflow.on.workflow_dispatch.inputs, "expected_sha") + ) { + throw new Error( + `Tooling SHA ${workflowSha} is missing workflow_dispatch input expected_sha in ${TRUSTED_WORKFLOW_PATH}`, + ); + } const verifierPath = RELEASE_EVIDENCE_VERIFIER_PATHS.find((relativePath) => pathExists(relativePath), ); @@ -503,12 +540,13 @@ function main() { const remoteTargetBranchRef = `refs/heads/${targetBranch}`; const dispatchInputs = { ref: targetBranch, + expected_sha: targetSha, ...(targetContextRef !== targetSha ? { target_context_ref: targetContextRef } : {}), ...args.inputs, }; - console.log(`Target SHA: ${targetSha}`); - console.log(`Trusted workflow SHA: ${workflowSha}`); + console.log(`Validation SHA: ${targetSha}`); + console.log(`Tooling SHA: ${workflowSha}`); console.log(`Temporary target ref: ${targetBranch}`); console.log(`Temporary workflow ref: ${branch}`); diff --git a/scripts/release-candidate-checklist.mts b/scripts/release-candidate-checklist.mts index 4f23433dc7f2..bb8771358b26 100644 --- a/scripts/release-candidate-checklist.mts +++ b/scripts/release-candidate-checklist.mts @@ -145,7 +145,8 @@ Options: --windows-node-tag Exact Windows Node release tag. Required for stable. --skip-dispatch Require both run ids; do not dispatch workflows. --skip-local-generated-check Do not run local generated release baseline checks before dispatch. - --skip-parallels Do not run local Parallels fresh/update candidate smoke. + --run-parallels Force candidate Parallels smoke; beta defaults to postpublish release:beta-smoke. + --skip-parallels Force-skip candidate Parallels smoke; stable/full run by default. --parallels-registry-package-artifact Add a verified plugin npm preflight artifact directory. Repeatable. --skip-telegram Do not run NPM Telegram E2E against the prepared tarball. @@ -189,7 +190,10 @@ export function parseArgs(argv: string[]) { >(), skipDispatch: false, skipLocalGeneratedCheck: false, + runParallels: false, skipParallels: false, + parallelsMode: "auto" as "auto" | "run" | "skip", + parallelsSkipReason: "", skipTelegram: false, telegramProviderMode: DEFAULT_TELEGRAM_PROVIDER_MODE, tag: "", @@ -234,6 +238,7 @@ export function parseArgs(argv: string[]) { ), booleanFlag("--skip-dispatch", "skipDispatch"), booleanFlag("--skip-local-generated-check", "skipLocalGeneratedCheck"), + booleanFlag("--run-parallels", "runParallels"), booleanFlag("--skip-parallels", "skipParallels"), booleanFlag("--skip-telegram", "skipTelegram"), ], @@ -270,6 +275,18 @@ export function parseArgs(argv: string[]) { if (!["beta", "stable", "full"].includes(options.releaseProfile)) { throw new Error("--release-profile must be beta, stable, or full"); } + if (options.runParallels && options.skipParallels) { + throw new Error("--run-parallels and --skip-parallels cannot be combined"); + } + options.parallelsMode = options.runParallels ? "run" : options.skipParallels ? "skip" : "auto"; + options.skipParallels = + options.parallelsMode === "skip" || + (options.parallelsMode === "auto" && options.releaseProfile === "beta"); + options.parallelsSkipReason = options.skipParallels + ? options.parallelsMode === "auto" + ? "deferred to postpublish release:beta-smoke" + : "operator skipped --skip-parallels" + : ""; if (options.skipDispatch && (!options.fullReleaseRunId || !options.npmPreflightRunId)) { throw new Error("--skip-dispatch requires --full-release-run and --npm-preflight-run"); } @@ -1597,7 +1614,7 @@ async function runParallelsIfNeeded( registryPackageTarballPaths: string[], ): Promise { if (options.skipParallels) { - return { status: "skipped", reason: "operator skipped --skip-parallels" }; + return { status: "skipped", reason: options.parallelsSkipReason }; } // This function runs inside trusted tooling, not the frozen target checkout. // Prepare its isolated dependencies here before importing the Parallels harness. diff --git a/test/scripts/full-release-validation-at-sha.test.ts b/test/scripts/full-release-validation-at-sha.test.ts index 7364b8e08c86..a386fb3c0216 100644 --- a/test/scripts/full-release-validation-at-sha.test.ts +++ b/test/scripts/full-release-validation-at-sha.test.ts @@ -5,6 +5,8 @@ import { join, resolve } from "node:path"; import { describe, expect, it } from "vitest"; import { assertTrustedWorkflowHarness, + FULL_RELEASE_WAIT_POLL_INTERVAL_MS, + FULL_RELEASE_WAIT_TIMEOUT_MINUTES, parseArgs, releaseProfileForTarget, releaseEvidenceVerificationArgs, @@ -23,7 +25,7 @@ function runGit(cwd: string, args: string[]): string { }).trim(); } -function createDispatchFixture() { +function createDispatchFixture(options: { workflowSource?: string } = {}) { const root = mkdtempSync(join(tmpdir(), "openclaw-release-dispatch-")); const origin = join(root, "origin.git"); const checkout = join(root, "checkout"); @@ -45,7 +47,14 @@ function createDispatchFixture() { writeFileSync(join(checkout, "package.json"), '{"version":"2026.8.1"}\n'); writeFileSync( join(checkout, ".github", "workflows", "full-release-validation.yml"), - "name: Full Release Validation\n", + options.workflowSource ?? + `name: Full Release Validation +on: + workflow_dispatch: + inputs: + expected_sha: + required: false +`, ); writeFileSync( join(checkout, "scripts", "release-ci-summary.mjs"), @@ -262,9 +271,15 @@ describe("full-release-validation-at-sha", () => { ); }); - it("reserves the candidate ref for the resolved --sha", () => { + it("reserves immutable candidate identity inputs for the resolved --sha", () => { expect(() => parseArgs(["-f", "ref=other"])).toThrow("reserves the ref input"); expect(() => parseArgs(["--", "ref=other"])).toThrow("reserves the ref input"); + expect(() => parseArgs(["-f", `expected_sha=${"a".repeat(40)}`])).toThrow( + "reserves expected_sha", + ); + expect(() => parseArgs(["--", `expected_sha=${"a".repeat(40)}`])).toThrow( + "reserves expected_sha", + ); }); it("validates direct and reused runs through the strict evidence verifier", () => { @@ -280,9 +295,18 @@ describe("full-release-validation-at-sha", () => { it("polls the exact workflow run without GraphQL quota use", () => { const source = readFileSync("scripts/full-release-validation-at-sha.mts", "utf8"); + expect(FULL_RELEASE_WAIT_TIMEOUT_MINUTES).toBe(720); + expect(FULL_RELEASE_WAIT_POLL_INTERVAL_MS).toBe(45_000); expect(source).toContain("actions/runs/${parentRunId}"); expect(source).toContain("workflowRun.head_sha !== workflowSha"); expect(source).toContain("return suite;"); + expect(source).toContain("Date.now() + FULL_RELEASE_WAIT_TIMEOUT_MINUTES * 60_000"); + expect(source).toContain("const remainingMs = deadline - Date.now();"); + expect(source).toContain("Math.min(FULL_RELEASE_WAIT_POLL_INTERVAL_MS, remainingMs)"); + expect(source).toContain( + "Timed out after ${FULL_RELEASE_WAIT_TIMEOUT_MINUTES} minutes waiting for Full Release Validation", + ); + expect(source).not.toContain("attempt < 480"); expect(source).not.toContain('"graphql"'); expect(source).not.toContain('["run", "watch"'); }); @@ -299,16 +323,31 @@ describe("full-release-validation-at-sha", () => { const verifierPath = "scripts/release-ci-summary.mjs"; const checked: string[] = []; expect( - assertTrustedWorkflowHarness("a".repeat(40), (relativePath) => { - checked.push(relativePath); - return relativePath === workflowPath || relativePath === verifierPath; - }), + assertTrustedWorkflowHarness( + "a".repeat(40), + (relativePath) => { + checked.push(relativePath); + return relativePath === workflowPath || relativePath === verifierPath; + }, + () => "on:\n workflow_dispatch:\n inputs:\n expected_sha: {}\n", + ), ).toBe(verifierPath); expect(checked).toEqual([workflowPath, verifierPath]); expect(() => assertTrustedWorkflowHarness("a".repeat(40), () => false)).toThrow(workflowPath); expect(() => - assertTrustedWorkflowHarness("a".repeat(40), (relativePath) => relativePath === workflowPath), + assertTrustedWorkflowHarness( + "a".repeat(40), + (relativePath) => relativePath === workflowPath, + () => "on:\n workflow_dispatch:\n inputs:\n expected_sha: {}\n", + ), ).toThrow("supported release evidence verifier"); + expect(() => + assertTrustedWorkflowHarness( + "b".repeat(40), + () => true, + () => "on:\n workflow_dispatch:\n inputs: {}\n", + ), + ).toThrow(`Tooling SHA ${"b".repeat(40)} is missing workflow_dispatch input expected_sha`); const source = readFileSync("scripts/full-release-validation-at-sha.mts", "utf8"); expect(source.indexOf("assertTrustedWorkflowHarness(workflowSha);")).toBeLessThan( @@ -386,9 +425,13 @@ describe("full-release-validation-at-sha", () => { "-f", `ref=${targetBranch}`, "-f", + `expected_sha=${fixture.targetSha}`, + "-f", `target_context_ref=${fixture.releaseRef}`, ]), ); + expect(result.stdout).toContain(`Validation SHA: ${fixture.targetSha}`); + expect(result.stdout).toContain(`Tooling SHA: ${fixture.workflowSha}`); expect(result.stdout).toContain( "Parent run: https://github.com/openclaw/openclaw/actions/runs/123", ); @@ -409,6 +452,24 @@ describe("full-release-validation-at-sha", () => { } }); + it("rejects pinned old-schema tooling before either remote ref is pushed", () => { + const fixture = createDispatchFixture({ + workflowSource: "name: Full Release Validation\non:\n workflow_dispatch:\n", + }); + try { + const result = fixture.run(["--workflow-sha", fixture.workflowSha]); + expect(result.status).toBe(1); + expect(result.stderr).toContain(`Tooling SHA ${fixture.workflowSha}`); + expect(result.stderr).toContain("missing workflow_dispatch input expected_sha"); + expect(fixture.readCalls(fixture.gitCallsPath).filter((args) => args[0] === "push")).toEqual( + [], + ); + expect(readFileSync(fixture.ghCallsPath, "utf8")).toBe(""); + } finally { + fixture.cleanup(); + } + }); + it("keeps both temporary refs with --keep-branch", () => { const fixture = createDispatchFixture(); try { diff --git a/test/scripts/package-acceptance-workflow.test.ts b/test/scripts/package-acceptance-workflow.test.ts index b3ebe4749021..714cfeaf0b4a 100644 --- a/test/scripts/package-acceptance-workflow.test.ts +++ b/test/scripts/package-acceptance-workflow.test.ts @@ -11,10 +11,15 @@ import { import { resolve } from "node:path"; import { afterEach, describe, expect, it } from "vitest"; import { parse } from "yaml"; +import { FULL_RELEASE_WAIT_TIMEOUT_MINUTES } from "../../scripts/full-release-validation-at-sha.mts"; +import { createReleaseWorkflowMatrixPlan } from "../../scripts/plan-release-workflow-matrix.mjs"; import { useAutoCleanupTempDirTracker } from "../helpers/temp-dir.js"; const PACKAGE_ACCEPTANCE_WORKFLOW = ".github/workflows/package-acceptance.yml"; const LIVE_E2E_WORKFLOW = ".github/workflows/openclaw-live-and-e2e-checks-reusable.yml"; +const INSTALL_SMOKE_REUSABLE_WORKFLOW = ".github/workflows/install-smoke-reusable.yml"; +const CROSS_OS_RELEASE_CHECKS_REUSABLE_WORKFLOW = + ".github/workflows/openclaw-cross-os-release-checks-reusable.yml"; const LIVE_MEDIA_RUNNER_DOCKERFILE = ".github/images/live-media-runner/Dockerfile"; const LIVE_MEDIA_RUNNER_IMAGE = "ghcr.io/openclaw/openclaw-live-media-runner:ubuntu-24.04"; const LIVE_MEDIA_RUNNER_IMAGE_WORKFLOW = ".github/workflows/live-media-runner-image.yml"; @@ -43,6 +48,8 @@ const ANDROID_RELEASE_WORKFLOW = ".github/workflows/android-release.yml"; const STABLE_MAIN_CLOSEOUT_WORKFLOW = ".github/workflows/openclaw-stable-main-closeout.yml"; const WINDOWS_NODE_RELEASE_WORKFLOW = ".github/workflows/windows-node-release.yml"; const FULL_RELEASE_VALIDATION_WORKFLOW = ".github/workflows/full-release-validation.yml"; +const CI_WORKFLOW = ".github/workflows/ci.yml"; +const PERFORMANCE_WORKFLOW = ".github/workflows/openclaw-performance.yml"; const FULL_RELEASE_CHILD_DISPATCHES = [ { jobName: "normal_ci", @@ -124,10 +131,12 @@ type WorkflowStep = { type WorkflowMatrixEntry = { advisory?: boolean; + chunk_id?: string; command?: string; profiles?: string; suite_group?: string; suite_id?: string; + timeout_minutes?: number; }; type WorkflowJob = { @@ -199,6 +208,113 @@ function workflowStep(job: WorkflowJob, stepName: string): WorkflowStep { return step; } +function jobNeeds(job: WorkflowJob): string[] { + return Array.isArray(job.needs) ? job.needs : job.needs ? [job.needs] : []; +} + +function timeoutForProfile( + timeout: number | string | undefined, + profile: "beta" | "stable" | "full", +): number { + if (typeof timeout === "number") { + return timeout; + } + const match = timeout?.match( + /^\$\{\{ inputs\.(?:release_profile|release_test_profile) == 'full' && ([0-9]+) \|\| ([0-9]+) \}\}$/u, + ); + if (!match) { + throw new Error(`Unsupported release timeout expression: ${String(timeout)}`); + } + return Number(profile === "full" ? match[1] : match[2]); +} + +function evaluatedJobTimeouts(path: string, jobName: string, job: WorkflowJob): number[] { + const timeout = job["timeout-minutes"]; + if (typeof timeout === "number") { + return [timeout]; + } + if (timeout?.includes("inputs.release_")) { + return (["beta", "stable", "full"] as const).map((profile) => + timeoutForProfile(timeout, profile), + ); + } + if (timeout !== "${{ matrix.timeout_minutes }}") { + throw new Error(`Unsupported timeout for ${path}:${jobName}: ${String(timeout)}`); + } + + const matrix = (job.strategy as { matrix?: unknown } | undefined)?.matrix; + if (matrix && typeof matrix === "object" && "include" in matrix) { + const include = (matrix as { include?: WorkflowMatrixEntry[] }).include; + if (!Array.isArray(include) || include.length === 0) { + throw new Error(`Missing static timeout matrix for ${path}:${jobName}`); + } + return include.map((entry) => { + if (typeof entry.timeout_minutes !== "number") { + throw new Error(`Missing matrix timeout for ${path}:${jobName}`); + } + return entry.timeout_minutes; + }); + } + + if (path === LIVE_E2E_WORKFLOW && jobName === "validate_docker_e2e") { + return (["beta", "stable", "full"] as const).flatMap((releaseProfile) => + createReleaseWorkflowMatrixPlan({ + includeReleasePathSuites: true, + releaseProfile, + }).dockerE2e.matrix.include.map((entry: WorkflowMatrixEntry) => { + if (typeof entry.timeout_minutes !== "number") { + throw new Error(`Missing planned timeout for ${releaseProfile}:${entry.chunk_id}`); + } + return entry.timeout_minutes; + }), + ); + } + + throw new Error(`Missing matrix timeout evaluator for ${path}:${jobName}`); +} + +function pluginPrereleaseTimeoutFloor( + pluginPrerelease: Workflow, + liveE2e: Workflow, + profile: "beta" | "stable" | "full", +): number { + const preflight = pluginPrerelease.jobs?.preflight; + const dockerSuite = pluginPrerelease.jobs?.["plugin-prerelease-docker-suite"]; + const suite = pluginPrerelease.jobs?.["plugin-prerelease-suite"]; + const validateSelectedRef = liveE2e.jobs?.validate_selected_ref; + const prepareImage = liveE2e.jobs?.prepare_docker_e2e_image; + const imageReady = liveE2e.jobs?.docker_e2e_image_ready; + const dockerLanes = liveE2e.jobs?.validate_docker_lanes; + if ( + !preflight || + !dockerSuite || + !suite || + !validateSelectedRef || + !prepareImage || + !imageReady || + !dockerLanes + ) { + throw new Error("Missing plugin prerelease timeout-chain job"); + } + + expect(jobNeeds(dockerSuite)).toContain("preflight"); + expect(jobNeeds(prepareImage)).toEqual(["validate_selected_ref"]); + expect(jobNeeds(imageReady)).toEqual(["prepare_docker_e2e_image"]); + expect(jobNeeds(dockerLanes)).toEqual( + expect.arrayContaining(["prepare_docker_e2e_image", "docker_e2e_image_ready"]), + ); + expect(jobNeeds(suite)).toContain("plugin-prerelease-docker-suite"); + + return [ + timeoutForProfile(preflight["timeout-minutes"], profile), + timeoutForProfile(validateSelectedRef["timeout-minutes"], profile), + timeoutForProfile(prepareImage["timeout-minutes"], profile), + timeoutForProfile(imageReady["timeout-minutes"], profile), + timeoutForProfile(dockerLanes["timeout-minutes"], profile), + timeoutForProfile(suite["timeout-minutes"], profile), + ].reduce((total, value) => total + value, 0); +} + function runFullReleaseInputValidation(releaseProfile: string, skipTelegram: string) { const step = workflowStep( workflowJob(FULL_RELEASE_VALIDATION_WORKFLOW, "resolve_target"), @@ -214,7 +330,13 @@ function runFullReleaseInputValidation(releaseProfile: string, skipTelegram: str }); } -function runReleaseChecksInputValidation(releaseProfile: string, skipTelegram: string) { +function runReleaseChecksInputValidation( + releaseProfile: string, + skipTelegram: string, + rerunGroup = "all", + runReleaseSoak = "false", + liveSuiteFilter = "", +) { const step = workflowStep( workflowJob(RELEASE_CHECKS_WORKFLOW, "resolve_target"), "Capture selected inputs", @@ -230,6 +352,7 @@ function runReleaseChecksInputValidation(releaseProfile: string, skipTelegram: s GITHUB_OUTPUT: outputPath, PATH: process.env.PATH, RELEASE_FAIL_FAST_INPUT: "false", + RELEASE_LIVE_SUITE_FILTER_INPUT: liveSuiteFilter, RELEASE_MODE_INPUT: "both", RELEASE_PROFILE_INPUT: releaseProfile, RELEASE_PROVIDER_INPUT: "openai", @@ -237,9 +360,9 @@ function runReleaseChecksInputValidation(releaseProfile: string, skipTelegram: s RELEASE_QA_SLACK_LIVE_CI_ENABLED: "false", RELEASE_QA_WHATSAPP_LIVE_CI_ENABLED: "false", RELEASE_REF_INPUT: "main", - RELEASE_RERUN_GROUP_INPUT: "all", + RELEASE_RERUN_GROUP_INPUT: rerunGroup, RELEASE_RUN_MATURITY_SCORECARD_INPUT: "false", - RELEASE_RUN_RELEASE_SOAK_INPUT: "false", + RELEASE_RUN_RELEASE_SOAK_INPUT: runReleaseSoak, RELEASE_SKIP_PACKAGE_TELEGRAM_E2E_INPUT: skipTelegram, }, }); @@ -411,7 +534,16 @@ if (args[0] === "workflow" && args[1] === "run") { `, ); chmodSync(ghPath, 0o755); - writeFileSync(sleepPath, "#!/bin/sh\nexit 0\n"); + writeFileSync( + sleepPath, + `#!/bin/sh +if [ -n "\${MOCK_SLEEP_SIGNAL:-}" ] && [ ! -e "\${MOCK_SLEEP_SIGNAL_SENT}" ]; then + : > "\${MOCK_SLEEP_SIGNAL_SENT}" + kill -"\${MOCK_SLEEP_SIGNAL}" "$PPID" +fi +exit 0 +`, + ); chmodSync(sleepPath, 0o755); const parentSha = "a".repeat(40); @@ -491,6 +623,7 @@ if (args[0] === "workflow" && args[1] === "run") { MOCK_GH_STATUSES: '["completed"]', MOCK_GH_STATUS_POLLS: statusPath, MOCK_GH_WORKFLOW_ID: "789", + MOCK_SLEEP_SIGNAL_SENT: resolve(workdir, "sleep-signal-sent"), PATH: `${workdir}:${process.env.PATH}`, ...overrides, }, @@ -2144,9 +2277,21 @@ describe("package acceptance workflow", () => { expect(adoptionCall?.args).toContain( "[.workflow_runs[] | select(.display_title == env.DISPATCH_RUN_NAME and .head_branch == env.CHILD_WORKFLOW_REF) | .id]", ); + expect(calls.filter(({ args }) => args[0] === "run" && args[1] === "cancel")).toHaveLength(0); }, ); + it("leaves the adopted child running when the monitor receives SIGTERM", () => { + const { calls, result } = runFullReleaseChildDispatch(FULL_RELEASE_CHILD_DISPATCHES[0], { + MOCK_GH_STATUSES: '["in_progress"]', + MOCK_SLEEP_SIGNAL: "TERM", + }); + + expect(result.status, `${result.stdout}\n${result.stderr}`).toBeNull(); + expect(result.signal).toBe("SIGTERM"); + expect(calls.filter(({ args }) => args[0] === "run" && args[1] === "cancel")).toHaveLength(0); + }); + it.each(FULL_RELEASE_CHILD_DISPATCHES)( "refuses duplicate exact adoption candidates for $jobName", (child) => { @@ -2193,7 +2338,7 @@ describe("package acceptance workflow", () => { ); it.each(FULL_RELEASE_CHILD_DISPATCHES)( - "cancels exactly the adopted $jobName child when monitoring fails unexpectedly", + "leaves the adopted $jobName child running when monitoring fails unexpectedly", (child) => { const { calls, result } = runFullReleaseChildDispatch(child, { MOCK_GH_STATUS_ERROR: "HTTP 403: Resource not accessible by integration", @@ -2201,9 +2346,10 @@ describe("package acceptance workflow", () => { expect(result.status).toBe(1); expect(result.stderr).toContain("HTTP 403"); - expect(calls.filter(({ args }) => args[0] === "run" && args[1] === "cancel")).toEqual([ - expect.objectContaining({ args: ["run", "cancel", "101"] }), - ]); + expect( + calls.filter(({ args }) => args[0] === "run" && args[1] === "cancel"), + `${result.stdout}\n${result.stderr}\n${JSON.stringify(calls)}`, + ).toEqual([]); }, ); @@ -2238,7 +2384,7 @@ describe("package acceptance workflow", () => { }, ); - it("keeps CI fail-fast job lookups advisory but npm Telegram job lookups fail-closed", () => { + it("keeps CI fail-fast job lookups advisory and npm Telegram fail-closed without cancellation", () => { const overrides = { FAIL_FAST: "true", MOCK_GH_JOBS_ERROR: "HTTP 403: Resource not accessible by integration", @@ -2261,7 +2407,7 @@ describe("package acceptance workflow", () => { expect( npmTelegram.calls.filter(({ args }) => args[0] === "run" && args[1] === "cancel"), `${npmTelegram.result.stdout}\n${npmTelegram.result.stderr}\n${JSON.stringify(npmTelegram.calls)}`, - ).toHaveLength(1); + ).toEqual([]); }); it.each([ @@ -3514,6 +3660,78 @@ describe("package artifact reuse", () => { }, ); + it.each([ + ["beta", "all", "false", "false", "false"], + ["beta", "all", "true", "true", "true"], + ["stable", "all", "false", "true", "true"], + ["full", "all", "false", "true", "true"], + ["beta", "qa", "false", "false", "true"], + ["beta", "qa-live", "false", "false", "true"], + ])( + "normalizes QA-live scheduling for profile=%s group=%s soak=%s", + (releaseProfile, rerunGroup, runReleaseSoak, expectedSoak, expectedScheduled) => { + const { outputPath, result } = runReleaseChecksInputValidation( + releaseProfile, + "false", + rerunGroup, + runReleaseSoak, + ); + + expect(result.status, result.stderr).toBe(0); + const output = readFileSync(outputPath, "utf8"); + expect(output).toContain(`run_release_soak=${expectedSoak}\n`); + expect(output).toContain(`qa_live_scheduled=${expectedScheduled}\n`); + }, + ); + + it("schedules only the selected QA-live lane for an all-group QA filter", () => { + const { outputPath, result } = runReleaseChecksInputValidation( + "beta", + "false", + "all", + "false", + "qa-live-telegram", + ); + + expect(result.status, result.stderr).toBe(0); + const output = readFileSync(outputPath, "utf8"); + expect(output).toContain("qa_live_scheduled=true\n"); + expect(output).toContain("qa_live_telegram_enabled=true\n"); + for (const lane of ["matrix", "buzz", "discord", "whatsapp", "slack"]) { + expect(output).toContain(`qa_live_${lane}_enabled=false\n`); + } + }); + + it("does not schedule QA-live for an all-group repo live filter without soak", () => { + const { outputPath, result } = runReleaseChecksInputValidation( + "beta", + "false", + "all", + "false", + "repo-e2e", + ); + + expect(result.status, result.stderr).toBe(0); + const output = readFileSync(outputPath, "utf8"); + expect(output).toContain("qa_live_scheduled=false\n"); + expect(output).toContain("repo_live_suite_filter=repo-e2e\n"); + }); + + it("does not let a QA-live filter override an unrelated rerun group", () => { + const { outputPath, result } = runReleaseChecksInputValidation( + "beta", + "false", + "install-smoke", + "false", + "qa-live-telegram", + ); + + expect(result.status, result.stderr).toBe(0); + const output = readFileSync(outputPath, "utf8"); + expect(output).toContain("qa_live_scheduled=false\n"); + expect(output).toContain("qa_live_telegram_enabled=true\n"); + }); + it("summarizes Telegram deferral only when Package Acceptance is scheduled", () => { const scheduled = runFullReleaseTargetSummary("release-checks", "true"); const unrelated = runFullReleaseTargetSummary("ci", "true"); @@ -3717,7 +3935,9 @@ describe("package artifact reuse", () => { OPENCLAW_QA_CONVEX_SITE_URL: "${{ secrets.OPENCLAW_QA_CONVEX_SITE_URL }}", }); expect(releaseJob.permissions).toEqual({ contents: "read", "pull-requests": "read" }); - expect(releaseJob.if).toContain('contains(fromJSON(\'["all","qa","qa-live"]\')'); + expect(releaseJob.if).toBe( + "needs.resolve_target.outputs.qa_live_scheduled == 'true' && needs.resolve_target.outputs.qa_live_matrix_enabled == 'true'", + ); expect(releaseJob.with).toMatchObject({ expected_sha: "${{ needs.resolve_target.outputs.revision }}", fail_fast: "${{ fromJSON(needs.resolve_target.outputs.fail_fast) }}", @@ -3789,8 +4009,9 @@ describe("package artifact reuse", () => { OPENCLAW_QA_CONVEX_SITE_URL: "${{ secrets.OPENCLAW_QA_CONVEX_SITE_URL }}", }); expect(releaseJob.permissions).toEqual({ contents: "read", "pull-requests": "read" }); - expect(releaseJob.if).toContain('contains(fromJSON(\'["all","qa","qa-live"]\')'); - expect(releaseJob.if).toContain("needs.resolve_target.outputs.qa_live_buzz_enabled == 'true'"); + expect(releaseJob.if).toBe( + "needs.resolve_target.outputs.qa_live_scheduled == 'true' && needs.resolve_target.outputs.qa_live_buzz_enabled == 'true'", + ); expect(releaseJob.with).toMatchObject({ buzz_scenario: "channel-canary,channel-mention-gating", expected_sha: "${{ needs.resolve_target.outputs.revision }}", @@ -3826,6 +4047,46 @@ describe("package artifact reuse", () => { expect(requireBuzz.run).toContain("exit 1"); }); + it("runs QA-live on soak or explicit QA groups, not beta all by default", () => { + const workflow = readFileSync(RELEASE_CHECKS_WORKFLOW, "utf8"); + const resolveTarget = workflowJob(RELEASE_CHECKS_WORKFLOW, "resolve_target"); + const liveJobs = [ + ["qa_live_release_checks", "qa_live_matrix_enabled"], + ["qa_live_buzz_release_checks", "qa_live_buzz_enabled"], + ["qa_live_telegram_release_checks", "qa_live_telegram_enabled"], + ["qa_live_discord_release_checks", "qa_live_discord_enabled"], + ["qa_live_whatsapp_release_checks", "qa_live_whatsapp_enabled"], + ["qa_live_slack_release_checks", "qa_live_slack_enabled"], + ] as const; + const selection = "needs.resolve_target.outputs.qa_live_scheduled == 'true'"; + + expect(resolveTarget.outputs?.qa_live_scheduled).toBe( + "${{ steps.inputs.outputs.qa_live_scheduled }}", + ); + + for (const [jobName, enabledOutput] of liveJobs) { + expect(workflowJob(RELEASE_CHECKS_WORKFLOW, jobName).if).toBe( + `${selection} && needs.resolve_target.outputs.${enabledOutput} == 'true'`, + ); + } + + const verifyStep = workflowStep( + workflowJob(RELEASE_CHECKS_WORKFLOW, "summary"), + "Verify release check results", + ); + expect(verifyStep.env?.QA_LIVE_TELEGRAM_SELECTED).toBe( + `\${{ ${selection} && needs.resolve_target.outputs.qa_live_telegram_enabled == 'true' }}`, + ); + const kickoffSummary = workflowStep(resolveTarget, "Summarize validated ref"); + expect(kickoffSummary.env?.QA_LIVE_SCHEDULED).toBe( + "${{ steps.inputs.outputs.qa_live_scheduled }}", + ); + expect(kickoffSummary.run).toContain("- QA-live scheduled:"); + expect(kickoffSummary.run).toContain("- QA-live lane eligibility:"); + expect(kickoffSummary.run).not.toContain("- QA live lanes:"); + expect(workflow).not.toContain('contains(fromJSON(\'["qa","qa-live"]\')'); + }); + it("runs live transport lanes nightly while release checks stay gated", () => { const releaseWorkflow = readFileSync(RELEASE_CHECKS_WORKFLOW, "utf8"); const qaWorkflow = readFileSync(QA_LIVE_TRANSPORTS_WORKFLOW, "utf8"); @@ -4290,11 +4551,9 @@ describe("package artifact reuse", () => { expect(npmTelegramJob.name).toBe("Run package Telegram E2E"); expect(npmTelegramJob.needs).toEqual(["resolve_target", "evidence_reuse"]); expect(npmTelegramJob["timeout-minutes"]).toBe( - "${{ inputs.release_profile == 'full' && 360 || 60 }}", - ); - expect(performanceJob["timeout-minutes"]).toBe( "${{ inputs.release_profile == 'full' && 360 || 120 }}", ); + expect(performanceJob["timeout-minutes"]).toBe(360); expect(npmTelegramJob.if).toContain( 'contains(fromJSON(\'["all","npm-telegram"]\'), inputs.rerun_group)', ); @@ -4318,6 +4577,7 @@ describe("package artifact reuse", () => { SKIP_PACKAGE_TELEGRAM_E2E: "${{ inputs.skip_package_telegram_e2e }}", }); expectTextToIncludeAll(targetSummaryStep.run, [ + "Validation SHA:", "Package Acceptance Telegram E2E deferred:", "Package Telegram E2E: deferred by \\`skip_package_telegram_e2e\\`", ]); @@ -4378,7 +4638,7 @@ describe("package artifact reuse", () => { 'args+=(-f cross_os_suite_filter="$CROSS_OS_SUITE_FILTER")', 'case "$RERUN_GROUP" in', "release-checks|install-smoke|cross-os|live-e2e|package|qa|qa-parity|qa-live)", - "cancel-in-progress: ${{ (inputs.ref == 'main' && inputs.rerun_group == 'all') || startsWith(inputs.ref, 'tideclaw/alpha/') || startsWith(inputs.ref, 'release/') }}", + "cancel-in-progress: false", "Verify release checks accepted Tideclaw alpha advisory lanes", "release_checks_advisory_only", "release_check_blocking_job", @@ -5923,16 +6183,17 @@ wait_for_run plugin-clawhub-new.yml 123 "${expectedSha}" || status=$? expect(mismatchedWaitSha.status, mismatchedWaitSha.stderr).toBe(0); }); - it("keeps release workflow setup and timeout budgets bounded", () => { - const fullRelease = readWorkflow(FULL_RELEASE_VALIDATION_WORKFLOW); + it("keeps release workflow setup aligned", () => { const releaseChecks = readWorkflow(RELEASE_CHECKS_WORKFLOW); - const crossOs = readWorkflow(".github/workflows/openclaw-cross-os-release-checks-reusable.yml"); + const installSmoke = readWorkflow(INSTALL_SMOKE_REUSABLE_WORKFLOW); + const crossOs = readWorkflow(CROSS_OS_RELEASE_CHECKS_REUSABLE_WORKFLOW); const liveE2e = readWorkflow(LIVE_E2E_WORKFLOW); + const qaLive = readWorkflow(QA_LIVE_TRANSPORTS_WORKFLOW); const releaseWorkflowPaths = [ FULL_RELEASE_VALIDATION_WORKFLOW, RELEASE_CHECKS_WORKFLOW, RELEASE_TELEGRAM_QA_WORKFLOW, - ".github/workflows/openclaw-cross-os-release-checks-reusable.yml", + CROSS_OS_RELEASE_CHECKS_REUSABLE_WORKFLOW, LIVE_E2E_WORKFLOW, NPM_TELEGRAM_WORKFLOW, ".github/workflows/openclaw-release-publish.yml", @@ -5950,8 +6211,6 @@ wait_for_run plugin-clawhub-new.yml 123 "${expectedSha}" || status=$? expect(workflow.env?.PNPM_VERSION, workflowPath).toBeUndefined(); } - expect(fullRelease.jobs?.release_checks?.["timeout-minutes"]).toBe(240); - expect(fullRelease.jobs?.prepare_release_package).toBeUndefined(); expect(releaseChecks.jobs?.prepare_release_package?.["timeout-minutes"]).toBe(15); expect( workflowStep( @@ -5959,7 +6218,21 @@ wait_for_run plugin-clawhub-new.yml 123 "${expectedSha}" || status=$? "Setup Node environment", ).with?.["install-deps"], ).toBe("true"); + expect(installSmoke.jobs?.preflight?.["timeout-minutes"]).toBe(15); + expect(installSmoke.jobs?.["install-smoke-fast"]?.["timeout-minutes"]).toBe(120); + expect(installSmoke.jobs?.root_dockerfile_image?.["timeout-minutes"]).toBe(60); + expect(installSmoke.jobs?.root_dockerfile_image_ready?.["timeout-minutes"]).toBe(5); + expect(installSmoke.jobs?.qr_package_install_smoke?.["timeout-minutes"]).toBe(30); + expect(installSmoke.jobs?.root_dockerfile_smokes?.["timeout-minutes"]).toBe(90); + expect(installSmoke.jobs?.installer_smoke?.["timeout-minutes"]).toBe(150); + expect(installSmoke.jobs?.bun_global_install_smoke?.["timeout-minutes"]).toBe(60); + expect(installSmoke.jobs?.["docker-e2e-fast"]?.["timeout-minutes"]).toBe(12); + expect(crossOs.jobs?.prepare?.["timeout-minutes"]).toBe(90); expect(crossOs.jobs?.cross_os_release_checks?.["timeout-minutes"]).toBe(60); + expect(qaLive.jobs?.authorize_actor?.["timeout-minutes"]).toBe(10); + expect(qaLive.jobs?.validate_selected_ref?.["timeout-minutes"]).toBe(30); + expect(liveE2e.jobs?.validate_live_suite_filter?.["timeout-minutes"]).toBe(10); + expect(liveE2e.jobs?.plan_release_workflow_matrices?.["timeout-minutes"]).toBe(10); expect(liveE2e.jobs?.validate_release_live_cache?.["timeout-minutes"]).toBe(20); expect(readFileSync(LIVE_E2E_WORKFLOW, "utf8")).toContain( "timeout --foreground --kill-after=30s 8m pnpm test:live:cache", @@ -5967,6 +6240,396 @@ wait_for_run plugin-clawhub-new.yml 123 "${expectedSha}" || status=$? expect(readFileSync(LIVE_E2E_WORKFLOW, "utf8")).toContain("live-cache attempt ${attempt}/2"); }); + it("keeps known bounded dominant child paths below parent monitors", () => { + const fullRelease = readWorkflow(FULL_RELEASE_VALIDATION_WORKFLOW); + const pluginPrerelease = readWorkflow(PLUGIN_PRERELEASE_WORKFLOW); + const liveE2e = readWorkflow(LIVE_E2E_WORKFLOW); + const releaseChecks = readWorkflow(RELEASE_CHECKS_WORKFLOW); + const installSmoke = readWorkflow(INSTALL_SMOKE_REUSABLE_WORKFLOW); + const crossOs = readWorkflow(CROSS_OS_RELEASE_CHECKS_REUSABLE_WORKFLOW); + const packageAcceptance = readWorkflow(PACKAGE_ACCEPTANCE_WORKFLOW); + const qaLive = readWorkflow(QA_LIVE_TRANSPORTS_WORKFLOW); + const performance = readWorkflow(PERFORMANCE_WORKFLOW); + const profiles = ["beta", "stable", "full"] as const; + + const ciPreflight = workflowJob(CI_WORKFLOW, "preflight"); + const ciIos = workflowJob(CI_WORKFLOW, "ios-build"); + const ciGate = workflowJob(CI_WORKFLOW, "ci-gate"); + expect(jobNeeds(ciIos)).toEqual(["preflight"]); + expect(jobNeeds(ciGate)).toEqual(expect.arrayContaining(["preflight", "ios-build"])); + const ciPath = [ + timeoutForProfile(ciPreflight["timeout-minutes"], "beta"), + timeoutForProfile(ciIos["timeout-minutes"], "beta"), + timeoutForProfile(ciGate["timeout-minutes"], "beta"), + ]; + expect(ciPath).toEqual([20, 150, 5]); + const ciChildTimeout = ciPath.reduce((total, timeout) => total + timeout, 0); + expect(ciChildTimeout).toBe(175); + const ciParentTimeout = timeoutForProfile( + fullRelease.jobs?.normal_ci?.["timeout-minutes"], + "beta", + ); + expect(ciChildTimeout).toBeLessThanOrEqual(ciParentTimeout); + expect(ciParentTimeout - ciChildTimeout).toBeGreaterThanOrEqual(60); + + expect(liveE2e.jobs?.validate_selected_ref?.["timeout-minutes"]).toBe(30); + const pluginMonitorTimeout = fullRelease.jobs?.plugin_prerelease?.["timeout-minutes"]; + const pluginChildTimeouts = Object.fromEntries( + profiles.map((profile) => [ + profile, + pluginPrereleaseTimeoutFloor(pluginPrerelease, liveE2e, profile), + ]), + ) as Record<(typeof profiles)[number], number>; + const pluginParentTimeouts = Object.fromEntries( + profiles.map((profile) => [profile, timeoutForProfile(pluginMonitorTimeout, profile)]), + ) as Record<(typeof profiles)[number], number>; + expect(pluginChildTimeouts).toEqual({ beta: 175, stable: 175, full: 205 }); + expect(pluginParentTimeouts).toEqual({ beta: 240, stable: 240, full: 300 }); + for (const profile of profiles) { + expect( + pluginParentTimeouts[profile] - pluginChildTimeouts[profile], + `plugin-prerelease:${profile}`, + ).toBeGreaterThanOrEqual(60); + } + + const releasePackageJob = workflowJob( + RELEASE_CHECKS_WORKFLOW, + "package_acceptance_release_checks", + ); + expect(jobNeeds(workflowJob(RELEASE_CHECKS_WORKFLOW, "prepare_release_package"))).toEqual([ + "resolve_target", + ]); + expect(jobNeeds(releasePackageJob)).toEqual(["resolve_target", "prepare_release_package"]); + expect(jobNeeds(workflowJob(PACKAGE_ACCEPTANCE_WORKFLOW, "package_integrity"))).toEqual([ + "resolve_package", + ]); + expect(jobNeeds(workflowJob(PACKAGE_ACCEPTANCE_WORKFLOW, "docker_acceptance"))).toEqual([ + "resolve_package", + "package_integrity", + ]); + expect(jobNeeds(workflowJob(LIVE_E2E_WORKFLOW, "prepare_docker_e2e_image"))).toEqual([ + "validate_selected_ref", + ]); + expect(jobNeeds(workflowJob(LIVE_E2E_WORKFLOW, "docker_e2e_image_ready"))).toEqual([ + "prepare_docker_e2e_image", + ]); + expect(jobNeeds(workflowJob(LIVE_E2E_WORKFLOW, "validate_docker_lanes"))).toEqual( + expect.arrayContaining([ + "validate_selected_ref", + "prepare_docker_e2e_image", + "docker_e2e_image_ready", + ]), + ); + expect(jobNeeds(workflowJob(PACKAGE_ACCEPTANCE_WORKFLOW, "summary"))).toContain( + "docker_acceptance", + ); + expect(jobNeeds(workflowJob(RELEASE_CHECKS_WORKFLOW, "summary"))).toContain( + "package_acceptance_release_checks", + ); + const releasePackagePaths = Object.fromEntries( + profiles.map((profile) => [ + profile, + [ + timeoutForProfile(releaseChecks.jobs?.resolve_target?.["timeout-minutes"], profile), + timeoutForProfile( + releaseChecks.jobs?.prepare_release_package?.["timeout-minutes"], + profile, + ), + timeoutForProfile(packageAcceptance.jobs?.resolve_package?.["timeout-minutes"], profile), + timeoutForProfile( + packageAcceptance.jobs?.package_integrity?.["timeout-minutes"], + profile, + ), + timeoutForProfile(liveE2e.jobs?.validate_selected_ref?.["timeout-minutes"], profile), + timeoutForProfile(liveE2e.jobs?.prepare_docker_e2e_image?.["timeout-minutes"], profile), + timeoutForProfile(liveE2e.jobs?.docker_e2e_image_ready?.["timeout-minutes"], profile), + timeoutForProfile(liveE2e.jobs?.validate_docker_lanes?.["timeout-minutes"], profile), + timeoutForProfile(packageAcceptance.jobs?.summary?.["timeout-minutes"], profile), + timeoutForProfile(releaseChecks.jobs?.summary?.["timeout-minutes"], profile), + ], + ]), + ) as Record<(typeof profiles)[number], number[]>; + expect(releasePackagePaths).toEqual({ + beta: [30, 15, 60, 10, 30, 60, 5, 60, 5, 5], + stable: [30, 15, 60, 10, 30, 60, 5, 60, 5, 5], + full: [30, 15, 60, 10, 30, 90, 5, 60, 5, 5], + }); + const releaseChecksParent = workflowJob(FULL_RELEASE_VALIDATION_WORKFLOW, "release_checks"); + expect(releaseChecksParent["runs-on"]).toBe("blacksmith-4vcpu-ubuntu-2404"); + expect(releaseChecksParent["timeout-minutes"]).toBe(420); + const releasePackageTimeouts = Object.fromEntries( + profiles.map((profile) => [ + profile, + releasePackagePaths[profile].reduce((total, timeout) => total + timeout, 0), + ]), + ); + expect(releasePackageTimeouts).toEqual({ beta: 280, stable: 280, full: 310 }); + for (const profile of profiles) { + const childTimeout = releasePackageTimeouts[profile]; + expect(childTimeout, `release-package:${profile}`).toBeLessThanOrEqual(420); + expect(420 - childTimeout, `release-package:${profile}`).toBeGreaterThanOrEqual(60); + } + + const releaseSummary = workflowJob(RELEASE_CHECKS_WORKFLOW, "summary"); + const releaseCrossOs = workflowJob(RELEASE_CHECKS_WORKFLOW, "cross_os_release_checks"); + expect(jobNeeds(releaseCrossOs)).toEqual(["resolve_target", "prepare_release_package"]); + expect(jobNeeds(workflowJob(CROSS_OS_RELEASE_CHECKS_REUSABLE_WORKFLOW, "prepare"))).toEqual([]); + expect( + jobNeeds(workflowJob(CROSS_OS_RELEASE_CHECKS_REUSABLE_WORKFLOW, "cross_os_release_checks")), + ).toEqual(["prepare"]); + expect(jobNeeds(releaseSummary)).toContain("cross_os_release_checks"); + const releaseCrossOsPath = [ + timeoutForProfile(releaseChecks.jobs?.resolve_target?.["timeout-minutes"], "stable"), + timeoutForProfile(releaseChecks.jobs?.prepare_release_package?.["timeout-minutes"], "stable"), + timeoutForProfile(crossOs.jobs?.prepare?.["timeout-minutes"], "stable"), + timeoutForProfile(crossOs.jobs?.cross_os_release_checks?.["timeout-minutes"], "stable"), + timeoutForProfile(releaseChecks.jobs?.summary?.["timeout-minutes"], "stable"), + ]; + expect(releaseCrossOsPath).toEqual([30, 15, 90, 60, 5]); + + const releaseInstall = workflowJob(RELEASE_CHECKS_WORKFLOW, "install_smoke_release_checks"); + expect(jobNeeds(releaseInstall)).toEqual(["resolve_target"]); + expect(jobNeeds(workflowJob(INSTALL_SMOKE_REUSABLE_WORKFLOW, "root_dockerfile_image"))).toEqual( + ["preflight"], + ); + expect( + jobNeeds(workflowJob(INSTALL_SMOKE_REUSABLE_WORKFLOW, "root_dockerfile_image_ready")), + ).toEqual(["preflight", "root_dockerfile_image"]); + expect(jobNeeds(workflowJob(INSTALL_SMOKE_REUSABLE_WORKFLOW, "installer_smoke"))).toEqual([ + "preflight", + "root_dockerfile_image", + "root_dockerfile_image_ready", + ]); + expect(jobNeeds(releaseSummary)).toContain("install_smoke_release_checks"); + const releaseInstallPath = [ + timeoutForProfile(releaseChecks.jobs?.resolve_target?.["timeout-minutes"], "stable"), + timeoutForProfile(installSmoke.jobs?.preflight?.["timeout-minutes"], "stable"), + timeoutForProfile(installSmoke.jobs?.root_dockerfile_image?.["timeout-minutes"], "stable"), + timeoutForProfile( + installSmoke.jobs?.root_dockerfile_image_ready?.["timeout-minutes"], + "stable", + ), + timeoutForProfile(installSmoke.jobs?.installer_smoke?.["timeout-minutes"], "stable"), + timeoutForProfile(releaseChecks.jobs?.summary?.["timeout-minutes"], "stable"), + ]; + expect(releaseInstallPath).toEqual([30, 15, 60, 5, 150, 5]); + + const releaseQaLive = workflowJob(RELEASE_CHECKS_WORKFLOW, "qa_live_release_checks"); + expect(jobNeeds(releaseQaLive)).toEqual(["resolve_target"]); + expect(jobNeeds(workflowJob(QA_LIVE_TRANSPORTS_WORKFLOW, "validate_selected_ref"))).toEqual([ + "authorize_actor", + ]); + expect(jobNeeds(workflowJob(QA_LIVE_TRANSPORTS_WORKFLOW, "run_live_matrix"))).toEqual([ + "authorize_actor", + "validate_selected_ref", + ]); + expect(jobNeeds(releaseSummary)).toContain("qa_live_release_checks"); + const releaseQaLivePath = [ + timeoutForProfile(releaseChecks.jobs?.resolve_target?.["timeout-minutes"], "stable"), + timeoutForProfile(qaLive.jobs?.authorize_actor?.["timeout-minutes"], "stable"), + timeoutForProfile(qaLive.jobs?.validate_selected_ref?.["timeout-minutes"], "stable"), + timeoutForProfile(qaLive.jobs?.run_live_matrix?.["timeout-minutes"], "stable"), + timeoutForProfile(releaseChecks.jobs?.summary?.["timeout-minutes"], "stable"), + ]; + expect(releaseQaLivePath).toEqual([30, 10, 30, 90, 5]); + + for (const [pathName, path] of [ + ["cross-os", releaseCrossOsPath], + ["install", releaseInstallPath], + ["qa-live", releaseQaLivePath], + ] as const) { + const childTimeout = path.reduce((total, timeout) => total + timeout, 0); + expect(childTimeout, `release-checks:${pathName}`).toBeLessThanOrEqual(420); + expect(420 - childTimeout, `release-checks:${pathName}`).toBeGreaterThanOrEqual(60); + } + expect(releaseCrossOsPath.reduce((total, timeout) => total + timeout, 0)).toBe(200); + expect(releaseInstallPath.reduce((total, timeout) => total + timeout, 0)).toBe(265); + expect(releaseQaLivePath.reduce((total, timeout) => total + timeout, 0)).toBe(165); + + expect( + jobNeeds(workflowJob(RELEASE_CHECKS_WORKFLOW, "qa_live_telegram_release_checks")), + ).toEqual(["resolve_target"]); + expect(jobNeeds(workflowJob(RELEASE_CHECKS_WORKFLOW, "summary"))).toContain( + "qa_live_telegram_release_checks", + ); + const releaseTelegramPath = [ + timeoutForProfile(releaseChecks.jobs?.resolve_target?.["timeout-minutes"], "beta"), + timeoutForProfile( + releaseChecks.jobs?.qa_live_telegram_release_checks?.["timeout-minutes"], + "beta", + ), + timeoutForProfile(releaseChecks.jobs?.summary?.["timeout-minutes"], "beta"), + ]; + expect(releaseTelegramPath).toEqual([30, 210, 5]); + const releaseTelegramTimeout = releaseTelegramPath.reduce( + (total, timeout) => total + timeout, + 0, + ); + expect(releaseTelegramTimeout).toBe(245); + expect(420 - releaseTelegramTimeout).toBeGreaterThanOrEqual(60); + + const npmTelegramChildTimeout = timeoutForProfile( + workflowJob(NPM_TELEGRAM_WORKFLOW, "run_package_telegram_e2e")["timeout-minutes"], + "beta", + ); + expect(npmTelegramChildTimeout).toBe(60); + for (const profile of profiles) { + const parentTimeout = timeoutForProfile( + fullRelease.jobs?.npm_telegram?.["timeout-minutes"], + profile, + ); + expect(parentTimeout).toBe(profile === "full" ? 360 : 120); + expect( + parentTimeout - npmTelegramChildTimeout, + `npm-telegram:${profile}`, + ).toBeGreaterThanOrEqual(60); + } + + const performanceResolve = workflowJob(PERFORMANCE_WORKFLOW, "resolve_target"); + const performanceKova = workflowJob(PERFORMANCE_WORKFLOW, "kova"); + const performanceSource = workflowJob(PERFORMANCE_WORKFLOW, "source_performance"); + const performancePublish = workflowJob(PERFORMANCE_WORKFLOW, "publish"); + const performanceArtifactGuard = workflowJob(PERFORMANCE_WORKFLOW, "artifact_only_guard"); + expect(jobNeeds(performanceKova)).toEqual(["resolve_target"]); + expect(jobNeeds(performanceSource)).toEqual(["resolve_target"]); + expect(jobNeeds(performancePublish)).toEqual(["resolve_target", "kova", "source_performance"]); + expect(jobNeeds(performanceArtifactGuard)).toEqual(["resolve_target", "kova", "publish"]); + expect(performancePublish.if).toContain("inputs.publish_reports == true"); + expect(performanceArtifactGuard.if).toContain("inputs.publish_reports != true"); + expect(timeoutForProfile(performanceSource["timeout-minutes"], "beta")).toBeLessThanOrEqual( + timeoutForProfile(performanceKova["timeout-minutes"], "beta"), + ); + const performanceArtifactPath = [ + timeoutForProfile(performanceResolve["timeout-minutes"], "beta"), + timeoutForProfile(performanceKova["timeout-minutes"], "beta"), + timeoutForProfile(performanceArtifactGuard["timeout-minutes"], "beta"), + ]; + const performancePublishPath = [ + timeoutForProfile(performanceResolve["timeout-minutes"], "beta"), + timeoutForProfile(performanceKova["timeout-minutes"], "beta"), + timeoutForProfile(performancePublish["timeout-minutes"], "beta"), + ]; + expect(performanceArtifactPath).toEqual([10, 240, 5]); + expect(performancePublishPath).toEqual([10, 240, 30]); + expect(performanceArtifactPath.reduce((total, timeout) => total + timeout, 0)).toBe(255); + expect(performancePublishPath.reduce((total, timeout) => total + timeout, 0)).toBe(280); + const performanceParent = workflowJob(FULL_RELEASE_VALIDATION_WORKFLOW, "performance"); + expect(performanceParent["timeout-minutes"]).toBe(360); + expect( + workflowStep(performanceParent, "Dispatch and monitor OpenClaw Performance").run, + ).toContain("-f publish_reports=false"); + for (const [pathName, path] of [ + ["artifact-only", performanceArtifactPath], + ["publish", performancePublishPath], + ] as const) { + const childTimeout = path.reduce((total, timeout) => total + timeout, 0); + expect(childTimeout, `performance:${pathName}`).toBeLessThanOrEqual(360); + expect(360 - childTimeout, `performance:${pathName}`).toBeGreaterThanOrEqual(60); + } + + const prepareReleaseCandidate = workflowJob( + FULL_RELEASE_VALIDATION_WORKFLOW, + "prepare_release_candidate", + ); + expect(jobNeeds(workflowJob(FULL_RELEASE_VALIDATION_WORKFLOW, "evidence_reuse"))).toEqual([ + "resolve_target", + ]); + expect(jobNeeds(prepareReleaseCandidate)).toEqual(["resolve_target", "evidence_reuse"]); + expect(prepareReleaseCandidate.with?.prepare_only).toBe(true); + expect(jobNeeds(releaseChecksParent)).toEqual([ + "resolve_target", + "evidence_reuse", + "prepare_release_candidate", + ]); + expect(jobNeeds(workflowJob(FULL_RELEASE_VALIDATION_WORKFLOW, "summary"))).toContain( + "release_checks", + ); + const fullParentPath = [ + timeoutForProfile(fullRelease.jobs?.resolve_target?.["timeout-minutes"], "full"), + timeoutForProfile(fullRelease.jobs?.evidence_reuse?.["timeout-minutes"], "full"), + timeoutForProfile(liveE2e.jobs?.validate_selected_ref?.["timeout-minutes"], "full"), + timeoutForProfile(liveE2e.jobs?.prepare_docker_e2e_image?.["timeout-minutes"], "full"), + timeoutForProfile(liveE2e.jobs?.docker_e2e_image_ready?.["timeout-minutes"], "full"), + timeoutForProfile(releaseChecksParent["timeout-minutes"], "full"), + timeoutForProfile(fullRelease.jobs?.summary?.["timeout-minutes"], "full"), + ]; + expect(fullParentPath).toEqual([10, 10, 30, 90, 5, 420, 5]); + const fullParentTimeoutFloor = fullParentPath.reduce((total, timeout) => total + timeout, 0); + expect(fullParentTimeoutFloor).toBe(570); + expect(FULL_RELEASE_WAIT_TIMEOUT_MINUTES - fullParentTimeoutFloor).toBeGreaterThanOrEqual(60); + }); + + it("bounds every direct job in nested release workflows", () => { + const boundedWorkflowPaths = [ + RELEASE_CHECKS_WORKFLOW, + INSTALL_SMOKE_REUSABLE_WORKFLOW, + CROSS_OS_RELEASE_CHECKS_REUSABLE_WORKFLOW, + LIVE_E2E_WORKFLOW, + PACKAGE_ACCEPTANCE_WORKFLOW, + QA_LIVE_TRANSPORTS_WORKFLOW, + RELEASE_TELEGRAM_QA_WORKFLOW, + NPM_TELEGRAM_WORKFLOW, + ]; + + for (const path of boundedWorkflowPaths) { + const jobs = readWorkflow(path).jobs ?? {}; + expect(Object.keys(jobs).length, path).toBeGreaterThan(0); + for (const [jobName, job] of Object.entries(jobs)) { + if (job.uses) { + // GitHub does not allow timeout-minutes on reusable-workflow caller jobs. + expect(job["timeout-minutes"], `${path}:${jobName}`).toBeUndefined(); + continue; + } + + const evaluatedTimeouts = evaluatedJobTimeouts(path, jobName, job); + expect(evaluatedTimeouts.length, `${path}:${jobName}`).toBeGreaterThan(0); + for (const timeout of evaluatedTimeouts) { + expect(Number.isFinite(timeout), `${path}:${jobName}`).toBe(true); + expect(timeout, `${path}:${jobName}`).toBeGreaterThan(0); + } + } + } + }); + + it("pins every documented raw Full Release Validation caller to one exact SHA", () => { + const nightly = readFileSync(".agents/skills/release-openclaw-nightly/SKILL.md", "utf8"); + const liveUpdater = readFileSync(".agents/skills/openclaw-live-updater/SKILL.md", "utf8"); + const releaseCi = readFileSync(".agents/skills/release-openclaw-ci/SKILL.md", "utf8"); + const releaseCiNotes = readFileSync( + ".agents/skills/release-openclaw-ci/references/release-ci-notes.md", + "utf8", + ); + const testing = readFileSync(".agents/skills/openclaw-testing/SKILL.md", "utf8"); + const parallels = readFileSync(".agents/skills/openclaw-parallels-smoke/SKILL.md", "utf8"); + const maintainer = readFileSync(RELEASE_MAINTAINER_SKILL, "utf8"); + const ciDocs = readFileSync("docs/ci.md", "utf8"); + const fullReleaseDocs = readFileSync("docs/reference/full-release-validation.md", "utf8"); + const releasingDocs = readFileSync("docs/reference/RELEASING.md", "utf8"); + + expect(nightly).toContain('-f expected_sha="$SHA"'); + expectTextToIncludeAll(liveUpdater, [ + 'MAIN_SHA=""', + '-f ref="$MAIN_SHA"', + '-f expected_sha="$MAIN_SHA"', + ]); + for (const text of [releaseCi, fullReleaseDocs, releasingDocs]) { + expectTextToIncludeAll(text, [ + 'RELEASE_SHA="$(git rev-parse HEAD)"', + "-f ref=extended-stable/YYYY.M.33", + '-f expected_sha="$RELEASE_SHA"', + ]); + } + expectTextToIncludeAll(ciDocs, [ + 'VALIDATION_SHA=""', + '-f ref="$VALIDATION_SHA"', + '-f expected_sha="$VALIDATION_SHA"', + ]); + for (const text of [releaseCi, releaseCiNotes, testing, parallels, ciDocs, maintainer]) { + expect(text).toContain("Validation SHA + Tooling SHA"); + } + expect(releaseCi).toContain("release lifecycle ledger: Code SHA, Release SHA, and Tooling SHA"); + }); + it("executes shared release candidate identity validation with its JSON input", () => { const selectedSha = "a".repeat(40); const candidate = { diff --git a/test/scripts/plugin-prerelease-test-plan.test.ts b/test/scripts/plugin-prerelease-test-plan.test.ts index 392178db6732..d7265beb6471 100644 --- a/test/scripts/plugin-prerelease-test-plan.test.ts +++ b/test/scripts/plugin-prerelease-test-plan.test.ts @@ -38,6 +38,59 @@ function readPluginPrereleaseWorkflow() { return parse(readFileSync(".github/workflows/plugin-prerelease.yml", "utf8")); } +function readLiveE2eWorkflow() { + return parse(readFileSync(".github/workflows/openclaw-live-and-e2e-checks-reusable.yml", "utf8")); +} + +function jobNeeds(job: { needs?: string | string[] }): string[] { + return Array.isArray(job.needs) ? job.needs : job.needs ? [job.needs] : []; +} + +function timeoutForProfile( + timeout: number | string | undefined, + profile: "beta" | "stable" | "full", +): number { + if (typeof timeout === "number") { + return timeout; + } + const match = timeout?.match( + /^\$\{\{ inputs\.(?:release_profile|release_test_profile) == 'full' && ([0-9]+) \|\| ([0-9]+) \}\}$/u, + ); + if (!match) { + throw new Error(`Unsupported release timeout expression: ${String(timeout)}`); + } + return Number(profile === "full" ? match[1] : match[2]); +} + +function pluginPrereleaseTimeoutFloor(profile: "beta" | "stable" | "full"): number { + const plugin = readPluginPrereleaseWorkflow(); + const liveE2e = readLiveE2eWorkflow(); + const preflight = plugin.jobs.preflight; + const dockerSuite = plugin.jobs["plugin-prerelease-docker-suite"]; + const suite = plugin.jobs["plugin-prerelease-suite"]; + const validateSelectedRef = liveE2e.jobs.validate_selected_ref; + const prepareImage = liveE2e.jobs.prepare_docker_e2e_image; + const imageReady = liveE2e.jobs.docker_e2e_image_ready; + const dockerLanes = liveE2e.jobs.validate_docker_lanes; + + expect(jobNeeds(dockerSuite)).toContain("preflight"); + expect(jobNeeds(prepareImage)).toEqual(["validate_selected_ref"]); + expect(jobNeeds(imageReady)).toEqual(["prepare_docker_e2e_image"]); + expect(jobNeeds(dockerLanes)).toEqual( + expect.arrayContaining(["prepare_docker_e2e_image", "docker_e2e_image_ready"]), + ); + expect(jobNeeds(suite)).toContain("plugin-prerelease-docker-suite"); + + return [ + timeoutForProfile(preflight["timeout-minutes"], profile), + timeoutForProfile(validateSelectedRef["timeout-minutes"], profile), + timeoutForProfile(prepareImage["timeout-minutes"], profile), + timeoutForProfile(imageReady["timeout-minutes"], profile), + timeoutForProfile(dockerLanes["timeout-minutes"], profile), + timeoutForProfile(suite["timeout-minutes"], profile), + ].reduce((total, value) => total + value, 0); +} + function getDockerLane(name: string) { const lane = findLaneByName(name); if (!lane) { @@ -610,7 +663,7 @@ describe("scripts/lib/plugin-prerelease-test-plan.mts", () => { ).toContain("plugin-prerelease-inspector advisory result"); }); - it("keeps release-check reruns independent while cancelling superseded umbrella runs", () => { + it("keeps exact release tuples independent without cancelling adopted children", () => { const releaseChecksWorkflow = parse( readFileSync(".github/workflows/openclaw-release-checks.yml", "utf8"), ); @@ -622,10 +675,26 @@ describe("scripts/lib/plugin-prerelease-test-plan.mts", () => { "cancel-in-progress": "${{ startsWith(github.ref, 'refs/heads/tideclaw/alpha/') }}", }); expect(fullReleaseWorkflow.concurrency).toEqual({ - group: "full-release-validation-${{ inputs.ref }}-${{ inputs.rerun_group }}", - "cancel-in-progress": - "${{ (inputs.ref == 'main' && inputs.rerun_group == 'all') || startsWith(inputs.ref, 'tideclaw/alpha/') || startsWith(inputs.ref, 'release/') }}", + group: + "full-release-validation-${{ inputs.expected_sha || inputs.ref }}-${{ github.sha }}-${{ inputs.rerun_group }}", + "cancel-in-progress": false, }); + expect(fullReleaseWorkflow.on.workflow_dispatch.inputs.expected_sha).toEqual({ + description: "Optional full Validation SHA that ref must resolve to", + required: false, + default: "", + type: "string", + }); + const resolveTargetStep = fullReleaseWorkflow.jobs.resolve_target.steps.find( + (step: WorkflowStep) => step.name === "Resolve target SHA", + ); + const targetSummaryStep = fullReleaseWorkflow.jobs.resolve_target.steps.find( + (step: WorkflowStep) => step.name === "Summarize target", + ); + expect(resolveTargetStep.env?.EXPECTED_SHA).toBe("${{ inputs.expected_sha }}"); + expect(resolveTargetStep.run).toContain('--expected-sha "$EXPECTED_SHA"'); + expect(targetSummaryStep.run).toContain("- Validation SHA:"); + expect(targetSummaryStep.run).not.toContain("- Code SHA:"); expect(releaseChecksWorkflow.jobs.resolve_target["runs-on"]).toBe("ubuntu-24.04"); expect(releaseChecksWorkflow.jobs.prepare_release_package["runs-on"]).toBe("ubuntu-24.04"); expect(releaseChecksWorkflow.jobs.summary["runs-on"]).toBe("ubuntu-24.04"); @@ -634,16 +703,14 @@ describe("scripts/lib/plugin-prerelease-test-plan.mts", () => { "docker_runtime_assets_preflight", "normal_ci", "plugin_prerelease", - "release_checks", "npm_telegram", "summary", ]) { expect(fullReleaseWorkflow.jobs[jobName]["runs-on"]).toBe("ubuntu-24.04"); } + expect(fullReleaseWorkflow.jobs.release_checks["runs-on"]).toBe("blacksmith-4vcpu-ubuntu-2404"); expect(fullReleaseWorkflow.jobs.performance["runs-on"]).toBe("blacksmith-4vcpu-ubuntu-2404"); - expect(fullReleaseWorkflow.jobs.normal_ci["timeout-minutes"]).toBe( - "${{ inputs.release_profile != 'beta' && 240 || 60 }}", - ); + expect(fullReleaseWorkflow.jobs.normal_ci["timeout-minutes"]).toBe(240); expect(fullReleaseWorkflow.jobs.normal_ci.needs).toEqual(["resolve_target", "evidence_reuse"]); expect(fullReleaseWorkflow.jobs.normal_ci.if).toContain( "needs.resolve_target.result == 'success'", @@ -670,10 +737,29 @@ describe("scripts/lib/plugin-prerelease-test-plan.mts", () => { (step: WorkflowStep) => step.name === "Build and smoke test final Docker runtime image", ), ).toBe(false); - expect(fullReleaseWorkflow.jobs.plugin_prerelease["timeout-minutes"]).toBe( - "${{ inputs.release_profile == 'full' && 300 || inputs.release_profile == 'stable' && 240 || 60 }}", + const pluginMonitorTimeout = fullReleaseWorkflow.jobs.plugin_prerelease["timeout-minutes"]; + const childTimeoutFloors = { + beta: pluginPrereleaseTimeoutFloor("beta"), + stable: pluginPrereleaseTimeoutFloor("stable"), + full: pluginPrereleaseTimeoutFloor("full"), + }; + const parentTimeouts = { + beta: timeoutForProfile(pluginMonitorTimeout, "beta"), + stable: timeoutForProfile(pluginMonitorTimeout, "stable"), + full: timeoutForProfile(pluginMonitorTimeout, "full"), + }; + expect(childTimeoutFloors).toEqual({ beta: 175, stable: 175, full: 205 }); + expect(parentTimeouts).toEqual({ beta: 240, stable: 240, full: 300 }); + for (const profile of ["beta", "stable", "full"] as const) { + expect(parentTimeouts[profile] - childTimeoutFloors[profile], profile).toBeGreaterThanOrEqual( + 60, + ); + } + expect(fullReleaseWorkflow.jobs.release_checks["timeout-minutes"]).toBe(420); + expect(fullReleaseWorkflow.jobs.npm_telegram["timeout-minutes"]).toBe( + "${{ inputs.release_profile == 'full' && 360 || 120 }}", ); - expect(fullReleaseWorkflow.jobs.release_checks["timeout-minutes"]).toBe(240); + expect(fullReleaseWorkflow.jobs.performance["timeout-minutes"]).toBe(360); const fullReleaseSource = readFileSync(".github/workflows/full-release-validation.yml", "utf8"); expect(fullReleaseWorkflow.on.workflow_dispatch.inputs.fail_fast).toEqual({ description: @@ -699,6 +785,17 @@ describe("scripts/lib/plugin-prerelease-test-plan.mts", () => { expect(fullReleaseSource).toContain( "npm-telegram-beta-e2e.yml has failed child jobs before the workflow completed; cancelling the remaining run.", ); + expect(fullReleaseSource).not.toContain("trap cancel_child"); + expect(fullReleaseSource).not.toContain("cancel_child_on_failure"); + expect(fullReleaseSource).not.toContain("exit_on_parent_signal"); + expect(fullReleaseSource).not.toContain("disable_child_cleanup"); + expect(fullReleaseSource).toContain( + "Parent cancellation leaves this child running; cancel it explicitly if no longer needed.", + ); + expect(fullReleaseSource).toContain( + 'if [[ "$child_head_sha" != "$PARENT_WORKFLOW_SHA" ]]; then', + ); + expect(fullReleaseSource).toContain("cancel_child\n exit 1"); expect(releaseChecksWorkflow.on.workflow_dispatch.inputs.fail_fast).toEqual({ description: "Stop the Matrix QA lane after its first failed check or scenario", required: false, diff --git a/test/scripts/release-candidate-checklist.test.ts b/test/scripts/release-candidate-checklist.test.ts index b18e494d23a6..9cafdae7427c 100644 --- a/test/scripts/release-candidate-checklist.test.ts +++ b/test/scripts/release-candidate-checklist.test.ts @@ -578,6 +578,79 @@ describe("release candidate checklist", () => { ).toBe("full"); }); + it("defaults beta and alpha Parallels to postpublish confidence", () => { + const beta = parseArgs(["--tag", "v2026.5.14-beta.3"]); + const alpha = parseArgs([ + "--tag", + "v2026.5.14-alpha.2", + "--workflow-ref", + "tideclaw/alpha/2026-07-10-1200Z", + "--npm-dist-tag", + "alpha", + ]); + + for (const options of [beta, alpha]) { + expect(options.releaseProfile).toBe("beta"); + expect(options.parallelsMode).toBe("auto"); + expect(options.skipParallels).toBe(true); + expect(options.parallelsSkipReason).toBe("deferred to postpublish release:beta-smoke"); + } + }); + + it("supports explicit and profile-default Parallels execution", () => { + const beta = parseArgs(["--tag", "v2026.5.14-beta.3", "--run-parallels"]); + const stable = parseArgs(["--tag", "v2026.5.14", "--windows-node-tag", "v0.6.3"]); + const full = parseArgs([ + "--tag", + "v2026.5.14", + "--windows-node-tag", + "v0.6.3", + "--release-profile", + "full", + ]); + + expect(beta).toMatchObject({ + parallelsMode: "run", + parallelsSkipReason: "", + skipParallels: false, + }); + for (const options of [stable, full]) { + expect(options.parallelsMode).toBe("auto"); + expect(options.skipParallels).toBe(false); + expect(options.parallelsSkipReason).toBe(""); + } + }); + + it("supports an explicit Parallels skip without changing persisted state shape", () => { + const options = parseArgs([ + "--tag", + "v2026.5.14", + "--windows-node-tag", + "v0.6.3", + "--skip-parallels", + ]); + const state = buildReleaseCandidateState(options, { + targetSha: "a".repeat(40), + toolingSha: "b".repeat(40), + }); + + expect(options).toMatchObject({ + parallelsMode: "skip", + parallelsSkipReason: "operator skipped --skip-parallels", + skipParallels: true, + }); + expect(state.skipParallels).toBe(true); + expect(state).not.toHaveProperty("parallelsMode"); + expect(state).not.toHaveProperty("parallelsSkipReason"); + expect(state).not.toHaveProperty("runParallels"); + }); + + it("rejects conflicting Parallels modes", () => { + expect(() => + parseArgs(["--tag", "v2026.5.14-beta.3", "--run-parallels", "--skip-parallels"]), + ).toThrow("--run-parallels and --skip-parallels cannot be combined"); + }); + it("runs Parallels against the exact prepared candidate tarball", () => { expect(candidateParallelsArgs(".artifacts/preflight/openclaw.tgz", [], "/trusted")).toEqual([ "exec", @@ -937,6 +1010,7 @@ describe("release candidate checklist", () => { duplicateOption("--windows-node-tag", "v0.6.3", "v0.6.4"), duplicateFlag("--skip-dispatch"), duplicateFlag("--skip-local-generated-check"), + duplicateFlag("--run-parallels"), duplicateFlag("--skip-parallels"), duplicateFlag("--skip-telegram"), duplicateOption("--telegram-provider-mode", "mock-openai", "live-frontier"), diff --git a/test/scripts/release-no-push-workflow.test.ts b/test/scripts/release-no-push-workflow.test.ts index 1d1459187576..f35d628dc6f7 100644 --- a/test/scripts/release-no-push-workflow.test.ts +++ b/test/scripts/release-no-push-workflow.test.ts @@ -309,7 +309,7 @@ describe("release validation no-push transport", () => { expect(releaseHelper.with?.["persist-credentials"]).toBe(false); }); - it("owns identified children before rejecting a mismatched workflow SHA", () => { + it("records adopted children before monitoring and cancels only a mismatched workflow SHA", () => { const full = readWorkflow(FULL_RELEASE); for (const [jobName, stepName] of [ ["normal_ci", "Dispatch and monitor CI"], @@ -326,24 +326,28 @@ describe("release validation no-push transport", () => { ); expect(dispatchRun.match(/\.head_sha == \$head_sha/gu), jobName).toBeNull(); expect(dispatchRun, jobName).toContain('run_json="$(validate_child_run "$run_id")"'); - expect(dispatchRun, jobName).toContain('active_child_run_id="$run_id"'); - expect(dispatchRun, jobName).toContain("trap cancel_child EXIT INT TERM"); + expect(dispatchRun, jobName).not.toContain("trap cancel_child"); + expect(dispatchRun, jobName).not.toContain("cancel_child_on_failure"); + expect(dispatchRun, jobName).not.toContain("exit_on_parent_signal"); + expect(dispatchRun, jobName).not.toContain("disable_child_cleanup"); expect( dispatchRun.indexOf('run_json="$(validate_child_run "$run_id")"'), jobName, - ).toBeLessThan(dispatchRun.indexOf('active_child_run_id="$run_id"')); - expect(dispatchRun.indexOf('active_child_run_id="$run_id"'), jobName).toBeLessThan( - dispatchRun.indexOf("trap cancel_child EXIT INT TERM"), - ); - expect(dispatchRun.indexOf("trap cancel_child EXIT INT TERM"), jobName).toBeLessThan( - dispatchRun.indexOf('if [[ "$child_head_sha" != "$PARENT_WORKFLOW_SHA" ]]'), - ); + ).toBeLessThan(dispatchRun.indexOf('echo "run_id=${run_id}" >> "$GITHUB_OUTPUT"')); + expect( + dispatchRun.indexOf('echo "run_id=${run_id}" >> "$GITHUB_OUTPUT"'), + jobName, + ).toBeLessThan(dispatchRun.indexOf("poll_count=0")); + expect( + dispatchRun.indexOf('run_json="$(validate_child_run "$run_id")"'), + jobName, + ).toBeLessThan(dispatchRun.indexOf('if [[ "$child_head_sha" != "$PARENT_WORKFLOW_SHA" ]]')); const shaMismatch = dispatchRun.slice( dispatchRun.indexOf('if [[ "$child_head_sha" != "$PARENT_WORKFLOW_SHA" ]]'), dispatchRun.indexOf("fail_fast_failed_jobs()"), ); expect(shaMismatch, jobName).toContain("cancel_child"); - expect(shaMismatch, jobName).toContain("trap - EXIT INT TERM"); + expect(shaMismatch, jobName).not.toContain("trap"); } const verify = step(job(full, "summary"), "Verify child workflow results"); diff --git a/test/scripts/resolve-openclaw-ref.test.ts b/test/scripts/resolve-openclaw-ref.test.ts index 8807405ef843..f3f42865e61a 100644 --- a/test/scripts/resolve-openclaw-ref.test.ts +++ b/test/scripts/resolve-openclaw-ref.test.ts @@ -143,4 +143,20 @@ describe("scripts/github/resolve-openclaw-ref.sh", () => { expect(result.stderr).toContain("Ref resolved ambiguously as both branch and tag: ambiguous"); expect(result.stdout).toBe(""); }); + + it("rejects a branch whose resolved SHA differs from expected_sha before emitting outputs", () => { + const expectedSha = "b".repeat(40); + const result = runResolver(remoteRepo, [ + "--ref", + "release/test", + "--expected-sha", + expectedSha, + ]); + + expect(result.status).toBe(1); + expect(result.stdout).toBe(""); + expect(result.stderr).toBe( + `Ref release/test resolved to ${remoteSha}, expected ${expectedSha}.\n`, + ); + }); });