ewilliams28/openclaw
1
0
Fork 0
mirror of https://github.com/openclaw/openclaw.git synced 2026-08-12 21:53:00 -06:00
Code Issues Packages Projects Releases Wiki Activity

fix(release): bound validation retries and soak

Browse Source
This commit is contained in:
Vincent Koc
2026-08-10 20:46:25 +08:00
parent fbebfc713f
commit 1f591bba56
27 changed files with 1399 additions and 279 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/skills/openclaw-live-updater/SKILL.md
+3 -1
View File
@@ -57,10 +57,12 @@ Load `$release-openclaw-ci` and `$openclaw-testing`. This is validation only, ne
2. Only when the cadence is due, confirm no full/all run is active, then snapshot exact current `origin/main` after checking mirror invariants. Run the provider-secret preflight without printing secrets and dispatch the trusted workflow once:
```bash
MAIN_SHA="<exact-main-sha>"
gh workflow run full-release-validation.yml \
--repo openclaw/openclaw \
--ref main \
-f ref=<exact-main-sha> \
-f ref="$MAIN_SHA" \
-f expected_sha="$MAIN_SHA" \
-f provider=openai \
-f mode=both \
-f release_profile=full \
.agents/skills/openclaw-parallels-smoke/SKILL.md
+11
View File
@@ -9,6 +9,17 @@ Use this skill for Parallels guest workflows and smoke interpretation. Do not lo
## Global rules
- Parallels is postpublish-confidence coverage by default, not a prerequisite
for the bounded beta-publish phase. Run it against one exact published
package version and record the Validation SHA + Tooling SHA run tuple that
authorized that package. Validation SHA maps to its Code SHA or Release SHA;
it is not a third release identity.
- A failed postpublish Parallels lane does not retroactively unpublish a beta.
Classify it as product, harness/tooling/provenance, infrastructure/credential,
or wrapper failure. Only a confirmed product defect enters the next beta.
- Use one coordinator and one lane per independent guest family. Diagnose and
retry one failed family once, then reassess; do not restart the aggregate
matrix automatically.
- Inventory existing VMs and snapshots before provisioning anything. When a preconfigured pristine
snapshot matches the requested baseline, switch to it and reuse its user, tools, and base setup.
Do not create a new VM, reinstall macOS, or rebuild the guest baseline for a "fresh" run.
.agents/skills/openclaw-testing/SKILL.md
+31 -26
View File
@@ -288,21 +288,25 @@ rerun after a focused patch.
### Full Release Validation
`Full Release Validation` (`.github/workflows/full-release-validation.yml`) is
the manual product-validation umbrella. Run the full child matrix on the
product-complete pre-changelog **Code SHA**. It resolves a target ref, then
the manual product-validation umbrella. Bind each run to the immutable
**Validation SHA + Tooling SHA** tuple. Validation SHA maps to the Code SHA for
product validation or the Release SHA for changelog-only validation; it is not
a third release identity. The workflow resolves it before child dispatch, then
dispatches:
- manual `CI` for the full normal CI graph, with Android enabled via
`include_android=true`
- `Plugin Prerelease` for release-only plugin static checks, extension shards,
the release-only `agentic-plugins` shard, and plugin product Docker lanes
- `OpenClaw Release Checks` for install smoke, cross-OS release checks, live and
E2E checks, Docker release-path suites, OpenWebUI, QA Lab, fast Matrix, and
Telegram release lanes
- `OpenClaw Release Checks` for install smoke, cross-OS release checks, package
acceptance, and QA parity; broad live/E2E and QA-live lanes join `all` only
when release soak is enabled
- optional post-publish Telegram E2E when a package spec is supplied
Run the full matrix only when validating an actual Code SHA, after broad shared
CI or release orchestration changes, or when explicitly asked:
For beta-publish, use `release_profile=beta` with
`run_release_soak=false`. Postpublish-confidence uses the exact published
package with `run_release_soak=true` or explicit focused groups.
Stable-publish uses `release_profile=stable`.
```bash
node scripts/full-release-validation-at-sha.mjs \
@@ -316,14 +320,12 @@ Validation directly from and against `extended-stable/YYYY.M.33` with
replaced by a `release-ci/*` run. Use `$release-openclaw-ci` for its failure
classification and run-identity rules.
The helper pins the trusted workflow revision on current `main` while targeting
the historical release SHA and recording the canonical release branch as
context. It infers `beta` for alpha/beta package versions and `stable` for
The helper pins the Tooling SHA on trusted `main`, passes the resolved Code SHA
as `expected_sha`, and records the canonical release branch as context. It
infers `beta` for alpha/beta package versions and `stable` for
stable/correction versions. Pass `-f release_profile=full` only for the broad
advisory provider/media sweep. Do not make `full` faster by silently dropping
suites; optimize setup, artifact reuse, and sharding instead. The parent
verifier job appends a child overview plus slowest-job tables for child runs;
rerun only that verifier after a child rerun turns green.
suites; use the bounded phase that matches the release decision.
Standalone manual `CI` dispatches do not run the plugin prerelease suite, the
extension batch sweep, or the release-only `agentic-plugins` Vitest shard. Those
@@ -331,15 +333,16 @@ lanes are intentionally reserved for the separate `Plugin Prerelease` child so
PRs, main pushes, and ad hoc broad CI checks do not spend Docker/package time or
all-plugin runtime time on release-only product coverage.
If a full run is already active on a newer `origin/main`, prefer watching that
run over dispatching a duplicate. Do not cancel release, release-check, or child
workflow runs unless Peter explicitly asks for cancellation.
Use one operator, one transition-only watcher, and at most one investigator for
the current failed surface. Parent timeout or cancellation leaves adopted exact
children running; cancel an exact child only by explicit operator action or the
workflow's identity-mismatch/fail-fast path.
The child-dispatch jobs record the child run ids. The final
`Verify full validation` job re-queries those child runs and is the canonical
parent gate. If a child workflow failed but was later rerun successfully, rerun
only the failed parent verifier job; do not dispatch a new full umbrella unless
the release evidence is stale.
The child-dispatch jobs record child run ids, and `Verify full validation`
re-queries them during that parent attempt. A later narrow green run is useful
recovery evidence but is not publish authorization by itself and there is no
standalone finalizer. The release owner must reassess the recorded evidence and
current publish gate.
Once the Code SHA is green, generate and commit only `CHANGELOG.md`. The new
**Release SHA** is eligible for product-evidence reuse only when GitHub proves
@@ -350,14 +353,16 @@ SHA children. Package, install/update, and release-note proof still runs on the
Release SHA because its tarball bytes changed. Any non-changelog path
invalidates reuse and requires a new Code SHA full matrix.
For bounded recovery after a focused fix, pass `-f rerun_group=<group>`.
For bounded recovery, classify the failure as product,
harness/tooling/provenance, infrastructure/credential, or wrapper before
editing. Only a confirmed product failure changes the Code SHA. Use one
diagnosis, one fix when needed, and one narrow retry with
`-f rerun_group=<group>`, then reassess.
Supported umbrella groups are `all`, `ci`, `plugin-prerelease`,
`release-checks`, `install-smoke`, `cross-os`, `live-e2e`, `package`, `qa`,
`qa-parity`, `qa-live`, and `npm-telegram`. Use the narrowest group that covers
the failed box. After a targeted release-check fix, do not restart the full
umbrella by habit: dispatch the matching `rerun_group` and rerun only the parent
verifier/evidence step after the child is green unless the release evidence is
stale. For a single failed live/E2E shard, use
the failed box. Do not automatically dispatch `all` after a narrow retry. For a
single failed live/E2E shard, use
`-f rerun_group=live-e2e -f live_suite_filter=<suite_id>` so the Blacksmith
workflow only spends setup and queue time on that suite.
.agents/skills/release-openclaw-ci/SKILL.md
+56 -34
View File
@@ -10,8 +10,19 @@ Use this with `$release-openclaw-maintainer` and `$openclaw-testing` when a rele
## Guardrails
- No version bump, tag, npm publish, GitHub release, or release promotion without explicit operator approval.
- After compaction, resume, or new steering, rewrite the effective goal and
current phase from the latest explicit operator instruction. Do not merge old
scope back into the active release.
- Hold the release scope once a release branch or Code SHA exists. Validate and
ship that exact release; do not turn moving `main` into a second work queue.
- Record every active validation run as the immutable tuple **Validation SHA +
Tooling SHA**. Validation SHA maps to the Code SHA for product validation or
the Release SHA for changelog-only validation; it is not a third release
identity. A branch or temporary ref is context and transport.
- Apply a release firebreak after the Code SHA is frozen. Admit only confirmed
product defects, package/provenance defects in the bytes to publish, security
defects, or failures that make publication impossible. Queue other findings
for postpublish confidence or the next beta.
- Use trusted `main` workflow revisions as immutable dispatch sources. Do not
adopt newer main code, repair unrelated main CI, wait for broad main health,
or expand a release fix because the workflow source lives on `main`.
@@ -33,6 +44,9 @@ Use this with `$release-openclaw-maintainer` and `$openclaw-testing` when a rele
- Full Release Validation collects independent child failures to terminal
completion by default. Pass `fail_fast=true` only when the shorter
first-failure cancellation path is preferable.
- Use one release operator, one transition-only watcher, and at most one
investigator for the current failed surface. Do not build audit-review-plan
trees around a single workflow transition.
- For regular beta/stable releases, treat the product-complete pre-changelog
commit as the Code SHA. Full product validation and performance evidence bind
to that SHA. The later Release SHA may reuse those results only when it is a
@@ -65,27 +79,29 @@ Use this with `$release-openclaw-maintainer` and `$openclaw-testing` when a rele
## Run identity and retry budget
Record the target SHA, target ref, parent run id, attempt, and effective
workflow SHA before watching or recovering Full Release Validation.
Record Validation SHA, Tooling SHA, target context ref, parent run id, attempt,
and phase before watching or recovering Full Release Validation. Keep Code SHA
and Release SHA separately in the lifecycle ledger.
- One target SHA has one active canonical `rerun_group=all` parent run by
default.
- Use GitHub's failed-job rerun on the same parent when its original inputs
still select the correct work.
- A new parent for the same target SHA is allowed only when no usable parent
exists, the existing run cannot consume a required trusted-workflow fix, its
evidence identity is invalid, or the operator explicitly requests a fresh
run. The replacement must also use `rerun_group=all`; record which terminal
parent it supersedes and why.
- A narrow `rerun_group` dispatch is supplemental diagnostic proof only. It
never supersedes the canonical parent and cannot satisfy publish evidence.
Run it only after the canonical parent is terminal, then obtain green
`rerun_group=all` evidence before publish.
- Never keep two parents active for the same SHA. Cancel only the superseded
task-owned run after the replacement is identified.
- After two unchanged retries of the same failure, stop repeating it. Recheck
classification and report one precise blocker or the missing evidence needed
for a different action.
- Conceptual phases map to current inputs as follows:
- `beta-publish`: `release_profile=beta`, `run_release_soak=false`
- `postpublish-confidence`: published package inputs with
`run_release_soak=true` or explicit focused groups
- `stable-publish`: `release_profile=stable`
- Keep at most one active parent for the same Validation SHA + Tooling SHA + rerun
group. Concurrency does not cancel an older exact child automatically.
- Parent cancellation or timeout leaves an adopted identity-checked child
running. The operator must cancel that exact child explicitly when it is no
longer useful.
- Recover one failed surface with one diagnosis, one fix when needed, and one
narrow retry. Then reassess the release decision. Do not automatically
dispatch `rerun_group=all`.
- A new all-group parent is justified only when shared orchestration changed,
earlier evidence is invalid for the selected tuple, or the operator explicitly
requests it. Record the invalidating event.
- Narrow child or group evidence does not by itself become publish
authorization. Keep it in the evidence ledger for the release owner to judge
against the current publish gate.
## Preflight
@@ -163,10 +179,11 @@ dispatching child lanes. Npm preflight and package/install acceptance still run
against the exact Release SHA and its new tarball bytes.
The SHA-pinned helper infers `beta` for alpha/beta package versions and `stable`
for stable/correction versions. Pass `release_profile=full` only when the
operator explicitly asks for the broad advisory provider/media matrix. Stable
and full profiles force the release soak; the beta profile may opt in with
`run_release_soak=true`. Use narrow `rerun_group` after focused fixes.
for stable/correction versions and passes the Validation SHA + Tooling SHA run
identity. `beta` without soak is the bounded beta-publish gate. Run broad live
QA and E2E as postpublish confidence with `run_release_soak=true` or explicit
groups. Stable and full profiles force the release soak. Use a narrow
`rerun_group` after focused fixes; never widen automatically.
Publish with `openclaw-release-publish.yml` using `release_profile=from-validation`
unless a maintainer intentionally wants to cross-check a specific profile; the
publish workflow reads the effective profile from the full-validation manifest.
@@ -177,9 +194,11 @@ For `.33+`, dispatch from and target the canonical branch; the regular
SHA-pinned helper would produce a rejected `release-ci/*` identity:
```bash
RELEASE_SHA="$(git rev-parse HEAD)"
gh workflow run full-release-validation.yml \
--ref extended-stable/YYYY.M.33 \
-f ref=extended-stable/YYYY.M.33 \
-f expected_sha="$RELEASE_SHA" \
-f release_profile=stable
```
@@ -225,18 +244,21 @@ Stop watchers before ending the turn or switching strategy.
Anthropic API-key lane.
5. For live-cache failures, inspect whether it is missing/invalid key, empty text, provider refusal, timeout, or baseline miss. Do not weaken release gates without clear provider evidence.
6. Classify before editing:
- product/code failure: fix the release branch, freeze a new Code SHA, run
focused proof, then obtain green full validation for that new SHA
- workflow/harness/infrastructure/credential failure: fix the smallest
owning surface and rerun failed jobs on the current parent when its inputs
still select the correct work; otherwise supersede the terminal parent once
with a new `rerun_group=all` parent on the required trusted-workflow
revision. Keep the same Code SHA; touching `main` still requires the active
release scope lock
- confirmed product/code failure: fix the release branch, freeze a new Code
SHA, and invalidate product evidence
- harness/tooling/provenance failure: keep the Code SHA, fix the smallest
owning surface, and retry only the failed surface with the required Tooling
SHA
- infrastructure/credential failure: keep both SHAs, repair the external
prerequisite, and retry only the failed surface
- wrapper/monitor failure: keep the child and candidate identities; record
the wrapper result separately from the child result
- changelog/release-note failure: change only `CHANGELOG.md`, keep Code SHA
evidence, and repeat Release SHA proof
- publish child/registry selector failure: keep Release SHA and resume the
failed child; never rebuild an immutable version that already published
Only the first class changes the Code SHA. After one diagnosis/fix/narrow
retry, reassess instead of starting another all-group cycle.
7. If a required PR CI run is capacity-stalled with queued jobs and no active
jobs, do not cancel unrelated work or accept a generic manual dispatch.
First verify the PR head carries the current fallback schema:
@@ -261,8 +283,8 @@ include_android=true -f release_gate=true`.
Record:
- release identity: Code/Release SHAs for regular releases; canonical branch,
exact SHA, and immutable tag for extended-stable
- release lifecycle ledger: Code SHA, Release SHA, and Tooling SHA for regular
releases; canonical branch, exact SHA, and immutable tag for extended-stable
- evidence-reuse policy and complete changed-path set
- active full parent run URL, attempt, workflow SHA, and any superseded parent
with the exact replacement reason
.agents/skills/release-openclaw-ci/references/release-ci-notes.md
+10 -8
View File
@@ -18,14 +18,16 @@
- Fetch failed-job logs only after a job reaches a terminal failing state.
- Prefer same-parent failed-job reruns when the original inputs still select the
right work.
- Keep one active parent per target SHA. Create a replacement only when the
existing parent cannot consume a required workflow fix or its evidence
identity is invalid; record the superseded run and reason. The replacement
remains `rerun_group=all`.
- Treat narrow `rerun_group` dispatches as supplemental diagnostics only; they
do not satisfy publish evidence.
- Stop after two unchanged retries and report the exact blocker instead of
creating another verification loop.
- Keep one active parent per exact Validation SHA + Tooling SHA + rerun group. Create
a replacement only when the current evidence is invalid or cannot consume a
required workflow fix; record the invalidating event and do not widen
automatically to `rerun_group=all`.
- Classify one failed surface, make one fix when needed, and retry the narrowest
failed group once. Then reassess whether to ship, explicitly waive, or block
instead of creating another verification loop.
- Preserve successful exact-tuple evidence when the documented finalization
rules allow reuse. Narrow evidence does not become publish authorization by
itself, and there is no standalone rerunnable finalizer today.
- Leave bad secrets unset. A 401 candidate from 1Password should not overwrite GitHub.
- Make the final release evidence note durable: parent URL, child run URLs, SHA, command proof, and gaps.
.agents/skills/release-openclaw-maintainer/SKILL.md
+59 -36
View File
@@ -17,7 +17,7 @@ Before validation or publication, write one compact state record and keep it
current:
- goal and terminal success criteria
- release version, tag, branch, cut SHA, Code SHA, and Release SHA
- release version, tag, branch, cut SHA, Code SHA, Tooling SHA, and Release SHA
- active Full Release Validation parent run id and attempt
- npm preflight and publish parent run ids
- completed phases and immutable child artifacts
@@ -26,6 +26,9 @@ current:
Use `references/release-handoff-template.md` when starting a release session,
recovering after compaction, or handing the release to another operator.
After compaction, resume, or new steering, the latest explicit operator
instruction replaces the effective goal and phase; do not merge superseded
scope back into the release.
Completed phases stay complete. Reopen one only when a named event invalidates
its evidence, such as a Code SHA change, a non-changelog Release SHA change, or
a workflow fix that the existing parent run cannot consume.
@@ -94,6 +97,14 @@ a workflow fix that the existing parent run cannot consume.
infrastructure, repair the smallest owning surface and rerun against the same
Code SHA. Touch `main` only under the active release scope lock above. Never
mutate the release candidate to satisfy newer tooling or heal unrelated main.
- Apply a release firebreak after the Code SHA is frozen. Admit only confirmed
product defects, package/provenance defects in the bytes to publish, security
defects, or failures that make publication impossible. Queue adjacent
improvements and broad confidence findings for the next beta or postpublish
work.
- Operate with one release owner, one transition-only watcher, and at most one
investigator for the current failed surface. One diagnosis, one fix when
needed, and one narrow retry consume the failure budget; then reassess.
- Generate `CHANGELOG.md` only after the Code SHA is green. The resulting
**Release SHA** must be a descendant whose complete diff from the Code SHA is
exactly `CHANGELOG.md`. Release-note checks, npm preflight/package bytes,
@@ -743,10 +754,10 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts <published-version>
- Parallels validation and any local live model QA for this train must use both
`OPENAI_API_KEY` and `ANTHROPIC_API_KEY`. If either cannot be injected, stop
before starting those local long lanes and report the missing key.
- Live credentialed channel QA is the GitHub Actions workflow
- Postpublish credentialed channel QA is the GitHub Actions workflow
`QA-Lab - All Lanes` (`.github/workflows/qa-live-transports-convex.yml`), not a
local substitute. Dispatch it from Actions against the release tag and wait
for it to pass before npm preflight/publish readiness. Use a SHA only when it
local substitute. Dispatch it from Actions against the published release tag
during postpublish confidence. Use a SHA only when it
satisfies the workflow's secret-bearing trust gate: main ancestor or open PR
head. It runs the QA Lab mock parity gate plus live Matrix and live Telegram
lanes using the `qa-live-shared` environment; Telegram uses Convex CI
@@ -759,21 +770,15 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts <published-version>
- `pnpm ui:build`
- `pnpm release:check`
- `OPENCLAW_INSTALL_SMOKE_SKIP_NONROOT=1 pnpm test:install:smoke`
- Full pre-npm beta test roster:
- default release checks above
- all Docker tests: `pnpm test:docker:all`, plus standalone Docker live lanes
not covered by the aggregate when operator says "all docker tests":
`pnpm test:docker:live-acp-bind`, `pnpm test:docker:live-cli-backend`, and
`pnpm test:docker:live-codex-harness`
- all Parallels install/update tests:
`pnpm test:parallels:npm-update -- --json` plus any needed individual
rerun lanes from `openclaw-parallels-smoke`
- all QA release validation: dispatch GitHub Actions > `QA-Lab - All Lanes`
against the release tag and require success. This is the release gate for
live credentialed Matrix/Telegram channel coverage. Use a SHA only when it
satisfies the workflow trust gate. Run local OpenAI/Anthropic suites or
repo-backed character evals only when the operator asks for extra model
coverage or a failure needs local debugging.
- Release validation phases:
- `beta-publish`: `release_profile=beta`, `run_release_soak=false`. This is
the bounded prepublish gate; it excludes broad live/E2E, QA-live, and
Parallels confidence work.
- `postpublish-confidence`: run against the exact published beta package with
`run_release_soak=true` or explicit focused groups. This is the default home
for QA-live, broad Docker/live E2E, mobile, and Parallels.
- `stable-publish`: `release_profile=stable`; require the stable publish
roster and accepted confidence evidence.
- Post-published beta verification roster:
- `node --import tsx scripts/openclaw-npm-postpublish-verify.ts <beta-version>`
- install/update smoke against the published beta channel
@@ -794,8 +799,8 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts <published-version>
operator explicitly scopes a harness-only isolation check; a lane that
disables bundled plugin installs is not valid plugin/dependency release
evidence.
- targeted QA reruns only for areas touched by fixes after the full pre-npm
roster, unless the operator requests the full QA roster again. If the fix
- targeted QA reruns only for areas touched by fixes after the beta-publish
gate, unless the operator requests the full QA roster again. If the fix
touches live channel QA, credential plumbing, Matrix, Telegram, or the QA
harness, rerun Actions > `QA-Lab - All Lanes`.
- Check all release-related build surfaces touched by the release, not only the npm package.
@@ -837,7 +842,10 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts <published-version>
Docker/Parallels matrix unless the beta evidence is stale, the stable build
differs materially from beta, or the operator explicitly asks for full
retesting.
- If any required build, packaging step, or release workflow is red, do not say the release is ready.
- A red required surface blocks readiness only after classification shows that
it invalidates the current phase or publish gate. Report harness, provenance,
infrastructure, credential, and wrapper failures separately from confirmed
product failures.
## Use the right auth flow
@@ -1018,9 +1026,12 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts <published-version>
8. Run the deterministic source preflight, then Full Release Validation against
the exact Code SHA with
`node scripts/full-release-validation-at-sha.mjs --sha <code-sha> --target-ref release/YYYY.M.PATCH`.
Use one transition watcher. Product failures return to step 6 with a new
Code SHA; tooling/harness failures are fixed separately and rerun against the
same Code SHA.
For beta-publish, keep `release_profile=beta` and
`run_release_soak=false`. Record the Validation SHA + Tooling SHA tuple
(Validation SHA is the Code SHA in this phase) and use one transition
watcher. Product failures return to step 6 with a new Code SHA;
tooling/harness/provenance, infrastructure/credential, and wrapper failures
keep the Code SHA and rerun only the failed surface.
9. After the Code SHA is green, run `/changelog` once for the stable-base target
version using current `origin/main` for canonical PR provenance. Keep the
heading as `## YYYY.M.PATCH`, not `## YYYY.M.PATCH-beta.N`.
@@ -1046,7 +1057,10 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts <published-version>
<release-sha-validation-run-id> --npm-preflight-run <preflight-run-id>
--skip-dispatch` to consume the existing reused full evidence and exact
Release SHA preflight instead of dispatching either again. It completes
package/install proof and prints the publish command.
package/install proof and prints the publish command. Beta and alpha
candidates defer Parallels to postpublish `pnpm release:beta-smoke` by
default; stable/full candidates run it prepublish. Use `--run-parallels` or
`--skip-parallels` only for an explicit operator override.
18. Start publication only after the candidate bundle is green. Reuse successful
immutable child runs/artifacts on retry; do not rebuild or republish versions
that already succeeded.
@@ -1063,12 +1077,15 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts <published-version>
`openclaw/releases/.github/workflows/openclaw-macos-publish.yml` with
`preflight_only=true` and wait for it to pass. Save that run id because the
real publish requires it to reuse the notarized mac artifacts.
23. Classify every failure before changing git state. Product defects return to
23. Classify every failure before changing git state. Confirmed product defects return to
step 6 and invalidate downstream Code/Release SHA evidence. Changelog or
release-note defects change only the Release SHA and reuse the green Code
SHA evidence after the exact delta is reverified. Tooling, credential,
approval, registry selector, or publication-child failures keep the
candidate unchanged and resume the smallest failed surface.
SHA evidence after the exact delta is reverified. Tooling, provenance,
credential, infrastructure, wrapper, approval, registry selector, or
publication-child failures keep the candidate unchanged and resume the
smallest failed surface. Only confirmed product failure changes the Code
SHA. After one diagnosis/fix/narrow retry, reassess; do not automatically
rerun `all`.
24. Start `.github/workflows/openclaw-release-publish.yml` from the exact pinned
trusted workflow source
with the same tag for the real beta or stable publish, choose `npm_dist_tag` (`beta` default,
@@ -1099,17 +1116,23 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts <published-version>
27. Run the post-published beta verification roster. Do not scan current `main`
for extra fixes unless the operator explicitly requests a backport audit.
Apply only operator-selected backports, and increment to the next beta if a
selected fix must change the already-published package. If any
lane fails after the beta package is published, fix, commit/push/pull,
increment to the next beta tag, and rerun the affected beta evidence. Once
selected fix must change the already-published package. A failed confidence
lane does not retroactively unpublish the beta. Classify it first; only a
confirmed product defect admitted by the release owner creates a fix,
increments the next beta tag, and reruns affected evidence. Once
the beta is live, start remote/manual rosters where they
can overlap safely, but keep local Docker and Parallels load controlled.
Ensure the full expensive roster has passed at least once before
stable/latest promotion. The roster includes the manual Actions >
`NPM Telegram Beta E2E` workflow against the exact published beta package.
If a pre-npm lane fails before any tag/package leaves the machine, fix and
rerun the same intended beta attempt. Repeat up to the operator's
authorized beta-attempt limit, normally 4.
If a pre-npm lane fails before any tag/package leaves the machine, classify
it first and use one diagnosis, one fix when needed, and one narrow retry.
Only an operator-admitted confirmed product defect may create a new intended
beta attempt. Harness/tooling/provenance, infrastructure/credential, and
wrapper failures keep the candidate identity and do not authorize another
all-group cycle. The operator's separate beta-attempt cap, normally 4,
remains a ceiling for admitted product attempts, not an automatic retry
budget.
28. Announce the beta/stable release on Discord best-effort using the configured secret workflow.
29. If the operator requested beta only, stop after beta verification and the
announcement.
.agents/skills/release-openclaw-maintainer/references/release-handoff-template.md
+22 -6
View File
@@ -7,6 +7,8 @@ captured by a durable run or artifact URL.
## Goal
Ship `<version>` on `<channel>` and stop when `<terminal success criteria>`.
After compaction or resume, replace this section with the latest explicit
operator steering. Do not preserve superseded scope.
## Immutable state
@@ -14,12 +16,14 @@ Ship `<version>` on `<channel>` and stop when `<terminal success criteria>`.
- branch: `<release/YYYY.M.PATCH | extended-stable/YYYY.M.33>`
- cut SHA: `<full sha>`
- Code SHA: `<regular release full sha | not applicable>`
- Tooling SHA: `<trusted workflow full sha>`
- Release SHA: `<regular release full sha | exact extended-stable branch tip>`
- tag: `v<version>`
- workflow ref: `<release-ci ref | canonical branch>`
- publication inventory: `<exact surfaces>`
- approved backports: `<none or exact PRs/commits>`
- approved main changes: `<none or exact blocker>`
- admitted release blockers: `<confirmed product/package/provenance/security blockers only>`
- frozen-target compatibility repairs: `<none or exact PRs/invariants>`
## Active evidence
@@ -34,27 +38,39 @@ Ship `<version>` on `<channel>` and stop when `<terminal success criteria>`.
## Phase
- conceptual phase: `<beta-publish | postpublish-confidence | stable-publish>`
- current input mapping: `<beta + no soak | published package + soak/focused groups | stable>`
- completed: `<phases that stay complete>`
- current: `<one phase>`
- next action: `<one concrete action>`
- roles: `<one operator | one transition watcher | zero or one current-failure investigator>`
- retry budget: `<one diagnosis/fix/narrow retry, then reassess>`
## Failure policy
- regular product/code failure: fix the release branch, freeze a new Code SHA,
and invalidate downstream product evidence
- confirmed product/code failure: fix the release branch, freeze a new Code
SHA, and invalidate downstream product evidence
- regular changelog-only failure: change only `CHANGELOG.md`, freeze a new
Release SHA, and reuse green Code SHA evidence after delta proof
- extended-stable branch change: land the approved product/changelog change or
smallest frozen-target repair by PR, record its source/invariant, and replace
all exact-head evidence
- workflow/tooling/credential failure: keep the candidate frozen and recover
the smallest owning surface
- harness/tooling/provenance failure: keep the Code SHA, change the Tooling SHA
only when needed, and recover the smallest owning surface
- infrastructure/credential failure: keep both SHAs and repair the external
prerequisite
- wrapper/monitor failure: record parent and child conclusions separately;
parent cancellation leaves adopted children running until the operator
cancels them explicitly
- postpublish-confidence failure: do not retroactively unpublish the beta;
admit a confirmed product fix to the next beta
- external approval or permission blocker: stop with the exact job, URL,
missing permission, and required operator action
Do not scan moving `main`, add optional backports, dispatch a replacement
validation parent, or repeat completed phases unless a named invalidating event
requires it.
validation parent, automatically rerun `all`, or repeat completed phases unless
a named invalidating event requires it. Narrow evidence informs the release
decision but is not publish authorization by itself.
## Stop conditions
.agents/skills/release-openclaw-nightly/SKILL.md
+1
View File
@@ -226,6 +226,7 @@ BRANCH="$(git branch --show-current)"
"$GH" workflow run full-release-validation.yml --repo openclaw/openclaw --ref "$BRANCH" \
-f ref="$BRANCH" \
-f expected_sha="$SHA" \
-f release_profile=beta \
-f rerun_group=all
.github/workflows/full-release-validation.yml
+34 -24
View File
@@ -8,6 +8,11 @@ on:
required: true
default: main
type: string
expected_sha:
description: Optional full Validation SHA that ref must resolve to
required: false
default: ""
type: string
target_context_ref:
description: Optional canonical release branch or tag context for an exact-SHA target
required: false
@@ -143,8 +148,8 @@ permissions:
contents: read
concurrency:
group: full-release-validation-${{ inputs.ref }}-${{ inputs.rerun_group }}
cancel-in-progress: ${{ (inputs.ref == 'main' && inputs.rerun_group == 'all') || startsWith(inputs.ref, 'tideclaw/alpha/') || startsWith(inputs.ref, 'release/') }}
group: full-release-validation-${{ inputs.expected_sha || inputs.ref }}-${{ github.sha }}-${{ inputs.rerun_group }}
cancel-in-progress: false
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
@@ -175,9 +180,11 @@ jobs:
env:
TARGET_REF: ${{ inputs.ref }}
TARGET_CONTEXT_REF: ${{ inputs.target_context_ref }}
EXPECTED_SHA: ${{ inputs.expected_sha }}
run: |
bash workflow/scripts/github/resolve-openclaw-ref.sh \
--ref "$TARGET_REF" \
--expected-sha "$EXPECTED_SHA" \
--github-output "$GITHUB_OUTPUT"
- name: Validate release inputs
@@ -195,7 +202,7 @@ jobs:
env:
TARGET_REF: ${{ inputs.ref }}
TARGET_SHA: ${{ steps.resolve.outputs.sha }}
CHILD_WORKFLOW_REF: ${{ github.ref_name }}
TOOLING_SHA: ${{ github.sha }}
NPM_TELEGRAM_PACKAGE_SPEC: ${{ inputs.npm_telegram_package_spec }}
RELEASE_PACKAGE_SPEC: ${{ inputs.release_package_spec }}
EVIDENCE_PACKAGE_SPEC: ${{ inputs.evidence_package_spec }}
@@ -214,8 +221,8 @@ jobs:
echo "## Full release validation"
echo
echo "- Target ref: \`${TARGET_REF}\`"
echo "- Target SHA: \`${TARGET_SHA}\`"
echo "- Child workflow ref: \`${CHILD_WORKFLOW_REF}\`"
echo "- Validation SHA: \`${TARGET_SHA}\`"
echo "- Tooling SHA: \`${TOOLING_SHA}\`"
echo "- Release soak lanes: \`${RUN_RELEASE_SOAK}\`"
echo "- Fail fast: \`${FAIL_FAST}\`"
echo "- Package Acceptance Telegram E2E deferred: \`${SKIP_PACKAGE_TELEGRAM_E2E}\`"
@@ -456,7 +463,8 @@ jobs:
needs: [resolve_target, evidence_reuse]
if: ${{ always() && needs.resolve_target.result == 'success' && contains(fromJSON('["all","ci"]'), inputs.rerun_group) && needs.evidence_reuse.outputs.reuse != 'true' }}
runs-on: ubuntu-24.04
timeout-minutes: ${{ inputs.release_profile != 'beta' && 240 || 60 }}
# The child owns lane timeouts; this monitor also covers queue delay and iOS.
timeout-minutes: 240
outputs:
run_id: ${{ steps.dispatch.outputs.run_id }}
url: ${{ steps.dispatch.outputs.url }}
@@ -678,23 +686,23 @@ jobs:
fi
cancel_child() {
if [[ -n "${active_child_run_id:-}" ]]; then
echo "Cancelling child workflow ${active_child_workflow}: ${active_child_run_id}" >&2
gh run cancel "$active_child_run_id" >/dev/null 2>&1 || true
if [[ -n "${run_id:-}" ]]; then
echo "Cancelling child workflow ${workflow}: ${run_id}" >&2
gh run cancel "$run_id" >/dev/null 2>&1 || true
fi
}
run_json="$(validate_child_run "$run_id")"
# Identity fields prove cancellation ownership before workflow-SHA validation.
# EXIT traps run after function locals unwind; preserve only adopted child identity.
active_child_workflow="$workflow"
active_child_run_id="$run_id"
trap cancel_child EXIT INT TERM
# Generic monitor failures and parent cancellation leave an
# identity-checked child running for explicit operator recovery.
{
echo "- Adopted child: \`${workflow}\` run \`${run_id}\`"
echo "- Parent cancellation leaves this child running; cancel it explicitly if no longer needed."
} >> "$GITHUB_STEP_SUMMARY"
child_head_sha="$(jq -r '.head_sha // ""' <<< "$run_json")"
if [[ "$child_head_sha" != "$PARENT_WORKFLOW_SHA" ]]; then
echo "::error::${workflow} child run used workflow SHA ${child_head_sha}, expected parent workflow SHA ${PARENT_WORKFLOW_SHA}." >&2
cancel_child
trap - EXIT INT TERM
exit 1
fi
@@ -760,7 +768,6 @@ jobs:
fi
jq '.[] | {name, conclusion, url: (.url // .html_url)}' <<< "$failed_jobs_json"
cancel_child
trap - EXIT INT TERM
exit 1
fi
}
@@ -781,7 +788,6 @@ jobs:
fi
sleep 60
done
trap - EXIT INT TERM
if [[ "$workflow" == "openclaw-release-checks.yml" ]]; then
jobs_json="$(fetch_child_jobs | jq -s '{jobs: [.[] | {name, conclusion, url: .html_url}]}')"
@@ -983,7 +989,8 @@ jobs:
needs: [resolve_target, evidence_reuse, prepare_release_candidate]
if: ${{ always() && needs.resolve_target.result == 'success' && (needs.prepare_release_candidate.result == 'success' || needs.prepare_release_candidate.result == 'skipped') && contains(fromJSON('["all","plugin-prerelease"]'), inputs.rerun_group) && needs.evidence_reuse.outputs.reuse != 'true' }}
runs-on: ubuntu-24.04
timeout-minutes: ${{ inputs.release_profile == 'full' && 300 || inputs.release_profile == 'stable' && 240 || 60 }}
# The child owns lane timeouts; this monitor also covers queue delay.
timeout-minutes: ${{ inputs.release_profile == 'full' && 300 || 240 }}
outputs:
run_id: ${{ steps.dispatch.outputs.run_id }}
url: ${{ steps.dispatch.outputs.url }}
@@ -1006,10 +1013,10 @@ jobs:
name: Run release/live/Docker/QA validation
needs: [resolve_target, evidence_reuse, prepare_release_candidate]
if: ${{ always() && needs.resolve_target.result == 'success' && (needs.prepare_release_candidate.result == 'success' || needs.prepare_release_candidate.result == 'skipped') && contains(fromJSON('["all","release-checks","install-smoke","cross-os","live-e2e","package","qa","qa-parity","qa-live"]'), inputs.rerun_group) && needs.evidence_reuse.outputs.reuse != 'true' }}
runs-on: ubuntu-24.04
# The child owns lane timeouts; this monitor must also tolerate queue delay
# so it does not cancel healthy release checks before their final verifier.
timeout-minutes: 240
runs-on: blacksmith-4vcpu-ubuntu-2404
# The bounded package critical path tops out at 310 minutes; 420 leaves
# queue/API margin. Parent timeout preserves the adopted child for exact cancellation.
timeout-minutes: 420
outputs:
run_id: ${{ steps.dispatch.outputs.run_id }}
url: ${{ steps.dispatch.outputs.url }}
@@ -1047,7 +1054,8 @@ jobs:
if: ${{ always() && needs.resolve_target.result == 'success' && contains(fromJSON('["all","npm-telegram"]'), inputs.rerun_group) && (inputs.npm_telegram_package_spec != '' || inputs.release_package_spec != '') && needs.evidence_reuse.outputs.reuse != 'true' }}
continue-on-error: ${{ startsWith(github.ref, 'refs/heads/tideclaw/alpha/') }}
runs-on: ubuntu-24.04
timeout-minutes: ${{ inputs.release_profile == 'full' && 360 || 60 }}
# The child owns lane timeouts; this monitor also covers queue delay.
timeout-minutes: ${{ inputs.release_profile == 'full' && 360 || 120 }}
outputs:
run_id: ${{ steps.dispatch.outputs.run_id }}
url: ${{ steps.dispatch.outputs.url }}
@@ -1074,7 +1082,9 @@ jobs:
# Keep this monitor off the four-slot GitHub-hosted pool so performance starts
# with the other child workflows instead of extending the critical path.
runs-on: blacksmith-4vcpu-ubuntu-2404
timeout-minutes: ${{ inputs.release_profile == 'full' && 360 || 120 }}
# Artifact-only and publish paths top out at 255 and 280 minutes; 360 leaves
# queue/API margin. Parent timeout preserves the adopted child for exact cancellation.
timeout-minutes: 360
outputs:
run_id: ${{ steps.dispatch.outputs.run_id }}
url: ${{ steps.dispatch.outputs.url }}
.github/workflows/install-smoke-reusable.yml
+8
View File
@@ -38,6 +38,7 @@ env:
jobs:
preflight:
runs-on: ubuntu-24.04
timeout-minutes: 15
outputs:
docs_only: ${{ steps.manifest.outputs.docs_only }}
run_install_smoke: ${{ steps.manifest.outputs.run_install_smoke }}
@@ -118,6 +119,7 @@ jobs:
needs: [preflight]
if: needs.preflight.outputs.run_fast_install_smoke == 'true' && needs.preflight.outputs.run_full_install_smoke != 'true'
runs-on: ubuntu-24.04
timeout-minutes: 120
env:
DOCKER_BUILD_SUMMARY: "false"
DOCKER_BUILD_RECORD_UPLOAD: "false"
@@ -228,6 +230,7 @@ jobs:
needs: [preflight]
if: needs.preflight.outputs.run_full_install_smoke == 'true'
runs-on: ubuntu-24.04
timeout-minutes: 60
permissions:
contents: read
packages: read
@@ -332,6 +335,7 @@ jobs:
needs: [preflight, root_dockerfile_image]
if: always() && needs.preflight.result == 'success' && needs.preflight.outputs.run_full_install_smoke == 'true'
runs-on: ubuntu-24.04
timeout-minutes: 5
permissions:
contents: read
steps:
@@ -349,6 +353,7 @@ jobs:
needs: [preflight]
if: needs.preflight.outputs.run_full_install_smoke == 'true'
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- name: Checkout CLI
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
@@ -365,6 +370,7 @@ jobs:
needs: [preflight, root_dockerfile_image, root_dockerfile_image_ready]
if: needs.preflight.outputs.run_full_install_smoke == 'true'
runs-on: ubuntu-24.04
timeout-minutes: 90
env:
OPENCLAW_DOCKER_E2E_REQUIRE_LOCAL_IMAGE: "1"
steps:
@@ -534,6 +540,7 @@ jobs:
needs: [preflight, root_dockerfile_image, root_dockerfile_image_ready]
if: needs.preflight.outputs.run_full_install_smoke == 'true'
runs-on: ubuntu-24.04
timeout-minutes: 150
env:
DOCKER_BUILD_SUMMARY: "false"
DOCKER_BUILD_RECORD_UPLOAD: "false"
@@ -700,6 +707,7 @@ jobs:
needs: [preflight, root_dockerfile_image, root_dockerfile_image_ready]
if: needs.preflight.outputs.run_full_install_smoke == 'true' && needs.preflight.outputs.run_bun_global_install_smoke == 'true'
runs-on: ubuntu-24.04
timeout-minutes: 60
env:
OPENCLAW_DOCKER_E2E_REQUIRE_LOCAL_IMAGE: "1"
steps:
.github/workflows/openclaw-cross-os-release-checks-reusable.yml
+1
View File
@@ -249,6 +249,7 @@ env:
jobs:
prepare:
runs-on: ubuntu-24.04
timeout-minutes: 90
continue-on-error: ${{ inputs.advisory }}
outputs:
baseline_artifact_digest: ${{ steps.upload_baseline.outputs.artifact-digest }}
.github/workflows/openclaw-live-and-e2e-checks-reusable.yml
+3
View File
@@ -542,6 +542,7 @@ env:
jobs:
validate_selected_ref:
runs-on: ubuntu-24.04
timeout-minutes: 30
outputs:
selected_sha: ${{ steps.validate.outputs.selected_sha }}
trusted_reason: ${{ steps.validate.outputs.trusted_reason }}
@@ -776,6 +777,7 @@ jobs:
validate_live_suite_filter:
runs-on: ubuntu-24.04
if: inputs.live_suite_filter != ''
timeout-minutes: 10
steps:
- name: Validate focused live suite filter
shell: bash
@@ -892,6 +894,7 @@ jobs:
plan_release_workflow_matrices:
needs: validate_selected_ref
runs-on: ubuntu-24.04
timeout-minutes: 10
outputs:
docker_e2e_count: ${{ steps.plan.outputs.docker_e2e_count }}
docker_e2e_matrix: ${{ steps.plan.outputs.docker_e2e_matrix }}
.github/workflows/openclaw-release-checks.yml
+19 -9
View File
@@ -143,6 +143,7 @@ jobs:
mode: ${{ steps.inputs.outputs.mode }}
release_profile: ${{ steps.inputs.outputs.release_profile }}
run_release_soak: ${{ steps.inputs.outputs.run_release_soak }}
qa_live_scheduled: ${{ steps.inputs.outputs.qa_live_scheduled }}
fail_fast: ${{ steps.inputs.outputs.fail_fast }}
run_maturity_scorecard: ${{ steps.inputs.outputs.run_maturity_scorecard }}
allow_unreleased_changelog: ${{ steps.inputs.outputs.allow_unreleased_changelog }}
@@ -430,10 +431,10 @@ jobs:
codex_plugin_spec="npm:@openclaw/codex@${BASH_REMATCH[1]}"
fi
qa_filter_seen=false
filter="$(printf '%s' "$RELEASE_LIVE_SUITE_FILTER_INPUT" | tr '[:upper:]' '[:lower:]')"
repo_live_suite_filter="$filter"
if [[ -n "${filter// }" ]]; then
qa_filter_seen=false
repo_filter_tokens=()
matrix_selected=false
buzz_selected=false
@@ -525,6 +526,12 @@ jobs:
fi
fi
qa_live_scheduled=false
if [[ "$RELEASE_RERUN_GROUP_INPUT" == "qa" || "$RELEASE_RERUN_GROUP_INPUT" == "qa-live" ]] ||
[[ "$RELEASE_RERUN_GROUP_INPUT" == "all" && ( "$run_release_soak" == "true" || "$qa_filter_seen" == "true" ) ]]; then
qa_live_scheduled=true
fi
cross_os_scheduled=false
if [[ "$RELEASE_RERUN_GROUP_INPUT" == "all" || "$RELEASE_RERUN_GROUP_INPUT" == "cross-os" ]]; then
cross_os_scheduled=true
@@ -542,6 +549,7 @@ jobs:
printf 'mode=%s\n' "$RELEASE_MODE_INPUT"
printf 'release_profile=%s\n' "$release_profile"
printf 'run_release_soak=%s\n' "$run_release_soak"
printf 'qa_live_scheduled=%s\n' "$qa_live_scheduled"
printf 'fail_fast=%s\n' "$fail_fast"
printf 'run_maturity_scorecard=%s\n' "$run_maturity_scorecard"
printf 'allow_unreleased_changelog=%s\n' "$allow_unreleased_changelog"
@@ -572,6 +580,7 @@ jobs:
RELEASE_MODE: ${{ inputs.mode }}
RELEASE_PROFILE: ${{ steps.inputs.outputs.release_profile }}
RUN_RELEASE_SOAK: ${{ steps.inputs.outputs.run_release_soak }}
QA_LIVE_SCHEDULED: ${{ steps.inputs.outputs.qa_live_scheduled }}
FAIL_FAST: ${{ steps.inputs.outputs.fail_fast }}
RUN_MATURITY_SCORECARD: ${{ steps.inputs.outputs.run_maturity_scorecard }}
SKIP_PACKAGE_TELEGRAM_E2E: ${{ steps.inputs.outputs.skip_package_telegram_e2e }}
@@ -593,6 +602,7 @@ jobs:
echo "- Cross-OS mode: \`${RELEASE_MODE}\`"
echo "- Release profile: \`${RELEASE_PROFILE}\`"
echo "- Release soak lanes: \`${RUN_RELEASE_SOAK}\`"
echo "- QA-live scheduled: \`${QA_LIVE_SCHEDULED}\`"
echo "- Matrix QA fail fast: \`${FAIL_FAST}\`"
echo "- Maturity scorecard docs: \`${RUN_MATURITY_SCORECARD}\`"
echo "- Package Acceptance Telegram E2E deferred: \`${SKIP_PACKAGE_TELEGRAM_E2E}\`"
@@ -604,7 +614,7 @@ jobs:
if [[ -n "${RELEASE_CROSS_OS_SUITE_FILTER// }" ]]; then
echo "- Cross-OS suite filter: \`${RELEASE_CROSS_OS_SUITE_FILTER}\`"
fi
echo "- QA live lanes: Matrix \`${{ steps.inputs.outputs.qa_live_matrix_enabled }}\`, Buzz \`${{ steps.inputs.outputs.qa_live_buzz_enabled }}\`, Telegram \`${{ steps.inputs.outputs.qa_live_telegram_enabled }}\`, Discord \`${{ steps.inputs.outputs.qa_live_discord_enabled }}\`, WhatsApp \`${{ steps.inputs.outputs.qa_live_whatsapp_enabled }}\`, Slack \`${{ steps.inputs.outputs.qa_live_slack_enabled }}\`"
echo "- QA-live lane eligibility: Matrix \`${{ steps.inputs.outputs.qa_live_matrix_enabled }}\`, Buzz \`${{ steps.inputs.outputs.qa_live_buzz_enabled }}\`, Telegram \`${{ steps.inputs.outputs.qa_live_telegram_enabled }}\`, Discord \`${{ steps.inputs.outputs.qa_live_discord_enabled }}\`, WhatsApp \`${{ steps.inputs.outputs.qa_live_whatsapp_enabled }}\`, Slack \`${{ steps.inputs.outputs.qa_live_slack_enabled }}\`"
if [[ -n "${RELEASE_PACKAGE_SPEC// }" ]]; then
echo "- Release package spec: \`${RELEASE_PACKAGE_SPEC}\`"
fi
@@ -1916,7 +1926,7 @@ jobs:
qa_live_release_checks:
name: Run QA Live Matrix catalog
needs: [resolve_target]
if: contains(fromJSON('["all","qa","qa-live"]'), needs.resolve_target.outputs.rerun_group) && needs.resolve_target.outputs.qa_live_matrix_enabled == 'true'
if: needs.resolve_target.outputs.qa_live_scheduled == 'true' && needs.resolve_target.outputs.qa_live_matrix_enabled == 'true'
permissions:
contents: read
pull-requests: read
@@ -1934,7 +1944,7 @@ jobs:
qa_live_buzz_release_checks:
name: Run QA Lab live Buzz lane
needs: [resolve_target]
if: contains(fromJSON('["all","qa","qa-live"]'), needs.resolve_target.outputs.rerun_group) && needs.resolve_target.outputs.qa_live_buzz_enabled == 'true'
if: needs.resolve_target.outputs.qa_live_scheduled == 'true' && needs.resolve_target.outputs.qa_live_buzz_enabled == 'true'
permissions:
contents: read
pull-requests: read
@@ -1954,7 +1964,7 @@ jobs:
qa_live_telegram_release_checks:
name: Run QA Lab live Telegram lane
needs: [resolve_target]
if: contains(fromJSON('["all","qa","qa-live"]'), needs.resolve_target.outputs.rerun_group) && needs.resolve_target.outputs.qa_live_telegram_enabled == 'true'
if: needs.resolve_target.outputs.qa_live_scheduled == 'true' && needs.resolve_target.outputs.qa_live_telegram_enabled == 'true'
runs-on: ubuntu-24.04
timeout-minutes: 210
permissions:
@@ -2079,7 +2089,7 @@ jobs:
qa_live_discord_release_checks:
name: Run QA Lab live Discord lane
needs: [resolve_target]
if: contains(fromJSON('["all","qa","qa-live"]'), needs.resolve_target.outputs.rerun_group) && needs.resolve_target.outputs.qa_live_discord_enabled == 'true'
if: needs.resolve_target.outputs.qa_live_scheduled == 'true' && needs.resolve_target.outputs.qa_live_discord_enabled == 'true'
continue-on-error: true
runs-on: ubuntu-24.04
timeout-minutes: 60
@@ -2194,7 +2204,7 @@ jobs:
qa_live_whatsapp_release_checks:
name: Run QA Lab live WhatsApp lane
needs: [resolve_target]
if: contains(fromJSON('["all","qa","qa-live"]'), needs.resolve_target.outputs.rerun_group) && needs.resolve_target.outputs.qa_live_whatsapp_enabled == 'true'
if: needs.resolve_target.outputs.qa_live_scheduled == 'true' && needs.resolve_target.outputs.qa_live_whatsapp_enabled == 'true'
continue-on-error: true
runs-on: ubuntu-24.04
timeout-minutes: 60
@@ -2293,7 +2303,7 @@ jobs:
qa_live_slack_release_checks:
name: Run QA Lab live Slack lane
needs: [resolve_target]
if: contains(fromJSON('["all","qa","qa-live"]'), needs.resolve_target.outputs.rerun_group) && needs.resolve_target.outputs.qa_live_slack_enabled == 'true'
if: needs.resolve_target.outputs.qa_live_scheduled == 'true' && needs.resolve_target.outputs.qa_live_slack_enabled == 'true'
continue-on-error: true
runs-on: ubuntu-24.04
timeout-minutes: 60
@@ -2528,7 +2538,7 @@ jobs:
QA_LIVE_RELEASE_CHECKS_RESULT: ${{ needs.qa_live_release_checks.result }}
QA_LIVE_BUZZ_RELEASE_CHECKS_RESULT: ${{ needs.qa_live_buzz_release_checks.result }}
QA_LIVE_TELEGRAM_RELEASE_CHECKS_RESULT: ${{ needs.qa_live_telegram_release_checks.result }}
QA_LIVE_TELEGRAM_SELECTED: ${{ contains(fromJSON('["all","qa","qa-live"]'), needs.resolve_target.outputs.rerun_group) && needs.resolve_target.outputs.qa_live_telegram_enabled == 'true' }}
QA_LIVE_TELEGRAM_SELECTED: ${{ needs.resolve_target.outputs.qa_live_scheduled == 'true' && needs.resolve_target.outputs.qa_live_telegram_enabled == 'true' }}
QA_LIVE_DISCORD_RELEASE_CHECKS_RESULT: ${{ needs.qa_live_discord_release_checks.result }}
QA_LIVE_WHATSAPP_RELEASE_CHECKS_RESULT: ${{ needs.qa_live_whatsapp_release_checks.result }}
QA_LIVE_SLACK_RELEASE_CHECKS_RESULT: ${{ needs.qa_live_slack_release_checks.result }}
.github/workflows/qa-live-transports-convex.yml
+2
View File
@@ -120,6 +120,7 @@ jobs:
authorize_actor:
name: Authorize workflow actor
runs-on: blacksmith-8vcpu-ubuntu-2404
timeout-minutes: 10
outputs:
authorized: ${{ steps.permission.outputs.authorized }}
steps:
@@ -158,6 +159,7 @@ jobs:
needs: authorize_actor
if: needs.authorize_actor.outputs.authorized == 'true'
runs-on: blacksmith-8vcpu-ubuntu-2404
timeout-minutes: 30
outputs:
selected_revision: ${{ steps.validate.outputs.selected_revision }}
trusted_reason: ${{ steps.validate.outputs.trusted_reason }}
CHANGELOG.md
+1
View File
@@ -6,6 +6,7 @@ Docs: https://docs.openclaw.ai
### Changes
- **Release validation:** defer beta candidate Parallels smoke to postpublish `release:beta-smoke` by default, keep stable/full prepublish coverage, and bound nested release workflow monitors with explicit job timeouts.
- **macOS app profiles:** isolate named app instances across state, preferences, Keychain, Gateway services, and duplicate-instance ownership while keeping host-global login and node services untouched.
- **Developer workflow:** remove the obsolete scoped-commit helper and use standard Git commands in isolated worktrees.
- **Plugin uninstall cleanup:** remove exact recorded install paths from `plugins.load.paths` for marketplace, npm, and other managed installs while preserving parent, child, prefix, and unrelated paths.
docs/ci.md
+30 -16
View File
@@ -164,7 +164,10 @@ PR max-lines checks derive the baseline from the checked-out synthetic merge tre
```bash
gh workflow run ci.yml --ref release/YYYY.M.PATCH
gh workflow run ci.yml --ref main -f target_ref=<branch-or-sha> -f include_android=true
gh workflow run full-release-validation.yml --ref main -f ref=<branch-or-sha>
VALIDATION_SHA="<full-commit-sha>"
gh workflow run full-release-validation.yml --ref main \
-f ref="$VALIDATION_SHA" \
-f expected_sha="$VALIDATION_SHA"
```
Gateway extended-stable runs npm preflight, Full Release Validation, and plugin
@@ -287,7 +290,16 @@ Every lane uploads its complete GitHub artifact, including CPU, heap, trace, and
## Full Release Validation
`Full Release Validation` is the manual umbrella workflow for "run everything before release." It accepts a branch, tag, or full commit SHA, dispatches the manual `CI` workflow with that target (including Android), dispatches `Plugin Prerelease` for release-only plugin/package/static/Docker proof, dispatches `OpenClaw Performance` against the target SHA, and dispatches `OpenClaw Release Checks` for install smoke, package acceptance, cross-OS package checks, QA Lab parity, Matrix, Telegram, and gated Discord, WhatsApp, and Slack lanes (advisory maturity scorecard rendering is opt-in via `run_maturity_scorecard`). Stable and full profiles always include exhaustive live/E2E and Docker release-path soak coverage; the beta profile can opt in with `run_release_soak=true`. The canonical package Telegram E2E runs inside Package Acceptance, so a full candidate does not start a duplicate live poller. After publishing, pass `release_package_spec` to reuse the shipped npm package across release checks, Package Acceptance, Docker, cross-OS, and Telegram without rebuilding. Use `npm_telegram_package_spec` only for a focused published-package Telegram rerun. The Codex plugin live package lane uses the same selected state by default: published `release_package_spec=openclaw@<tag>` derives `codex_plugin_spec=npm:@openclaw/codex@<tag>`, while SHA/artifact runs pack `extensions/codex` from the selected ref. Set `codex_plugin_spec` explicitly for custom plugin sources such as `npm:`, `npm-pack:`, or `git:` specs. Its live agent proof sends visible progress, continues through randomized workspace reads and an exact artifact write, then sends completion.
`Full Release Validation` is the manual release umbrella. Every run binds an
exact Validation SHA + Tooling SHA tuple and rejects an `expected_sha` mismatch
before child dispatch. Validation SHA maps to the Code SHA for product
validation or the Release SHA for changelog-only validation; it is not a third
release identity. Beta-publish maps to `release_profile=beta` with
`run_release_soak=false`; its `all` run includes normal CI, Plugin Prerelease,
package/install/cross-OS checks, performance, and QA parity, but excludes broad
live/E2E and QA-live. Postpublish-confidence uses the exact published package
with soak or explicit focused groups. Stable-publish maps to
`release_profile=stable`.
See [Full release validation](/reference/full-release-validation) for the
stage matrix, exact workflow job names, profile differences, artifacts, and
@@ -331,13 +343,10 @@ pnpm ci:full-release --sha <full-sha>
```
GitHub workflow dispatch refs must be branches or tags, not raw commit SHAs. The
helper pushes a temporary `release-ci/<sha>-...` branch at a trusted `main`
workflow SHA, passes the requested target SHA through the workflow `ref` input,
reuses strict exact-target evidence when available, verifies every child
workflow `headSha` matches the trusted workflow SHA, and deletes the temporary
branch when the run completes. Pass `-f reuse_evidence=false` to force fresh
validation. The umbrella verifier also fails if any child workflow ran at a
different workflow SHA.
helper pushes a temporary `release-ci/<sha>-...` branch at a trusted Tooling
SHA, passes the requested Validation SHA through `ref` and `expected_sha`, reuses
strict exact-target evidence when available, and verifies every child workflow
`headSha` matches the Tooling SHA.
`release_profile` controls live/provider breadth passed into release checks. The
manual release workflows default to `stable`; use `full` only when you
@@ -355,17 +364,22 @@ CLI's own first-scenario cancellation.
- `stable` adds the stable provider/backend set.
- `full` runs the broad advisory provider/media matrix.
The umbrella records the dispatched child run ids, and the final `Verify full validation` job re-checks current child run conclusions and appends slowest-job tables for each child run. If a child workflow is rerun and turns green, rerun only the parent verifier job to refresh the umbrella result and timing summary.
The umbrella records dispatched child run ids, and `Verify full validation`
checks them during that parent attempt. Parent cancellation or timeout leaves
adopted exact children running; cancel one explicitly when it is no longer
needed.
For recovery, both `Full Release Validation` and `OpenClaw Release Checks` accept `rerun_group`. Use `all` for a release candidate, `ci` for only the normal full CI child, `plugin-prerelease` for only the plugin prerelease child, `performance` for only the OpenClaw Performance child, `release-checks` for every release child, or a narrower group: `install-smoke`, `cross-os`, `live-e2e`, `package`, `qa`, `qa-parity`, `qa-live`, or `npm-telegram` on the umbrella. This keeps a failed release box rerun bounded after a focused fix. For one failed cross-OS lane, combine `rerun_group=cross-os` with `cross_os_suite_filter`, for example `windows/packaged-upgrade`; long cross-OS commands emit heartbeat lines and packaged-upgrade summaries include per-phase timings. Selected Matrix and Telegram QA lanes block normal release validation, as does the core runtime-pair tool coverage gate. QA parity, runtime parity, and the gated Discord, WhatsApp, and Slack live lanes are advisory.
For recovery, classify product, harness/tooling/provenance,
infrastructure/credential, and wrapper failures before editing. Only confirmed
product failure changes the Code SHA. Use one diagnosis, one fix when needed,
and one narrow `rerun_group` retry, then reassess; never widen automatically to
`all`. Narrow evidence is not publish authorization by itself.
`OpenClaw Release Checks` uses the trusted workflow ref to resolve the selected ref once into a `release-package-under-test` tarball, then passes that artifact to cross-OS checks and Package Acceptance, plus the live/E2E release-path Docker workflow when soak coverage runs. That keeps the package bytes consistent across release boxes and avoids repacking the same candidate in multiple child jobs. For the Codex npm-plugin live lane, release checks either pass a matching published plugin spec derived from `release_package_spec`, pass the operator-supplied `codex_plugin_spec`, or leave the input blank so the Docker script packs the selected checkout's Codex plugin.
Duplicate `Full Release Validation` runs for `ref=main` and `rerun_group=all`
supersede the older umbrella. The parent monitor cancels any child workflow it
has already dispatched when the parent is cancelled, so newer main validation
does not sit behind a stale two-hour release-check run. Release branch/tag
validation and focused rerun groups keep `cancel-in-progress: false`.
Full Release Validation concurrency is keyed by Validation SHA, Tooling SHA,
and rerun group with `cancel-in-progress: false`. Parent cancellation does not
cancel adopted children.
## Live and E2E shards
docs/help/testing.md
+10 -10
View File
@@ -168,17 +168,17 @@ These commands sit beside the main test suites when you need QA-lab realism.
CI runs QA Lab in dedicated workflows. Agentic parity is nested under
`QA-Lab - All Lanes` and release validation, not a standalone PR workflow.
Broad validation should use `Full Release Validation` with
`rerun_group=qa-parity` or the release-checks QA group. Stable/default release
checks keep exhaustive live/Docker soak behind `run_release_soak=true`; the
`full` profile forces soak on. `QA-Lab - All Lanes` runs nightly on `main` and
from manual dispatch with the mock parity lane, live Matrix lane,
Convex-managed live Telegram lane, and Convex-managed live Discord lane as
parallel jobs. Scheduled QA and release checks run the catalog-derived Matrix
selection through the shared live adapter. `OpenClaw Release Checks` runs parity plus the
reusable Matrix live-adapter lane and Telegram lane before release approval. Release
`rerun_group=qa-parity` or the release-checks QA group. Stable/full,
soak-enabled, and explicit `qa`/`qa-live` release checks include the QA-live
Matrix and Telegram lanes. Bounded beta-publish `all` without soak runs parity
but defers those live lanes to postpublish-confidence. `QA-Lab - All Lanes` runs
nightly on `main` and from manual dispatch with the mock parity lane, live
Matrix lane, Convex-managed live Telegram lane, and Convex-managed live Discord
lane as parallel jobs. Scheduled QA and selected release checks run the
catalog-derived Matrix selection through the shared live adapter. Release
transport checks use `mock-openai/gpt-5.6-luna` so they stay deterministic and
avoid normal provider-plugin startup. These live transport gateways
disable memory search; memory behavior stays covered by the QA parity suites.
avoid normal provider-plugin startup. These live transport gateways disable
memory search; memory behavior stays covered by the QA parity suites.
Full release live media shards use
`ghcr.io/openclaw/openclaw-live-media-runner:ubuntu-24.04`, which already has
docs/reference/RELEASING.md
+13 -10
View File
@@ -86,6 +86,7 @@ gh workflow run openclaw-npm-release.yml \
gh workflow run full-release-validation.yml \
--ref extended-stable/YYYY.M.33 \
-f ref=extended-stable/YYYY.M.33 \
-f expected_sha="$RELEASE_SHA" \
-f release_profile=stable
```
@@ -229,8 +230,8 @@ This checklist is the public shape of the release flow. Private credentials, sig
1. Start from current `main`: pull latest, confirm the target commit is pushed, and confirm `main` CI is green enough to branch from.
2. Create `release/YYYY.M.PATCH` from that commit. Backports are optional; apply only the operator-selected set. Bump every required version location, run `pnpm release:prep`, finish release fixes and required forward-ports, and review `src/plugins/compat/registry.ts` plus `src/commands/doctor/shared/deprecation-compat.ts`.
3. Freeze the product-complete pre-changelog commit as the **Code SHA**. Run the deterministic source preflight, then use `node scripts/full-release-validation-at-sha.mjs --sha <code-sha> --target-ref release/YYYY.M.PATCH`. This pins trusted workflow tooling while the full Vitest, Docker, QA, package, and performance matrix targets the exact Code SHA.
4. Classify failures before editing. A product/code failure creates a new Code SHA and requires green full validation for that SHA. A workflow, harness, credential, approval, or infrastructure failure is repaired in its owning surface and rerun against the same Code SHA.
3. Freeze the product-complete pre-changelog commit as the **Code SHA** and record the trusted **Tooling SHA**. Run the deterministic source preflight, then use `node scripts/full-release-validation-at-sha.mjs --sha <code-sha> --target-ref release/YYYY.M.PATCH`. Beta-publish uses `release_profile=beta` without soak; postpublish-confidence owns broad live, QA-live, mobile, and Parallels work.
4. Classify failures before editing as product, harness/tooling/provenance, infrastructure/credential, or wrapper. Only confirmed product failure creates a new Code SHA. Use one diagnosis, one fix when needed, and one narrow retry, then reassess.
5. Only after the Code SHA is green, generate the top `CHANGELOG.md` section from merged PRs and direct commits since the last reachable shipped tag. Keep entries user-facing and deduplicated. When a divergent shipped tag or later forward-port re-associates already-released PRs, pass it explicitly as `--shipped-ref`.
6. Commit only `CHANGELOG.md`. This commit is the **Release SHA**. The complete diff from Code SHA to Release SHA must be exactly `CHANGELOG.md`; any other changed path returns the release to step 2.
7. Run SHA-pinned Full Release Validation for the Release SHA with evidence reuse enabled. The lightweight parent must record `changelog-only-release-v1`, point at the green Code SHA, and dispatch no product child lanes. This reuses product evidence; it does not reuse package bytes.
@@ -246,7 +247,7 @@ This checklist is the public shape of the release flow. Private credentials, sig
--skip-dispatch
```
For stable, also pass `--windows-node-tag vX.Y.Z`. The helper verifies release-note provenance, npm preflight bytes, Parallels install/update proof, Telegram package proof, and plugin publish plans, then prints the publish command. After it completes green, create and push the final signed tag at that same Release SHA, then run the printed publish command.
For stable, also pass `--windows-node-tag vX.Y.Z`. Beta and alpha candidates defer Parallels install/update proof to the postpublish `pnpm release:beta-smoke` roster by default; pass `--run-parallels` only when the operator explicitly wants that proof before publish. Stable and full candidates run Parallels by default. The helper still verifies release-note provenance, npm preflight bytes, Telegram package proof, and plugin publish plans, then prints the publish command. After it completes green, create and push the final signed tag at that same Release SHA, then run the printed publish command.
`pnpm release:candidate` validates the current frozen branch tip by default (or the explicit `--target-sha`), and rejects a tag that already exists. It records evidence before the final signed tag is pushed.
@@ -328,7 +329,7 @@ A legacy fallback correction tag may reuse base-package evidence only when the c
Manually dispatch `Windows Node Release` only for recovery, and always pass an exact tag, never `latest`, plus the explicit `expected_installer_digests` JSON map from the approved source release. Website download links should target exact OpenClaw release asset URLs for the current stable release, or `releases/latest/download/...` only after verifying GitHub's latest redirect points at that same release; do not link only to the companion repo release page.
- Release checks now run in a separate manual workflow: `OpenClaw Release Checks`. It also runs the QA Lab mock parity lane plus the Matrix catalog and Telegram QA lane before release approval. The live lanes use the `qa-live-shared` environment; Telegram also uses Convex CI credential leases.
- Release checks now run in a separate manual workflow: `OpenClaw Release Checks`. It always runs the QA Lab mock parity lane. The Matrix catalog and Telegram QA-live lanes run for stable/full validation, soak-enabled validation, or an explicit `qa`/`qa-live` rerun group. Bounded beta-publish `all` without soak defers those live lanes to postpublish-confidence. The live lanes use the `qa-live-shared` environment; Telegram also uses Convex CI credential leases.
- Cross-OS install and upgrade runtime validation is part of public `OpenClaw Release Checks` and `Full Release Validation`, which call the reusable workflow `.github/workflows/openclaw-cross-os-release-checks-reusable.yml` directly. This split is intentional: keep the real npm release path short, deterministic, and artifact-focused, while slower live checks stay in their own lane so they do not stall or block publish.
- Secret-bearing release checks should be dispatched through `Full Release Validation` or from the `main`/release workflow ref so workflow logic and secrets stay controlled.
- `OpenClaw Release Checks` accepts a branch, tag, or full commit SHA as long as the resolved commit is reachable from an OpenClaw branch or release tag.
@@ -367,7 +368,7 @@ pnpm ci:full-release \
--target-ref release/YYYY.M.PATCH
```
The helper fetches current `origin/main`, pushes `release-ci/<workflow-sha>-...` at that trusted workflow commit, infers `beta` from alpha/beta package versions and `stable` otherwise, dispatches `Full Release Validation` from the temporary branch with `ref=<target-sha>`, verifies every child workflow `headSha` matches the pinned parent workflow SHA, then deletes the temporary branch. Pass `-f reuse_evidence=false` to force a fresh run, `-f release_profile=full` for the broad advisory sweep, or `--workflow-sha <trusted-main-sha>` to pin an older commit that is still reachable from current `origin/main`. The workflow itself never writes repository refs. This keeps main-only release tooling available without adding tooling commits to the candidate and avoids proving a newer `main` child run by accident.
The helper fetches current `origin/main`, pushes `release-ci/<workflow-sha>-...` at that trusted Tooling SHA, infers `beta` from alpha/beta package versions and `stable` otherwise, and dispatches `Full Release Validation` with the Validation SHA as `expected_sha`. Target resolution rejects a mismatch before child dispatch. Every child workflow `headSha` must match the Tooling SHA. Pass `-f reuse_evidence=false` to force a fresh run, `-f release_profile=full` for the broad advisory sweep, or `--workflow-sha <trusted-main-sha>` to pin a compatible older commit still reachable from current `origin/main`. The helper rejects pinned tooling that lacks the `expected_sha` dispatch input and never silently selects a newer Tooling SHA. The workflow itself never writes repository refs.
After the Code SHA is green, commit only `CHANGELOG.md` and run the same helper with the Release SHA:
@@ -379,7 +380,7 @@ pnpm ci:full-release \
The second parent reuses product evidence only when GitHub proves the Release SHA descends from the Code SHA and the complete changed path set is exactly `CHANGELOG.md`. It records `changelog-only-release-v1` and dispatches no product children. Npm preflight and package/install acceptance still run on the Release SHA because its tarball bytes changed.
For a fresh Code SHA, the workflow resolves the target, dispatches manual `CI`, then dispatches `OpenClaw Release Checks`. `OpenClaw Release Checks` fans out install smoke, cross-OS release checks, live/E2E Docker release-path coverage when soak is enabled, Package Acceptance with the canonical Telegram package E2E, QA Lab parity, live Matrix, and live Telegram. A full/all run is only acceptable when the `Full Release Validation` summary shows `normal_ci`, `plugin_prerelease`, and `release_checks` as successful, unless a focused rerun intentionally skipped the separate `Plugin Prerelease` child. Use the standalone `npm-telegram` child only for a focused published-package rerun with `release_package_spec` or `npm_telegram_package_spec`. The final verifier summary includes slowest-job tables for each child run, so the release manager can see the current critical path without downloading logs.
For a fresh Code SHA, the workflow resolves the target, dispatches manual `CI`, then dispatches `OpenClaw Release Checks`. Beta-publish maps to `release_profile=beta` and `run_release_soak=false`; its `all` run excludes broad live/E2E and QA-live lanes. Postpublish-confidence uses the exact published package with soak or explicit focused groups. Stable-publish maps to `release_profile=stable`. The final verifier summary includes slowest-job tables for each child run.
The product-performance child is artifact-only in this release path. The
umbrella dispatches it with `publish_reports=false`, and validation is rejected
@@ -421,10 +422,11 @@ pnpm ci:full-release \
--target-ref release/YYYY.M.PATCH \
-f release_package_spec=openclaw@YYYY.M.PATCH-beta.N \
-f evidence_package_spec=openclaw@YYYY.M.PATCH-beta.N \
-f run_release_soak=true \
-f npm_telegram_provider_mode=mock-openai
```
Do not use the full umbrella as the first rerun after a focused fix. If one box fails, use the failed child workflow, job, Docker lane, package profile, model provider, or QA lane for the next proof. Run the full umbrella again only when the fix changed shared release orchestration or made earlier all-box evidence stale. The umbrella's final verifier re-checks the recorded child workflow run ids, so after a child workflow is rerun successfully, rerun only the failed `Verify full validation` parent job.
Do not use the full umbrella as the first rerun after a focused fix. Classify the failure as product, harness/tooling/provenance, infrastructure/credential, or wrapper. Only confirmed product failure changes the Code SHA. Use one diagnosis, one fix when needed, and one narrow retry, then reassess. A narrow green run is evidence, not publish authorization by itself; there is no standalone parent finalizer.
`rerun_group=all` may reuse a prior green umbrella run when the release profile,
effective soak setting, and validation inputs match and either the target SHA
@@ -435,9 +437,10 @@ is exactly `CHANGELOG.md`. Exact-target reuse records
preflight, package bytes, release-note provenance, and install/update acceptance
must still run against the Release SHA. Any version, source, generated,
dependency, package, or workflow-owned target change requires a new Code SHA
and fresh full validation. Newer umbrella runs for the same `release/*` ref and
rerun group supersede in-progress ones automatically. Pass
`reuse_evidence=false` to force a fresh full run.
and fresh full validation. Concurrency is keyed by Validation SHA, Tooling SHA,
and rerun group and does not cancel prior runs. Parent cancellation leaves
adopted children running until the operator cancels the exact child. Pass
`reuse_evidence=false` only when a fresh full run is intentionally required.
For bounded recovery, pass `rerun_group` to the umbrella. `all` is the real release-candidate run, `ci` runs only the normal CI child, `plugin-prerelease` runs only the release-only plugin child, `release-checks` runs every release box, and the narrower release groups are `install-smoke`, `cross-os`, `live-e2e`, `package`, `qa`, `qa-parity`, `qa-live`, and `npm-telegram`. Focused `npm-telegram` reruns require `release_package_spec` or `npm_telegram_package_spec`; full/all runs use the canonical package Telegram E2E inside Package Acceptance. Focused cross-OS reruns can add `cross_os_suite_filter=windows/packaged-upgrade` or another OS/suite filter. QA release-check failures block normal release validation, including OpenClaw dynamic tool drift in the core runtime-pair lane. Tideclaw alpha runs may still treat non-package-safety release-check lanes as advisory. With `release_profile=beta`, the `Run repo/live E2E validation` live-provider suites are advisory (warnings, not blockers); stable and full profiles keep them blocking. When `live_suite_filter` explicitly requests a gated QA live lane such as Discord, WhatsApp, or Slack, the matching `OPENCLAW_RELEASE_QA_*_LIVE_CI_ENABLED` repo variable must be enabled; otherwise input capture fails instead of silently skipping the lane.
docs/reference/full-release-validation.md
+52 -36
View File
@@ -14,7 +14,8 @@ whole release. Run release preparation before freezing the Code SHA; it
refreshes Control UI locale output when the background bot has not landed it
yet, then enforces the same strict zero-fallback check used by release CI.
Freeze the product-complete pre-changelog commit as the **Code SHA**, then run:
Freeze the product-complete pre-changelog commit as the **Code SHA** and select
one trusted workflow commit as the **Tooling SHA**, then run:
```bash
pnpm ci:full-release \
@@ -30,13 +31,18 @@ package versions and `stable` otherwise. Pass alternate workflow inputs with
independent failures together. Pass `-f fail_fast=true` when the shorter
first-failure cancellation path is preferable.
The helper creates a temporary `release-ci/*` ref pinned to one trusted
`origin/main` workflow SHA, passes the target SHA only as the candidate `ref`,
and deletes the temporary ref after validation. Every dispatched child must
report that same workflow SHA. Pass
The helper creates a temporary `release-ci/*` ref pinned to the Tooling SHA,
passes the Validation SHA as both the candidate ref and `expected_sha`, and
deletes the temporary ref after validation. The Validation SHA equals the Code
SHA for product validation or the Release SHA for changelog-only validation; it
is not a third release identity. The workflow rejects malformed or mismatched
expected SHAs before child dispatch. Every child must report the same Tooling
SHA. Pass
`-f reuse_evidence=false` to force a fresh run or
`--workflow-sha <trusted-main-sha>` to select an older workflow commit still
reachable from current `origin/main`. The workflow never creates or updates
`--workflow-sha <trusted-main-sha>` to select a compatible older workflow
commit still reachable from current `origin/main`. The helper rejects a pinned
Tooling SHA that does not declare the `expected_sha` dispatch input; it never
silently substitutes newer tooling. The workflow never creates or updates
repository refs itself.
## Extended-stable exception
@@ -45,9 +51,11 @@ Extended-stable publish requires a run whose workflow and target are both the
canonical branch:
```bash
RELEASE_SHA="$(git rev-parse HEAD)"
gh workflow run full-release-validation.yml \
--ref extended-stable/YYYY.M.33 \
-f ref=extended-stable/YYYY.M.33 \
-f expected_sha="$RELEASE_SHA" \
-f release_profile=stable
```
@@ -67,10 +75,16 @@ SHA descends from the Code SHA and the complete changed path set is exactly
`CHANGELOG.md`; npm preflight and package/install acceptance still run on the
Release SHA.
`release_profile=stable` and `release_profile=full` always run the exhaustive
live/Docker soak. Pass `run_release_soak=true` to include the same soak lanes
with the `beta` profile. Stable publication rejects a validation manifest
without this soak and blocking product-performance evidence.
The conceptual phases map to current inputs:
- `beta-publish`: `release_profile=beta`, `run_release_soak=false`
- `postpublish-confidence`: exact published package plus
`run_release_soak=true` or explicit focused groups
- `stable-publish`: `release_profile=stable`
Beta-publish `all` excludes broad live/E2E soak and QA-live lanes. Stable and
full always run the soak. Stable publication rejects a validation manifest
without soak and blocking product-performance evidence.
Package Acceptance normally builds the candidate tarball from the resolved
`ref`, including full-SHA runs dispatched with `pnpm ci:full-release`. After a
@@ -122,17 +136,17 @@ the `runtime-assets` Docker target with
other stages and is enforced by the umbrella verifier; lanes no longer wait for
it before dispatching. A narrower `rerun_group` skips this preflight.
| Stage | Details |
| ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Target resolution | **Job:** `Resolve target ref`<br />**Child workflow:** none<br />**Proves:** resolves the release branch, tag, or full commit SHA and records selected inputs.<br />**Rerun:** rerun the umbrella if this fails. |
| Shared candidate | **Job:** `Prepare shared release candidate`<br />**Child workflow:** `OpenClaw Live And E2E Checks (Reusable)`<br />**Proves:** packs and validates one exact-SHA package, builds one functional Docker image, and records immutable package and image artifact tuples for both package-facing child workflows.<br />**Rerun:** rerun the affected package, plugin-prerelease, cross-OS, or live/E2E group. |
| Docker assets preflight | **Job:** `Verify Docker runtime image assets`<br />**Child workflow:** none<br />**Proves:** the `runtime-assets` Docker build target still succeeds before any other stage dispatches. Runs only for `rerun_group=all`.<br />**Rerun:** rerun the umbrella with `rerun_group=all`. |
| Vitest and normal CI | **Job:** `Run normal full CI`<br />**Child workflow:** `CI`<br />**Proves:** manual full CI graph against the target ref, including Linux Node lanes, bundled plugin shards, plugin and channel contract shards, Node 22 compatibility, `check-*`, `check-additional-*`, built-artifact smoke checks, docs checks, Python skills, Windows, macOS, Control UI i18n, and Android via the umbrella.<br />**Rerun:** `rerun_group=ci`. |
| Plugin prerelease | **Job:** `Run plugin prerelease validation`<br />**Child workflow:** `Plugin Prerelease`<br />**Proves:** release-only plugin static checks, agentic plugin coverage, full plugin batch shards, plugin prerelease Docker lanes, and a non-blocking `plugin-inspector-advisory` artifact for compatibility triage.<br />**Rerun:** `rerun_group=plugin-prerelease`. |
| Release checks | **Job:** `Run release/live/Docker/QA validation`<br />**Child workflow:** `OpenClaw Release Checks`<br />**Proves:** install smoke, cross-OS package checks, Package Acceptance, QA Lab parity, live Matrix, Buzz, and Telegram, plus gated advisory Discord, WhatsApp, and Slack lanes. Stable and full profiles also run exhaustive live/E2E suites and Docker release-path chunks; beta can opt in with `run_release_soak=true`.<br />**Rerun:** `rerun_group=release-checks` or a narrower release-checks handle. |
| Package Telegram | **Job:** `Run package Telegram E2E`<br />**Child workflow:** `NPM Telegram Beta E2E`<br />**Proves:** a focused published-package Telegram E2E when `release_package_spec` or `npm_telegram_package_spec` is set. Full candidate validation uses the canonical Package Acceptance Telegram E2E instead.<br />**Rerun:** `rerun_group=npm-telegram` with `release_package_spec` or `npm_telegram_package_spec`. |
| Product performance | **Job:** `Run product performance evidence`<br />**Child workflow:** `OpenClaw Performance`<br />**Proves:** release-profile performance run (`profile=release`, `repeat=3`, `fail_on_regression=true`, `publish_reports=false`) against the target SHA. Kova output stays in workflow artifacts and the child must prove its report publisher was skipped. Required (blocking) only for `rerun_group=all` or `rerun_group=performance`; not required for narrower rerun groups.<br />**Rerun:** `rerun_group=performance`. |
| Umbrella verifier | **Job:** `Verify full validation`<br />**Child workflow:** none<br />**Proves:** re-checks recorded child run conclusions and appends slowest-job tables from child workflows.<br />**Rerun:** rerun only this job after rerunning a failed child to green. |
| Stage | Details |
| ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Target resolution | **Job:** `Resolve target ref`<br />**Child workflow:** none<br />**Proves:** resolves the release branch, tag, or full commit SHA and records selected inputs.<br />**Rerun:** rerun the umbrella if this fails. |
| Shared candidate | **Job:** `Prepare shared release candidate`<br />**Child workflow:** `OpenClaw Live And E2E Checks (Reusable)`<br />**Proves:** packs and validates one exact-SHA package, builds one functional Docker image, and records immutable package and image artifact tuples for both package-facing child workflows.<br />**Rerun:** rerun the affected package, plugin-prerelease, cross-OS, or live/E2E group. |
| Docker assets preflight | **Job:** `Verify Docker runtime image assets`<br />**Child workflow:** none<br />**Proves:** the `runtime-assets` Docker build target still succeeds before any other stage dispatches. Runs only for `rerun_group=all`.<br />**Rerun:** rerun the umbrella with `rerun_group=all`. |
| Vitest and normal CI | **Job:** `Run normal full CI`<br />**Child workflow:** `CI`<br />**Proves:** manual full CI graph against the target ref, including Linux Node lanes, bundled plugin shards, plugin and channel contract shards, Node 22 compatibility, `check-*`, `check-additional-*`, built-artifact smoke checks, docs checks, Python skills, Windows, macOS, Control UI i18n, and Android via the umbrella.<br />**Rerun:** `rerun_group=ci`. |
| Plugin prerelease | **Job:** `Run plugin prerelease validation`<br />**Child workflow:** `Plugin Prerelease`<br />**Proves:** release-only plugin static checks, agentic plugin coverage, full plugin batch shards, plugin prerelease Docker lanes, and a non-blocking `plugin-inspector-advisory` artifact for compatibility triage.<br />**Rerun:** `rerun_group=plugin-prerelease`. |
| Release checks | **Job:** `Run release/live/Docker/QA validation`<br />**Child workflow:** `OpenClaw Release Checks`<br />**Proves:** install smoke, cross-OS package checks, Package Acceptance, and QA Lab parity. QA-live Matrix, Buzz, and Telegram plus gated advisory Discord, WhatsApp, and Slack run for stable/full, beta with `run_release_soak=true`, or explicit `qa`/`qa-live` groups. Stable and full profiles also run exhaustive live/E2E suites and Docker release-path chunks.<br />**Rerun:** `rerun_group=release-checks` or a narrower release-checks handle. |
| Package Telegram | **Job:** `Run package Telegram E2E`<br />**Child workflow:** `NPM Telegram Beta E2E`<br />**Proves:** a focused published-package Telegram E2E when `release_package_spec` or `npm_telegram_package_spec` is set. Full candidate validation uses the canonical Package Acceptance Telegram E2E instead.<br />**Rerun:** `rerun_group=npm-telegram` with `release_package_spec` or `npm_telegram_package_spec`. |
| Product performance | **Job:** `Run product performance evidence`<br />**Child workflow:** `OpenClaw Performance`<br />**Proves:** release-profile performance run (`profile=release`, `repeat=3`, `fail_on_regression=true`, `publish_reports=false`) against the target SHA. Kova output stays in workflow artifacts and the child must prove its report publisher was skipped. Required (blocking) only for `rerun_group=all` or `rerun_group=performance`; not required for narrower rerun groups.<br />**Rerun:** `rerun_group=performance`. |
| Umbrella verifier | **Job:** `Verify full validation`<br />**Child workflow:** none<br />**Proves:** re-checks recorded child run conclusions and appends slowest-job tables from child workflows.<br />**Rerun:** rerun only this job after rerunning a failed child to green. |
The umbrella always dispatches product performance in artifact-only mode.
`OpenClaw Performance` permits report publication only for scheduled runs or a
@@ -153,11 +167,10 @@ publish consumers. The verifier always prefers the attempt-qualified artifact;
as a transition, it accepts the stable name only for an attempt-1 manifest v2
producer. It rejects that legacy name for later attempts and manifest v3.
For `ref=main` with `rerun_group=all`, for `release/*` refs, and for Tideclaw
alpha refs, a newer umbrella run supersedes an older one with the same ref and
rerun group. When the parent is cancelled, its monitor cancels any child
workflow it already dispatched. Tag and pinned-SHA validation runs do not
cancel each other.
Concurrency is keyed by Validation SHA, Tooling SHA, and rerun group and does
not cancel an older run. Parent cancellation or timeout leaves an adopted
identity-checked child running. Cancel that exact child explicitly when it is
no longer useful.
## Release checks stages
@@ -168,7 +181,7 @@ artifact when package or Docker-facing stages need it.
| Stage | Details |
| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Release target | **Job:** `Resolve target ref`<br />**Backing workflow:** none<br />**Tests:** selected ref, optional expected SHA, profile, rerun group, and focused live suite filter.<br />**Rerun:** `rerun_group=release-checks`. |
| Release target | **Job:** `Resolve target ref`<br />**Backing workflow:** none<br />**Tests:** selected ref, optional expected Validation SHA, profile, rerun group, and focused live suite filter.<br />**Rerun:** `rerun_group=release-checks`. |
| Package artifact | **Job:** `Prepare release package artifact`<br />**Backing workflow:** none<br />**Tests:** validates the umbrella's immutable package tuple, or packs one candidate tarball for a direct/focused Release Checks dispatch, then exposes it to downstream package-facing checks.<br />**Rerun:** the affected package, cross-OS, or live/E2E group. |
| Install smoke | **Job:** `Run install smoke`<br />**Backing workflow:** `Install Smoke`<br />**Tests:** full install path with root Dockerfile smoke image reuse, QR package install, root and gateway Docker smokes, installer Docker tests, and Bun global install image-provider smoke.<br />**Rerun:** `rerun_group=install-smoke`. |
| Cross-OS | **Job:** `cross_os_release_checks`<br />**Backing workflow:** `OpenClaw Cross-OS Release Checks (Reusable)`<br />**Tests:** fresh and upgrade lanes on Linux, Windows, and macOS for the selected provider and mode, using the candidate tarball plus a baseline package.<br />**Rerun:** `rerun_group=cross-os`. |
@@ -211,11 +224,11 @@ commands with package artifact and image reuse inputs when available.
`release_profile` mostly controls live/provider breadth inside release checks.
It does not remove normal full CI, Plugin Prerelease, install smoke, package
acceptance, or QA Lab. Stable and full profiles always run exhaustive repo/live
E2E and Docker release-path soak coverage. The beta profile can opt in with
`run_release_soak=true`. Package Acceptance supplies the canonical package
Telegram E2E for every full candidate, so the umbrella does not duplicate that
live poller.
acceptance, or QA parity. Stable and full profiles always run exhaustive
repo/live E2E, Docker release-path, and QA-live soak coverage. The beta profile
adds those lanes only with `run_release_soak=true` or an explicit `qa` or
`qa-live` rerun. Package Acceptance supplies the canonical package Telegram
E2E for every candidate, so the umbrella does not duplicate that live poller.
| Profile | Intended use | Included live/provider coverage |
| -------- | --------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
@@ -247,7 +260,7 @@ Use `rerun_group` to avoid repeating unrelated release boxes:
| Handle | Scope |
| ------------------- | ----------------------------------------------------------------------------------------------- |
| `all` | All Full Release Validation stages. |
| `all` | Phase-default stages; beta without soak excludes broad live/E2E and QA-live. |
| `ci` | Manual full CI child only. |
| `plugin-prerelease` | Plugin Prerelease child only. |
| `release-checks` | All OpenClaw Release Checks stages. |
@@ -301,8 +314,11 @@ need fresh QA evidence.
## Evidence to keep
Keep the `Full Release Validation` summary as the release-level index. It links
child run ids and includes slowest-job tables. For failures, inspect the child
workflow first, then rerun the smallest matching handle above.
child run ids and includes slowest-job tables. Classify failures as product,
harness/tooling/provenance, infrastructure/credential, or wrapper. Only a
confirmed product failure changes the Code SHA. Use one diagnosis, one fix when
needed, and one narrow retry, then reassess; do not automatically rerun `all`.
Narrow evidence is not publish authorization by itself.
For a regular release, record both Code SHA and Release SHA, the reuse policy
and changed-path set, the green Code SHA parent run, and the lightweight Release
scripts/full-release-validation-at-sha.mts
+46 -8
View File
@@ -9,6 +9,7 @@ import { existsSync, mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { pathToFileURL } from "node:url";
import { parse as parseYaml } from "yaml";
import { execGhRead } from "./lib/plain-gh.mjs";
const WORKFLOW = "full-release-validation.yml";
@@ -18,6 +19,8 @@ const RELEASE_EVIDENCE_VERIFIER_PATHS = [
".agents/skills/release-openclaw-ci/scripts/release-ci-summary.mjs",
];
const GH_READ_TIMEOUT_MS = 60_000;
export const FULL_RELEASE_WAIT_TIMEOUT_MINUTES = 720;
export const FULL_RELEASE_WAIT_POLL_INTERVAL_MS = 45_000;
const GH_READ_OPTIONS = {
encoding: "utf8",
killSignal: "SIGKILL",
@@ -67,9 +70,9 @@ function displayValue(value: unknown): string {
function usage() {
console.error(`Usage: node scripts/full-release-validation-at-sha.mjs [--sha <target-sha>] [--target-ref <canonical-release-branch-or-tag>] [--workflow-sha <trusted-main-ref>] [--keep-branch] [--dry-run] [-- -f key=value ...]
Creates temporary remote branches pinned to trusted main release tooling and
the exact target commit, dispatches Full Release Validation with the target
branch as its ref input,
Creates temporary remote branches pinned to the exact Tooling SHA and Validation SHA,
dispatches Full Release Validation with the Validation SHA branch as its ref input
and expected_sha as its immutable identity,
watches the parent run, verifies all child workflow head SHAs match the trusted
workflow lineage through the release evidence manifest, then deletes both
temporary branches by default. --keep-branch retains both branches. Exact-target and changelog-only Release SHA
@@ -213,6 +216,9 @@ export function parseArgs(argv: string[]) {
if (Object.hasOwn(args.inputs, "ref")) {
throw new Error("SHA-pinned release validation reserves the ref input for --sha");
}
if (Object.hasOwn(args.inputs, "expected_sha")) {
throw new Error("SHA-pinned release validation reserves expected_sha for the resolved --sha");
}
if (
args.targetRef &&
!RELEASE_BRANCH_PATTERN.test(args.targetRef) &&
@@ -375,7 +381,8 @@ function readWorkflowRun(parentRunId: string, workflowSha: string) {
function waitForWorkflowRun(parentRunId: string, workflowSha: string) {
let lastSummary = "";
let consecutiveErrors = 0;
for (let attempt = 0; attempt < 480; attempt += 1) {
const deadline = Date.now() + FULL_RELEASE_WAIT_TIMEOUT_MINUTES * 60_000;
while (Date.now() < deadline) {
let suite: Record<string, unknown> | undefined;
try {
suite = readWorkflowRun(parentRunId, workflowSha);
@@ -404,10 +411,19 @@ function waitForWorkflowRun(parentRunId: string, workflowSha: string) {
`Full Release Validation concluded ${stringValue(suite.conclusion, "unknown").toLowerCase()}: https://github.com/openclaw/openclaw/actions/runs/${parentRunId}`,
);
}
Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, 45_000);
const remainingMs = deadline - Date.now();
if (remainingMs <= 0) {
break;
}
Atomics.wait(
new Int32Array(new SharedArrayBuffer(4)),
0,
0,
Math.min(FULL_RELEASE_WAIT_POLL_INTERVAL_MS, remainingMs),
);
}
throw new Error(
`Timed out waiting for Full Release Validation: https://github.com/openclaw/openclaw/actions/runs/${parentRunId}`,
`Timed out after ${FULL_RELEASE_WAIT_TIMEOUT_MINUTES} minutes waiting for Full Release Validation: https://github.com/openclaw/openclaw/actions/runs/${parentRunId}`,
);
}
@@ -431,12 +447,33 @@ export function assertTrustedWorkflowHarness(
runStatus("git", ["cat-file", "-e", `${workflowSha}:${relativePath}`], {
stdio: ["ignore", "ignore", "ignore"],
}).status === 0,
readPath: (relativePath: string) => string = (relativePath) =>
run("git", ["show", `${workflowSha}:${relativePath}`]),
) {
if (!pathExists(TRUSTED_WORKFLOW_PATH)) {
throw new Error(
`trusted workflow SHA ${workflowSha} does not contain ${TRUSTED_WORKFLOW_PATH}`,
);
}
let workflow: unknown;
try {
workflow = parseYaml(readPath(TRUSTED_WORKFLOW_PATH));
} catch (error) {
throw new Error(
`Tooling SHA ${workflowSha} contains invalid ${TRUSTED_WORKFLOW_PATH}: ${error instanceof Error ? error.message : String(error)}`,
);
}
if (
!isRecord(workflow) ||
!isRecord(workflow.on) ||
!isRecord(workflow.on.workflow_dispatch) ||
!isRecord(workflow.on.workflow_dispatch.inputs) ||
!Object.hasOwn(workflow.on.workflow_dispatch.inputs, "expected_sha")
) {
throw new Error(
`Tooling SHA ${workflowSha} is missing workflow_dispatch input expected_sha in ${TRUSTED_WORKFLOW_PATH}`,
);
}
const verifierPath = RELEASE_EVIDENCE_VERIFIER_PATHS.find((relativePath) =>
pathExists(relativePath),
);
@@ -503,12 +540,13 @@ function main() {
const remoteTargetBranchRef = `refs/heads/${targetBranch}`;
const dispatchInputs = {
ref: targetBranch,
expected_sha: targetSha,
...(targetContextRef !== targetSha ? { target_context_ref: targetContextRef } : {}),
...args.inputs,
};
console.log(`Target SHA: ${targetSha}`);
console.log(`Trusted workflow SHA: ${workflowSha}`);
console.log(`Validation SHA: ${targetSha}`);
console.log(`Tooling SHA: ${workflowSha}`);
console.log(`Temporary target ref: ${targetBranch}`);
console.log(`Temporary workflow ref: ${branch}`);
scripts/release-candidate-checklist.mts
+19 -2
View File
@@ -145,7 +145,8 @@ Options:
--windows-node-tag <tag> Exact Windows Node release tag. Required for stable.
--skip-dispatch Require both run ids; do not dispatch workflows.
--skip-local-generated-check Do not run local generated release baseline checks before dispatch.
--skip-parallels Do not run local Parallels fresh/update candidate smoke.
--run-parallels Force candidate Parallels smoke; beta defaults to postpublish release:beta-smoke.
--skip-parallels Force-skip candidate Parallels smoke; stable/full run by default.
--parallels-registry-package-artifact <dir>
Add a verified plugin npm preflight artifact directory. Repeatable.
--skip-telegram Do not run NPM Telegram E2E against the prepared tarball.
@@ -189,7 +190,10 @@ export function parseArgs(argv: string[]) {
>(),
skipDispatch: false,
skipLocalGeneratedCheck: false,
runParallels: false,
skipParallels: false,
parallelsMode: "auto" as "auto" | "run" | "skip",
parallelsSkipReason: "",
skipTelegram: false,
telegramProviderMode: DEFAULT_TELEGRAM_PROVIDER_MODE,
tag: "",
@@ -234,6 +238,7 @@ export function parseArgs(argv: string[]) {
),
booleanFlag("--skip-dispatch", "skipDispatch"),
booleanFlag("--skip-local-generated-check", "skipLocalGeneratedCheck"),
booleanFlag("--run-parallels", "runParallels"),
booleanFlag("--skip-parallels", "skipParallels"),
booleanFlag("--skip-telegram", "skipTelegram"),
],
@@ -270,6 +275,18 @@ export function parseArgs(argv: string[]) {
if (!["beta", "stable", "full"].includes(options.releaseProfile)) {
throw new Error("--release-profile must be beta, stable, or full");
}
if (options.runParallels && options.skipParallels) {
throw new Error("--run-parallels and --skip-parallels cannot be combined");
}
options.parallelsMode = options.runParallels ? "run" : options.skipParallels ? "skip" : "auto";
options.skipParallels =
options.parallelsMode === "skip" ||
(options.parallelsMode === "auto" && options.releaseProfile === "beta");
options.parallelsSkipReason = options.skipParallels
? options.parallelsMode === "auto"
? "deferred to postpublish release:beta-smoke"
: "operator skipped --skip-parallels"
: "";
if (options.skipDispatch && (!options.fullReleaseRunId || !options.npmPreflightRunId)) {
throw new Error("--skip-dispatch requires --full-release-run and --npm-preflight-run");
}
@@ -1597,7 +1614,7 @@ async function runParallelsIfNeeded(
registryPackageTarballPaths: string[],
): Promise<LocalCheckResult> {
if (options.skipParallels) {
return { status: "skipped", reason: "operator skipped --skip-parallels" };
return { status: "skipped", reason: options.parallelsSkipReason };
}
// This function runs inside trusted tooling, not the frozen target checkout.
// Prepare its isolated dependencies here before importing the Parallels harness.
test/scripts/full-release-validation-at-sha.test.ts
+69 -8
View File
@@ -5,6 +5,8 @@ import { join, resolve } from "node:path";
import { describe, expect, it } from "vitest";
import {
assertTrustedWorkflowHarness,
FULL_RELEASE_WAIT_POLL_INTERVAL_MS,
FULL_RELEASE_WAIT_TIMEOUT_MINUTES,
parseArgs,
releaseProfileForTarget,
releaseEvidenceVerificationArgs,
@@ -23,7 +25,7 @@ function runGit(cwd: string, args: string[]): string {
}).trim();
}
function createDispatchFixture() {
function createDispatchFixture(options: { workflowSource?: string } = {}) {
const root = mkdtempSync(join(tmpdir(), "openclaw-release-dispatch-"));
const origin = join(root, "origin.git");
const checkout = join(root, "checkout");
@@ -45,7 +47,14 @@ function createDispatchFixture() {
writeFileSync(join(checkout, "package.json"), '{"version":"2026.8.1"}\n');
writeFileSync(
join(checkout, ".github", "workflows", "full-release-validation.yml"),
"name: Full Release Validation\n",
options.workflowSource ??
`name: Full Release Validation
on:
workflow_dispatch:
inputs:
expected_sha:
required: false
`,
);
writeFileSync(
join(checkout, "scripts", "release-ci-summary.mjs"),
@@ -262,9 +271,15 @@ describe("full-release-validation-at-sha", () => {
);
});
it("reserves the candidate ref for the resolved --sha", () => {
it("reserves immutable candidate identity inputs for the resolved --sha", () => {
expect(() => parseArgs(["-f", "ref=other"])).toThrow("reserves the ref input");
expect(() => parseArgs(["--", "ref=other"])).toThrow("reserves the ref input");
expect(() => parseArgs(["-f", `expected_sha=${"a".repeat(40)}`])).toThrow(
"reserves expected_sha",
);
expect(() => parseArgs(["--", `expected_sha=${"a".repeat(40)}`])).toThrow(
"reserves expected_sha",
);
});
it("validates direct and reused runs through the strict evidence verifier", () => {
@@ -280,9 +295,18 @@ describe("full-release-validation-at-sha", () => {
it("polls the exact workflow run without GraphQL quota use", () => {
const source = readFileSync("scripts/full-release-validation-at-sha.mts", "utf8");
expect(FULL_RELEASE_WAIT_TIMEOUT_MINUTES).toBe(720);
expect(FULL_RELEASE_WAIT_POLL_INTERVAL_MS).toBe(45_000);
expect(source).toContain("actions/runs/${parentRunId}");
expect(source).toContain("workflowRun.head_sha !== workflowSha");
expect(source).toContain("return suite;");
expect(source).toContain("Date.now() + FULL_RELEASE_WAIT_TIMEOUT_MINUTES * 60_000");
expect(source).toContain("const remainingMs = deadline - Date.now();");
expect(source).toContain("Math.min(FULL_RELEASE_WAIT_POLL_INTERVAL_MS, remainingMs)");
expect(source).toContain(
"Timed out after ${FULL_RELEASE_WAIT_TIMEOUT_MINUTES} minutes waiting for Full Release Validation",
);
expect(source).not.toContain("attempt < 480");
expect(source).not.toContain('"graphql"');
expect(source).not.toContain('["run", "watch"');
});
@@ -299,16 +323,31 @@ describe("full-release-validation-at-sha", () => {
const verifierPath = "scripts/release-ci-summary.mjs";
const checked: string[] = [];
expect(
assertTrustedWorkflowHarness("a".repeat(40), (relativePath) => {
checked.push(relativePath);
return relativePath === workflowPath || relativePath === verifierPath;
}),
assertTrustedWorkflowHarness(
"a".repeat(40),
(relativePath) => {
checked.push(relativePath);
return relativePath === workflowPath || relativePath === verifierPath;
},
() => "on:\n workflow_dispatch:\n inputs:\n expected_sha: {}\n",
),
).toBe(verifierPath);
expect(checked).toEqual([workflowPath, verifierPath]);
expect(() => assertTrustedWorkflowHarness("a".repeat(40), () => false)).toThrow(workflowPath);
expect(() =>
assertTrustedWorkflowHarness("a".repeat(40), (relativePath) => relativePath === workflowPath),
assertTrustedWorkflowHarness(
"a".repeat(40),
(relativePath) => relativePath === workflowPath,
() => "on:\n workflow_dispatch:\n inputs:\n expected_sha: {}\n",
),
).toThrow("supported release evidence verifier");
expect(() =>
assertTrustedWorkflowHarness(
"b".repeat(40),
() => true,
() => "on:\n workflow_dispatch:\n inputs: {}\n",
),
).toThrow(`Tooling SHA ${"b".repeat(40)} is missing workflow_dispatch input expected_sha`);
const source = readFileSync("scripts/full-release-validation-at-sha.mts", "utf8");
expect(source.indexOf("assertTrustedWorkflowHarness(workflowSha);")).toBeLessThan(
@@ -386,9 +425,13 @@ describe("full-release-validation-at-sha", () => {
"-f",
`ref=${targetBranch}`,
"-f",
`expected_sha=${fixture.targetSha}`,
"-f",
`target_context_ref=${fixture.releaseRef}`,
]),
);
expect(result.stdout).toContain(`Validation SHA: ${fixture.targetSha}`);
expect(result.stdout).toContain(`Tooling SHA: ${fixture.workflowSha}`);
expect(result.stdout).toContain(
"Parent run: https://github.com/openclaw/openclaw/actions/runs/123",
);
@@ -409,6 +452,24 @@ describe("full-release-validation-at-sha", () => {
}
});
it("rejects pinned old-schema tooling before either remote ref is pushed", () => {
const fixture = createDispatchFixture({
workflowSource: "name: Full Release Validation\non:\n workflow_dispatch:\n",
});
try {
const result = fixture.run(["--workflow-sha", fixture.workflowSha]);
expect(result.status).toBe(1);
expect(result.stderr).toContain(`Tooling SHA ${fixture.workflowSha}`);
expect(result.stderr).toContain("missing workflow_dispatch input expected_sha");
expect(fixture.readCalls(fixture.gitCallsPath).filter((args) => args[0] === "push")).toEqual(
[],
);
expect(readFileSync(fixture.ghCallsPath, "utf8")).toBe("");
} finally {
fixture.cleanup();
}
});
it("keeps both temporary refs with --keep-branch", () => {
const fixture = createDispatchFixture();
try {
test/scripts/package-acceptance-workflow.test.ts
+686 -23
View File
@@ -11,10 +11,15 @@ import {
import { resolve } from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { parse } from "yaml";
import { FULL_RELEASE_WAIT_TIMEOUT_MINUTES } from "../../scripts/full-release-validation-at-sha.mts";
import { createReleaseWorkflowMatrixPlan } from "../../scripts/plan-release-workflow-matrix.mjs";
import { useAutoCleanupTempDirTracker } from "../helpers/temp-dir.js";
const PACKAGE_ACCEPTANCE_WORKFLOW = ".github/workflows/package-acceptance.yml";
const LIVE_E2E_WORKFLOW = ".github/workflows/openclaw-live-and-e2e-checks-reusable.yml";
const INSTALL_SMOKE_REUSABLE_WORKFLOW = ".github/workflows/install-smoke-reusable.yml";
const CROSS_OS_RELEASE_CHECKS_REUSABLE_WORKFLOW =
".github/workflows/openclaw-cross-os-release-checks-reusable.yml";
const LIVE_MEDIA_RUNNER_DOCKERFILE = ".github/images/live-media-runner/Dockerfile";
const LIVE_MEDIA_RUNNER_IMAGE = "ghcr.io/openclaw/openclaw-live-media-runner:ubuntu-24.04";
const LIVE_MEDIA_RUNNER_IMAGE_WORKFLOW = ".github/workflows/live-media-runner-image.yml";
@@ -43,6 +48,8 @@ const ANDROID_RELEASE_WORKFLOW = ".github/workflows/android-release.yml";
const STABLE_MAIN_CLOSEOUT_WORKFLOW = ".github/workflows/openclaw-stable-main-closeout.yml";
const WINDOWS_NODE_RELEASE_WORKFLOW = ".github/workflows/windows-node-release.yml";
const FULL_RELEASE_VALIDATION_WORKFLOW = ".github/workflows/full-release-validation.yml";
const CI_WORKFLOW = ".github/workflows/ci.yml";
const PERFORMANCE_WORKFLOW = ".github/workflows/openclaw-performance.yml";
const FULL_RELEASE_CHILD_DISPATCHES = [
{
jobName: "normal_ci",
@@ -124,10 +131,12 @@ type WorkflowStep = {
type WorkflowMatrixEntry = {
advisory?: boolean;
chunk_id?: string;
command?: string;
profiles?: string;
suite_group?: string;
suite_id?: string;
timeout_minutes?: number;
};
type WorkflowJob = {
@@ -199,6 +208,113 @@ function workflowStep(job: WorkflowJob, stepName: string): WorkflowStep {
return step;
}
function jobNeeds(job: WorkflowJob): string[] {
return Array.isArray(job.needs) ? job.needs : job.needs ? [job.needs] : [];
}
function timeoutForProfile(
timeout: number | string | undefined,
profile: "beta" | "stable" | "full",
): number {
if (typeof timeout === "number") {
return timeout;
}
const match = timeout?.match(
/^\$\{\{ inputs\.(?:release_profile|release_test_profile) == 'full' && ([0-9]+) \|\| ([0-9]+) \}\}$/u,
);
if (!match) {
throw new Error(`Unsupported release timeout expression: ${String(timeout)}`);
}
return Number(profile === "full" ? match[1] : match[2]);
}
function evaluatedJobTimeouts(path: string, jobName: string, job: WorkflowJob): number[] {
const timeout = job["timeout-minutes"];
if (typeof timeout === "number") {
return [timeout];
}
if (timeout?.includes("inputs.release_")) {
return (["beta", "stable", "full"] as const).map((profile) =>
timeoutForProfile(timeout, profile),
);
}
if (timeout !== "${{ matrix.timeout_minutes }}") {
throw new Error(`Unsupported timeout for ${path}:${jobName}: ${String(timeout)}`);
}
const matrix = (job.strategy as { matrix?: unknown } | undefined)?.matrix;
if (matrix && typeof matrix === "object" && "include" in matrix) {
const include = (matrix as { include?: WorkflowMatrixEntry[] }).include;
if (!Array.isArray(include) || include.length === 0) {
throw new Error(`Missing static timeout matrix for ${path}:${jobName}`);
}
return include.map((entry) => {
if (typeof entry.timeout_minutes !== "number") {
throw new Error(`Missing matrix timeout for ${path}:${jobName}`);
}
return entry.timeout_minutes;
});
}
if (path === LIVE_E2E_WORKFLOW && jobName === "validate_docker_e2e") {
return (["beta", "stable", "full"] as const).flatMap((releaseProfile) =>
createReleaseWorkflowMatrixPlan({
includeReleasePathSuites: true,
releaseProfile,
}).dockerE2e.matrix.include.map((entry: WorkflowMatrixEntry) => {
if (typeof entry.timeout_minutes !== "number") {
throw new Error(`Missing planned timeout for ${releaseProfile}:${entry.chunk_id}`);
}
return entry.timeout_minutes;
}),
);
}
throw new Error(`Missing matrix timeout evaluator for ${path}:${jobName}`);
}
function pluginPrereleaseTimeoutFloor(
pluginPrerelease: Workflow,
liveE2e: Workflow,
profile: "beta" | "stable" | "full",
): number {
const preflight = pluginPrerelease.jobs?.preflight;
const dockerSuite = pluginPrerelease.jobs?.["plugin-prerelease-docker-suite"];
const suite = pluginPrerelease.jobs?.["plugin-prerelease-suite"];
const validateSelectedRef = liveE2e.jobs?.validate_selected_ref;
const prepareImage = liveE2e.jobs?.prepare_docker_e2e_image;
const imageReady = liveE2e.jobs?.docker_e2e_image_ready;
const dockerLanes = liveE2e.jobs?.validate_docker_lanes;
if (
!preflight ||
!dockerSuite ||
!suite ||
!validateSelectedRef ||
!prepareImage ||
!imageReady ||
!dockerLanes
) {
throw new Error("Missing plugin prerelease timeout-chain job");
}
expect(jobNeeds(dockerSuite)).toContain("preflight");
expect(jobNeeds(prepareImage)).toEqual(["validate_selected_ref"]);
expect(jobNeeds(imageReady)).toEqual(["prepare_docker_e2e_image"]);
expect(jobNeeds(dockerLanes)).toEqual(
expect.arrayContaining(["prepare_docker_e2e_image", "docker_e2e_image_ready"]),
);
expect(jobNeeds(suite)).toContain("plugin-prerelease-docker-suite");
return [
timeoutForProfile(preflight["timeout-minutes"], profile),
timeoutForProfile(validateSelectedRef["timeout-minutes"], profile),
timeoutForProfile(prepareImage["timeout-minutes"], profile),
timeoutForProfile(imageReady["timeout-minutes"], profile),
timeoutForProfile(dockerLanes["timeout-minutes"], profile),
timeoutForProfile(suite["timeout-minutes"], profile),
].reduce((total, value) => total + value, 0);
}
function runFullReleaseInputValidation(releaseProfile: string, skipTelegram: string) {
const step = workflowStep(
workflowJob(FULL_RELEASE_VALIDATION_WORKFLOW, "resolve_target"),
@@ -214,7 +330,13 @@ function runFullReleaseInputValidation(releaseProfile: string, skipTelegram: str
});
}
function runReleaseChecksInputValidation(releaseProfile: string, skipTelegram: string) {
function runReleaseChecksInputValidation(
releaseProfile: string,
skipTelegram: string,
rerunGroup = "all",
runReleaseSoak = "false",
liveSuiteFilter = "",
) {
const step = workflowStep(
workflowJob(RELEASE_CHECKS_WORKFLOW, "resolve_target"),
"Capture selected inputs",
@@ -230,6 +352,7 @@ function runReleaseChecksInputValidation(releaseProfile: string, skipTelegram: s
GITHUB_OUTPUT: outputPath,
PATH: process.env.PATH,
RELEASE_FAIL_FAST_INPUT: "false",
RELEASE_LIVE_SUITE_FILTER_INPUT: liveSuiteFilter,
RELEASE_MODE_INPUT: "both",
RELEASE_PROFILE_INPUT: releaseProfile,
RELEASE_PROVIDER_INPUT: "openai",
@@ -237,9 +360,9 @@ function runReleaseChecksInputValidation(releaseProfile: string, skipTelegram: s
RELEASE_QA_SLACK_LIVE_CI_ENABLED: "false",
RELEASE_QA_WHATSAPP_LIVE_CI_ENABLED: "false",
RELEASE_REF_INPUT: "main",
RELEASE_RERUN_GROUP_INPUT: "all",
RELEASE_RERUN_GROUP_INPUT: rerunGroup,
RELEASE_RUN_MATURITY_SCORECARD_INPUT: "false",
RELEASE_RUN_RELEASE_SOAK_INPUT: "false",
RELEASE_RUN_RELEASE_SOAK_INPUT: runReleaseSoak,
RELEASE_SKIP_PACKAGE_TELEGRAM_E2E_INPUT: skipTelegram,
},
});
@@ -411,7 +534,16 @@ if (args[0] === "workflow" && args[1] === "run") {
`,
);
chmodSync(ghPath, 0o755);
writeFileSync(sleepPath, "#!/bin/sh\nexit 0\n");
writeFileSync(
sleepPath,
`#!/bin/sh
if [ -n "\${MOCK_SLEEP_SIGNAL:-}" ] && [ ! -e "\${MOCK_SLEEP_SIGNAL_SENT}" ]; then
: > "\${MOCK_SLEEP_SIGNAL_SENT}"
kill -"\${MOCK_SLEEP_SIGNAL}" "$PPID"
fi
exit 0
`,
);
chmodSync(sleepPath, 0o755);
const parentSha = "a".repeat(40);
@@ -491,6 +623,7 @@ if (args[0] === "workflow" && args[1] === "run") {
MOCK_GH_STATUSES: '["completed"]',
MOCK_GH_STATUS_POLLS: statusPath,
MOCK_GH_WORKFLOW_ID: "789",
MOCK_SLEEP_SIGNAL_SENT: resolve(workdir, "sleep-signal-sent"),
PATH: `${workdir}:${process.env.PATH}`,
...overrides,
},
@@ -2144,9 +2277,21 @@ describe("package acceptance workflow", () => {
expect(adoptionCall?.args).toContain(
"[.workflow_runs[] | select(.display_title == env.DISPATCH_RUN_NAME and .head_branch == env.CHILD_WORKFLOW_REF) | .id]",
);
expect(calls.filter(({ args }) => args[0] === "run" && args[1] === "cancel")).toHaveLength(0);
},
);
it("leaves the adopted child running when the monitor receives SIGTERM", () => {
const { calls, result } = runFullReleaseChildDispatch(FULL_RELEASE_CHILD_DISPATCHES[0], {
MOCK_GH_STATUSES: '["in_progress"]',
MOCK_SLEEP_SIGNAL: "TERM",
});
expect(result.status, `${result.stdout}\n${result.stderr}`).toBeNull();
expect(result.signal).toBe("SIGTERM");
expect(calls.filter(({ args }) => args[0] === "run" && args[1] === "cancel")).toHaveLength(0);
});
it.each(FULL_RELEASE_CHILD_DISPATCHES)(
"refuses duplicate exact adoption candidates for $jobName",
(child) => {
@@ -2193,7 +2338,7 @@ describe("package acceptance workflow", () => {
);
it.each(FULL_RELEASE_CHILD_DISPATCHES)(
"cancels exactly the adopted $jobName child when monitoring fails unexpectedly",
"leaves the adopted $jobName child running when monitoring fails unexpectedly",
(child) => {
const { calls, result } = runFullReleaseChildDispatch(child, {
MOCK_GH_STATUS_ERROR: "HTTP 403: Resource not accessible by integration",
@@ -2201,9 +2346,10 @@ describe("package acceptance workflow", () => {
expect(result.status).toBe(1);
expect(result.stderr).toContain("HTTP 403");
expect(calls.filter(({ args }) => args[0] === "run" && args[1] === "cancel")).toEqual([
expect.objectContaining({ args: ["run", "cancel", "101"] }),
]);
expect(
calls.filter(({ args }) => args[0] === "run" && args[1] === "cancel"),
`${result.stdout}\n${result.stderr}\n${JSON.stringify(calls)}`,
).toEqual([]);
},
);
@@ -2238,7 +2384,7 @@ describe("package acceptance workflow", () => {
},
);
it("keeps CI fail-fast job lookups advisory but npm Telegram job lookups fail-closed", () => {
it("keeps CI fail-fast job lookups advisory and npm Telegram fail-closed without cancellation", () => {
const overrides = {
FAIL_FAST: "true",
MOCK_GH_JOBS_ERROR: "HTTP 403: Resource not accessible by integration",
@@ -2261,7 +2407,7 @@ describe("package acceptance workflow", () => {
expect(
npmTelegram.calls.filter(({ args }) => args[0] === "run" && args[1] === "cancel"),
`${npmTelegram.result.stdout}\n${npmTelegram.result.stderr}\n${JSON.stringify(npmTelegram.calls)}`,
).toHaveLength(1);
).toEqual([]);
});
it.each([
@@ -3514,6 +3660,78 @@ describe("package artifact reuse", () => {
},
);
it.each([
["beta", "all", "false", "false", "false"],
["beta", "all", "true", "true", "true"],
["stable", "all", "false", "true", "true"],
["full", "all", "false", "true", "true"],
["beta", "qa", "false", "false", "true"],
["beta", "qa-live", "false", "false", "true"],
])(
"normalizes QA-live scheduling for profile=%s group=%s soak=%s",
(releaseProfile, rerunGroup, runReleaseSoak, expectedSoak, expectedScheduled) => {
const { outputPath, result } = runReleaseChecksInputValidation(
releaseProfile,
"false",
rerunGroup,
runReleaseSoak,
);
expect(result.status, result.stderr).toBe(0);
const output = readFileSync(outputPath, "utf8");
expect(output).toContain(`run_release_soak=${expectedSoak}\n`);
expect(output).toContain(`qa_live_scheduled=${expectedScheduled}\n`);
},
);
it("schedules only the selected QA-live lane for an all-group QA filter", () => {
const { outputPath, result } = runReleaseChecksInputValidation(
"beta",
"false",
"all",
"false",
"qa-live-telegram",
);
expect(result.status, result.stderr).toBe(0);
const output = readFileSync(outputPath, "utf8");
expect(output).toContain("qa_live_scheduled=true\n");
expect(output).toContain("qa_live_telegram_enabled=true\n");
for (const lane of ["matrix", "buzz", "discord", "whatsapp", "slack"]) {
expect(output).toContain(`qa_live_${lane}_enabled=false\n`);
}
});
it("does not schedule QA-live for an all-group repo live filter without soak", () => {
const { outputPath, result } = runReleaseChecksInputValidation(
"beta",
"false",
"all",
"false",
"repo-e2e",
);
expect(result.status, result.stderr).toBe(0);
const output = readFileSync(outputPath, "utf8");
expect(output).toContain("qa_live_scheduled=false\n");
expect(output).toContain("repo_live_suite_filter=repo-e2e\n");
});
it("does not let a QA-live filter override an unrelated rerun group", () => {
const { outputPath, result } = runReleaseChecksInputValidation(
"beta",
"false",
"install-smoke",
"false",
"qa-live-telegram",
);
expect(result.status, result.stderr).toBe(0);
const output = readFileSync(outputPath, "utf8");
expect(output).toContain("qa_live_scheduled=false\n");
expect(output).toContain("qa_live_telegram_enabled=true\n");
});
it("summarizes Telegram deferral only when Package Acceptance is scheduled", () => {
const scheduled = runFullReleaseTargetSummary("release-checks", "true");
const unrelated = runFullReleaseTargetSummary("ci", "true");
@@ -3717,7 +3935,9 @@ describe("package artifact reuse", () => {
OPENCLAW_QA_CONVEX_SITE_URL: "${{ secrets.OPENCLAW_QA_CONVEX_SITE_URL }}",
});
expect(releaseJob.permissions).toEqual({ contents: "read", "pull-requests": "read" });
expect(releaseJob.if).toContain('contains(fromJSON(\'["all","qa","qa-live"]\')');
expect(releaseJob.if).toBe(
"needs.resolve_target.outputs.qa_live_scheduled == 'true' && needs.resolve_target.outputs.qa_live_matrix_enabled == 'true'",
);
expect(releaseJob.with).toMatchObject({
expected_sha: "${{ needs.resolve_target.outputs.revision }}",
fail_fast: "${{ fromJSON(needs.resolve_target.outputs.fail_fast) }}",
@@ -3789,8 +4009,9 @@ describe("package artifact reuse", () => {
OPENCLAW_QA_CONVEX_SITE_URL: "${{ secrets.OPENCLAW_QA_CONVEX_SITE_URL }}",
});
expect(releaseJob.permissions).toEqual({ contents: "read", "pull-requests": "read" });
expect(releaseJob.if).toContain('contains(fromJSON(\'["all","qa","qa-live"]\')');
expect(releaseJob.if).toContain("needs.resolve_target.outputs.qa_live_buzz_enabled == 'true'");
expect(releaseJob.if).toBe(
"needs.resolve_target.outputs.qa_live_scheduled == 'true' && needs.resolve_target.outputs.qa_live_buzz_enabled == 'true'",
);
expect(releaseJob.with).toMatchObject({
buzz_scenario: "channel-canary,channel-mention-gating",
expected_sha: "${{ needs.resolve_target.outputs.revision }}",
@@ -3826,6 +4047,46 @@ describe("package artifact reuse", () => {
expect(requireBuzz.run).toContain("exit 1");
});
it("runs QA-live on soak or explicit QA groups, not beta all by default", () => {
const workflow = readFileSync(RELEASE_CHECKS_WORKFLOW, "utf8");
const resolveTarget = workflowJob(RELEASE_CHECKS_WORKFLOW, "resolve_target");
const liveJobs = [
["qa_live_release_checks", "qa_live_matrix_enabled"],
["qa_live_buzz_release_checks", "qa_live_buzz_enabled"],
["qa_live_telegram_release_checks", "qa_live_telegram_enabled"],
["qa_live_discord_release_checks", "qa_live_discord_enabled"],
["qa_live_whatsapp_release_checks", "qa_live_whatsapp_enabled"],
["qa_live_slack_release_checks", "qa_live_slack_enabled"],
] as const;
const selection = "needs.resolve_target.outputs.qa_live_scheduled == 'true'";
expect(resolveTarget.outputs?.qa_live_scheduled).toBe(
"${{ steps.inputs.outputs.qa_live_scheduled }}",
);
for (const [jobName, enabledOutput] of liveJobs) {
expect(workflowJob(RELEASE_CHECKS_WORKFLOW, jobName).if).toBe(
`${selection} && needs.resolve_target.outputs.${enabledOutput} == 'true'`,
);
}
const verifyStep = workflowStep(
workflowJob(RELEASE_CHECKS_WORKFLOW, "summary"),
"Verify release check results",
);
expect(verifyStep.env?.QA_LIVE_TELEGRAM_SELECTED).toBe(
`\${{ ${selection} && needs.resolve_target.outputs.qa_live_telegram_enabled == 'true' }}`,
);
const kickoffSummary = workflowStep(resolveTarget, "Summarize validated ref");
expect(kickoffSummary.env?.QA_LIVE_SCHEDULED).toBe(
"${{ steps.inputs.outputs.qa_live_scheduled }}",
);
expect(kickoffSummary.run).toContain("- QA-live scheduled:");
expect(kickoffSummary.run).toContain("- QA-live lane eligibility:");
expect(kickoffSummary.run).not.toContain("- QA live lanes:");
expect(workflow).not.toContain('contains(fromJSON(\'["qa","qa-live"]\')');
});
it("runs live transport lanes nightly while release checks stay gated", () => {
const releaseWorkflow = readFileSync(RELEASE_CHECKS_WORKFLOW, "utf8");
const qaWorkflow = readFileSync(QA_LIVE_TRANSPORTS_WORKFLOW, "utf8");
@@ -4290,11 +4551,9 @@ describe("package artifact reuse", () => {
expect(npmTelegramJob.name).toBe("Run package Telegram E2E");
expect(npmTelegramJob.needs).toEqual(["resolve_target", "evidence_reuse"]);
expect(npmTelegramJob["timeout-minutes"]).toBe(
"${{ inputs.release_profile == 'full' && 360 || 60 }}",
);
expect(performanceJob["timeout-minutes"]).toBe(
"${{ inputs.release_profile == 'full' && 360 || 120 }}",
);
expect(performanceJob["timeout-minutes"]).toBe(360);
expect(npmTelegramJob.if).toContain(
'contains(fromJSON(\'["all","npm-telegram"]\'), inputs.rerun_group)',
);
@@ -4318,6 +4577,7 @@ describe("package artifact reuse", () => {
SKIP_PACKAGE_TELEGRAM_E2E: "${{ inputs.skip_package_telegram_e2e }}",
});
expectTextToIncludeAll(targetSummaryStep.run, [
"Validation SHA:",
"Package Acceptance Telegram E2E deferred:",
"Package Telegram E2E: deferred by \\`skip_package_telegram_e2e\\`",
]);
@@ -4378,7 +4638,7 @@ describe("package artifact reuse", () => {
'args+=(-f cross_os_suite_filter="$CROSS_OS_SUITE_FILTER")',
'case "$RERUN_GROUP" in',
"release-checks|install-smoke|cross-os|live-e2e|package|qa|qa-parity|qa-live)",
"cancel-in-progress: ${{ (inputs.ref == 'main' && inputs.rerun_group == 'all') || startsWith(inputs.ref, 'tideclaw/alpha/') || startsWith(inputs.ref, 'release/') }}",
"cancel-in-progress: false",
"Verify release checks accepted Tideclaw alpha advisory lanes",
"release_checks_advisory_only",
"release_check_blocking_job",
@@ -5923,16 +6183,17 @@ wait_for_run plugin-clawhub-new.yml 123 "${expectedSha}" || status=$?
expect(mismatchedWaitSha.status, mismatchedWaitSha.stderr).toBe(0);
});
it("keeps release workflow setup and timeout budgets bounded", () => {
const fullRelease = readWorkflow(FULL_RELEASE_VALIDATION_WORKFLOW);
it("keeps release workflow setup aligned", () => {
const releaseChecks = readWorkflow(RELEASE_CHECKS_WORKFLOW);
const crossOs = readWorkflow(".github/workflows/openclaw-cross-os-release-checks-reusable.yml");
const installSmoke = readWorkflow(INSTALL_SMOKE_REUSABLE_WORKFLOW);
const crossOs = readWorkflow(CROSS_OS_RELEASE_CHECKS_REUSABLE_WORKFLOW);
const liveE2e = readWorkflow(LIVE_E2E_WORKFLOW);
const qaLive = readWorkflow(QA_LIVE_TRANSPORTS_WORKFLOW);
const releaseWorkflowPaths = [
FULL_RELEASE_VALIDATION_WORKFLOW,
RELEASE_CHECKS_WORKFLOW,
RELEASE_TELEGRAM_QA_WORKFLOW,
".github/workflows/openclaw-cross-os-release-checks-reusable.yml",
CROSS_OS_RELEASE_CHECKS_REUSABLE_WORKFLOW,
LIVE_E2E_WORKFLOW,
NPM_TELEGRAM_WORKFLOW,
".github/workflows/openclaw-release-publish.yml",
@@ -5950,8 +6211,6 @@ wait_for_run plugin-clawhub-new.yml 123 "${expectedSha}" || status=$?
expect(workflow.env?.PNPM_VERSION, workflowPath).toBeUndefined();
}
expect(fullRelease.jobs?.release_checks?.["timeout-minutes"]).toBe(240);
expect(fullRelease.jobs?.prepare_release_package).toBeUndefined();
expect(releaseChecks.jobs?.prepare_release_package?.["timeout-minutes"]).toBe(15);
expect(
workflowStep(
@@ -5959,7 +6218,21 @@ wait_for_run plugin-clawhub-new.yml 123 "${expectedSha}" || status=$?
"Setup Node environment",
).with?.["install-deps"],
).toBe("true");
expect(installSmoke.jobs?.preflight?.["timeout-minutes"]).toBe(15);
expect(installSmoke.jobs?.["install-smoke-fast"]?.["timeout-minutes"]).toBe(120);
expect(installSmoke.jobs?.root_dockerfile_image?.["timeout-minutes"]).toBe(60);
expect(installSmoke.jobs?.root_dockerfile_image_ready?.["timeout-minutes"]).toBe(5);
expect(installSmoke.jobs?.qr_package_install_smoke?.["timeout-minutes"]).toBe(30);
expect(installSmoke.jobs?.root_dockerfile_smokes?.["timeout-minutes"]).toBe(90);
expect(installSmoke.jobs?.installer_smoke?.["timeout-minutes"]).toBe(150);
expect(installSmoke.jobs?.bun_global_install_smoke?.["timeout-minutes"]).toBe(60);
expect(installSmoke.jobs?.["docker-e2e-fast"]?.["timeout-minutes"]).toBe(12);
expect(crossOs.jobs?.prepare?.["timeout-minutes"]).toBe(90);
expect(crossOs.jobs?.cross_os_release_checks?.["timeout-minutes"]).toBe(60);
expect(qaLive.jobs?.authorize_actor?.["timeout-minutes"]).toBe(10);
expect(qaLive.jobs?.validate_selected_ref?.["timeout-minutes"]).toBe(30);
expect(liveE2e.jobs?.validate_live_suite_filter?.["timeout-minutes"]).toBe(10);
expect(liveE2e.jobs?.plan_release_workflow_matrices?.["timeout-minutes"]).toBe(10);
expect(liveE2e.jobs?.validate_release_live_cache?.["timeout-minutes"]).toBe(20);
expect(readFileSync(LIVE_E2E_WORKFLOW, "utf8")).toContain(
"timeout --foreground --kill-after=30s 8m pnpm test:live:cache",
@@ -5967,6 +6240,396 @@ wait_for_run plugin-clawhub-new.yml 123 "${expectedSha}" || status=$?
expect(readFileSync(LIVE_E2E_WORKFLOW, "utf8")).toContain("live-cache attempt ${attempt}/2");
});
it("keeps known bounded dominant child paths below parent monitors", () => {
const fullRelease = readWorkflow(FULL_RELEASE_VALIDATION_WORKFLOW);
const pluginPrerelease = readWorkflow(PLUGIN_PRERELEASE_WORKFLOW);
const liveE2e = readWorkflow(LIVE_E2E_WORKFLOW);
const releaseChecks = readWorkflow(RELEASE_CHECKS_WORKFLOW);
const installSmoke = readWorkflow(INSTALL_SMOKE_REUSABLE_WORKFLOW);
const crossOs = readWorkflow(CROSS_OS_RELEASE_CHECKS_REUSABLE_WORKFLOW);
const packageAcceptance = readWorkflow(PACKAGE_ACCEPTANCE_WORKFLOW);
const qaLive = readWorkflow(QA_LIVE_TRANSPORTS_WORKFLOW);
const performance = readWorkflow(PERFORMANCE_WORKFLOW);
const profiles = ["beta", "stable", "full"] as const;
const ciPreflight = workflowJob(CI_WORKFLOW, "preflight");
const ciIos = workflowJob(CI_WORKFLOW, "ios-build");
const ciGate = workflowJob(CI_WORKFLOW, "ci-gate");
expect(jobNeeds(ciIos)).toEqual(["preflight"]);
expect(jobNeeds(ciGate)).toEqual(expect.arrayContaining(["preflight", "ios-build"]));
const ciPath = [
timeoutForProfile(ciPreflight["timeout-minutes"], "beta"),
timeoutForProfile(ciIos["timeout-minutes"], "beta"),
timeoutForProfile(ciGate["timeout-minutes"], "beta"),
];
expect(ciPath).toEqual([20, 150, 5]);
const ciChildTimeout = ciPath.reduce((total, timeout) => total + timeout, 0);
expect(ciChildTimeout).toBe(175);
const ciParentTimeout = timeoutForProfile(
fullRelease.jobs?.normal_ci?.["timeout-minutes"],
"beta",
);
expect(ciChildTimeout).toBeLessThanOrEqual(ciParentTimeout);
expect(ciParentTimeout - ciChildTimeout).toBeGreaterThanOrEqual(60);
expect(liveE2e.jobs?.validate_selected_ref?.["timeout-minutes"]).toBe(30);
const pluginMonitorTimeout = fullRelease.jobs?.plugin_prerelease?.["timeout-minutes"];
const pluginChildTimeouts = Object.fromEntries(
profiles.map((profile) => [
profile,
pluginPrereleaseTimeoutFloor(pluginPrerelease, liveE2e, profile),
]),
) as Record<(typeof profiles)[number], number>;
const pluginParentTimeouts = Object.fromEntries(
profiles.map((profile) => [profile, timeoutForProfile(pluginMonitorTimeout, profile)]),
) as Record<(typeof profiles)[number], number>;
expect(pluginChildTimeouts).toEqual({ beta: 175, stable: 175, full: 205 });
expect(pluginParentTimeouts).toEqual({ beta: 240, stable: 240, full: 300 });
for (const profile of profiles) {
expect(
pluginParentTimeouts[profile] - pluginChildTimeouts[profile],
`plugin-prerelease:${profile}`,
).toBeGreaterThanOrEqual(60);
}
const releasePackageJob = workflowJob(
RELEASE_CHECKS_WORKFLOW,
"package_acceptance_release_checks",
);
expect(jobNeeds(workflowJob(RELEASE_CHECKS_WORKFLOW, "prepare_release_package"))).toEqual([
"resolve_target",
]);
expect(jobNeeds(releasePackageJob)).toEqual(["resolve_target", "prepare_release_package"]);
expect(jobNeeds(workflowJob(PACKAGE_ACCEPTANCE_WORKFLOW, "package_integrity"))).toEqual([
"resolve_package",
]);
expect(jobNeeds(workflowJob(PACKAGE_ACCEPTANCE_WORKFLOW, "docker_acceptance"))).toEqual([
"resolve_package",
"package_integrity",
]);
expect(jobNeeds(workflowJob(LIVE_E2E_WORKFLOW, "prepare_docker_e2e_image"))).toEqual([
"validate_selected_ref",
]);
expect(jobNeeds(workflowJob(LIVE_E2E_WORKFLOW, "docker_e2e_image_ready"))).toEqual([
"prepare_docker_e2e_image",
]);
expect(jobNeeds(workflowJob(LIVE_E2E_WORKFLOW, "validate_docker_lanes"))).toEqual(
expect.arrayContaining([
"validate_selected_ref",
"prepare_docker_e2e_image",
"docker_e2e_image_ready",
]),
);
expect(jobNeeds(workflowJob(PACKAGE_ACCEPTANCE_WORKFLOW, "summary"))).toContain(
"docker_acceptance",
);
expect(jobNeeds(workflowJob(RELEASE_CHECKS_WORKFLOW, "summary"))).toContain(
"package_acceptance_release_checks",
);
const releasePackagePaths = Object.fromEntries(
profiles.map((profile) => [
profile,
[
timeoutForProfile(releaseChecks.jobs?.resolve_target?.["timeout-minutes"], profile),
timeoutForProfile(
releaseChecks.jobs?.prepare_release_package?.["timeout-minutes"],
profile,
),
timeoutForProfile(packageAcceptance.jobs?.resolve_package?.["timeout-minutes"], profile),
timeoutForProfile(
packageAcceptance.jobs?.package_integrity?.["timeout-minutes"],
profile,
),
timeoutForProfile(liveE2e.jobs?.validate_selected_ref?.["timeout-minutes"], profile),
timeoutForProfile(liveE2e.jobs?.prepare_docker_e2e_image?.["timeout-minutes"], profile),
timeoutForProfile(liveE2e.jobs?.docker_e2e_image_ready?.["timeout-minutes"], profile),
timeoutForProfile(liveE2e.jobs?.validate_docker_lanes?.["timeout-minutes"], profile),
timeoutForProfile(packageAcceptance.jobs?.summary?.["timeout-minutes"], profile),
timeoutForProfile(releaseChecks.jobs?.summary?.["timeout-minutes"], profile),
],
]),
) as Record<(typeof profiles)[number], number[]>;
expect(releasePackagePaths).toEqual({
beta: [30, 15, 60, 10, 30, 60, 5, 60, 5, 5],
stable: [30, 15, 60, 10, 30, 60, 5, 60, 5, 5],
full: [30, 15, 60, 10, 30, 90, 5, 60, 5, 5],
});
const releaseChecksParent = workflowJob(FULL_RELEASE_VALIDATION_WORKFLOW, "release_checks");
expect(releaseChecksParent["runs-on"]).toBe("blacksmith-4vcpu-ubuntu-2404");
expect(releaseChecksParent["timeout-minutes"]).toBe(420);
const releasePackageTimeouts = Object.fromEntries(
profiles.map((profile) => [
profile,
releasePackagePaths[profile].reduce((total, timeout) => total + timeout, 0),
]),
);
expect(releasePackageTimeouts).toEqual({ beta: 280, stable: 280, full: 310 });
for (const profile of profiles) {
const childTimeout = releasePackageTimeouts[profile];
expect(childTimeout, `release-package:${profile}`).toBeLessThanOrEqual(420);
expect(420 - childTimeout, `release-package:${profile}`).toBeGreaterThanOrEqual(60);
}
const releaseSummary = workflowJob(RELEASE_CHECKS_WORKFLOW, "summary");
const releaseCrossOs = workflowJob(RELEASE_CHECKS_WORKFLOW, "cross_os_release_checks");
expect(jobNeeds(releaseCrossOs)).toEqual(["resolve_target", "prepare_release_package"]);
expect(jobNeeds(workflowJob(CROSS_OS_RELEASE_CHECKS_REUSABLE_WORKFLOW, "prepare"))).toEqual([]);
expect(
jobNeeds(workflowJob(CROSS_OS_RELEASE_CHECKS_REUSABLE_WORKFLOW, "cross_os_release_checks")),
).toEqual(["prepare"]);
expect(jobNeeds(releaseSummary)).toContain("cross_os_release_checks");
const releaseCrossOsPath = [
timeoutForProfile(releaseChecks.jobs?.resolve_target?.["timeout-minutes"], "stable"),
timeoutForProfile(releaseChecks.jobs?.prepare_release_package?.["timeout-minutes"], "stable"),
timeoutForProfile(crossOs.jobs?.prepare?.["timeout-minutes"], "stable"),
timeoutForProfile(crossOs.jobs?.cross_os_release_checks?.["timeout-minutes"], "stable"),
timeoutForProfile(releaseChecks.jobs?.summary?.["timeout-minutes"], "stable"),
];
expect(releaseCrossOsPath).toEqual([30, 15, 90, 60, 5]);
const releaseInstall = workflowJob(RELEASE_CHECKS_WORKFLOW, "install_smoke_release_checks");
expect(jobNeeds(releaseInstall)).toEqual(["resolve_target"]);
expect(jobNeeds(workflowJob(INSTALL_SMOKE_REUSABLE_WORKFLOW, "root_dockerfile_image"))).toEqual(
["preflight"],
);
expect(
jobNeeds(workflowJob(INSTALL_SMOKE_REUSABLE_WORKFLOW, "root_dockerfile_image_ready")),
).toEqual(["preflight", "root_dockerfile_image"]);
expect(jobNeeds(workflowJob(INSTALL_SMOKE_REUSABLE_WORKFLOW, "installer_smoke"))).toEqual([
"preflight",
"root_dockerfile_image",
"root_dockerfile_image_ready",
]);
expect(jobNeeds(releaseSummary)).toContain("install_smoke_release_checks");
const releaseInstallPath = [
timeoutForProfile(releaseChecks.jobs?.resolve_target?.["timeout-minutes"], "stable"),
timeoutForProfile(installSmoke.jobs?.preflight?.["timeout-minutes"], "stable"),
timeoutForProfile(installSmoke.jobs?.root_dockerfile_image?.["timeout-minutes"], "stable"),
timeoutForProfile(
installSmoke.jobs?.root_dockerfile_image_ready?.["timeout-minutes"],
"stable",
),
timeoutForProfile(installSmoke.jobs?.installer_smoke?.["timeout-minutes"], "stable"),
timeoutForProfile(releaseChecks.jobs?.summary?.["timeout-minutes"], "stable"),
];
expect(releaseInstallPath).toEqual([30, 15, 60, 5, 150, 5]);
const releaseQaLive = workflowJob(RELEASE_CHECKS_WORKFLOW, "qa_live_release_checks");
expect(jobNeeds(releaseQaLive)).toEqual(["resolve_target"]);
expect(jobNeeds(workflowJob(QA_LIVE_TRANSPORTS_WORKFLOW, "validate_selected_ref"))).toEqual([
"authorize_actor",
]);
expect(jobNeeds(workflowJob(QA_LIVE_TRANSPORTS_WORKFLOW, "run_live_matrix"))).toEqual([
"authorize_actor",
"validate_selected_ref",
]);
expect(jobNeeds(releaseSummary)).toContain("qa_live_release_checks");
const releaseQaLivePath = [
timeoutForProfile(releaseChecks.jobs?.resolve_target?.["timeout-minutes"], "stable"),
timeoutForProfile(qaLive.jobs?.authorize_actor?.["timeout-minutes"], "stable"),
timeoutForProfile(qaLive.jobs?.validate_selected_ref?.["timeout-minutes"], "stable"),
timeoutForProfile(qaLive.jobs?.run_live_matrix?.["timeout-minutes"], "stable"),
timeoutForProfile(releaseChecks.jobs?.summary?.["timeout-minutes"], "stable"),
];
expect(releaseQaLivePath).toEqual([30, 10, 30, 90, 5]);
for (const [pathName, path] of [
["cross-os", releaseCrossOsPath],
["install", releaseInstallPath],
["qa-live", releaseQaLivePath],
] as const) {
const childTimeout = path.reduce((total, timeout) => total + timeout, 0);
expect(childTimeout, `release-checks:${pathName}`).toBeLessThanOrEqual(420);
expect(420 - childTimeout, `release-checks:${pathName}`).toBeGreaterThanOrEqual(60);
}
expect(releaseCrossOsPath.reduce((total, timeout) => total + timeout, 0)).toBe(200);
expect(releaseInstallPath.reduce((total, timeout) => total + timeout, 0)).toBe(265);
expect(releaseQaLivePath.reduce((total, timeout) => total + timeout, 0)).toBe(165);
expect(
jobNeeds(workflowJob(RELEASE_CHECKS_WORKFLOW, "qa_live_telegram_release_checks")),
).toEqual(["resolve_target"]);
expect(jobNeeds(workflowJob(RELEASE_CHECKS_WORKFLOW, "summary"))).toContain(
"qa_live_telegram_release_checks",
);
const releaseTelegramPath = [
timeoutForProfile(releaseChecks.jobs?.resolve_target?.["timeout-minutes"], "beta"),
timeoutForProfile(
releaseChecks.jobs?.qa_live_telegram_release_checks?.["timeout-minutes"],
"beta",
),
timeoutForProfile(releaseChecks.jobs?.summary?.["timeout-minutes"], "beta"),
];
expect(releaseTelegramPath).toEqual([30, 210, 5]);
const releaseTelegramTimeout = releaseTelegramPath.reduce(
(total, timeout) => total + timeout,
0,
);
expect(releaseTelegramTimeout).toBe(245);
expect(420 - releaseTelegramTimeout).toBeGreaterThanOrEqual(60);
const npmTelegramChildTimeout = timeoutForProfile(
workflowJob(NPM_TELEGRAM_WORKFLOW, "run_package_telegram_e2e")["timeout-minutes"],
"beta",
);
expect(npmTelegramChildTimeout).toBe(60);
for (const profile of profiles) {
const parentTimeout = timeoutForProfile(
fullRelease.jobs?.npm_telegram?.["timeout-minutes"],
profile,
);
expect(parentTimeout).toBe(profile === "full" ? 360 : 120);
expect(
parentTimeout - npmTelegramChildTimeout,
`npm-telegram:${profile}`,
).toBeGreaterThanOrEqual(60);
}
const performanceResolve = workflowJob(PERFORMANCE_WORKFLOW, "resolve_target");
const performanceKova = workflowJob(PERFORMANCE_WORKFLOW, "kova");
const performanceSource = workflowJob(PERFORMANCE_WORKFLOW, "source_performance");
const performancePublish = workflowJob(PERFORMANCE_WORKFLOW, "publish");
const performanceArtifactGuard = workflowJob(PERFORMANCE_WORKFLOW, "artifact_only_guard");
expect(jobNeeds(performanceKova)).toEqual(["resolve_target"]);
expect(jobNeeds(performanceSource)).toEqual(["resolve_target"]);
expect(jobNeeds(performancePublish)).toEqual(["resolve_target", "kova", "source_performance"]);
expect(jobNeeds(performanceArtifactGuard)).toEqual(["resolve_target", "kova", "publish"]);
expect(performancePublish.if).toContain("inputs.publish_reports == true");
expect(performanceArtifactGuard.if).toContain("inputs.publish_reports != true");
expect(timeoutForProfile(performanceSource["timeout-minutes"], "beta")).toBeLessThanOrEqual(
timeoutForProfile(performanceKova["timeout-minutes"], "beta"),
);
const performanceArtifactPath = [
timeoutForProfile(performanceResolve["timeout-minutes"], "beta"),
timeoutForProfile(performanceKova["timeout-minutes"], "beta"),
timeoutForProfile(performanceArtifactGuard["timeout-minutes"], "beta"),
];
const performancePublishPath = [
timeoutForProfile(performanceResolve["timeout-minutes"], "beta"),
timeoutForProfile(performanceKova["timeout-minutes"], "beta"),
timeoutForProfile(performancePublish["timeout-minutes"], "beta"),
];
expect(performanceArtifactPath).toEqual([10, 240, 5]);
expect(performancePublishPath).toEqual([10, 240, 30]);
expect(performanceArtifactPath.reduce((total, timeout) => total + timeout, 0)).toBe(255);
expect(performancePublishPath.reduce((total, timeout) => total + timeout, 0)).toBe(280);
const performanceParent = workflowJob(FULL_RELEASE_VALIDATION_WORKFLOW, "performance");
expect(performanceParent["timeout-minutes"]).toBe(360);
expect(
workflowStep(performanceParent, "Dispatch and monitor OpenClaw Performance").run,
).toContain("-f publish_reports=false");
for (const [pathName, path] of [
["artifact-only", performanceArtifactPath],
["publish", performancePublishPath],
] as const) {
const childTimeout = path.reduce((total, timeout) => total + timeout, 0);
expect(childTimeout, `performance:${pathName}`).toBeLessThanOrEqual(360);
expect(360 - childTimeout, `performance:${pathName}`).toBeGreaterThanOrEqual(60);
}
const prepareReleaseCandidate = workflowJob(
FULL_RELEASE_VALIDATION_WORKFLOW,
"prepare_release_candidate",
);
expect(jobNeeds(workflowJob(FULL_RELEASE_VALIDATION_WORKFLOW, "evidence_reuse"))).toEqual([
"resolve_target",
]);
expect(jobNeeds(prepareReleaseCandidate)).toEqual(["resolve_target", "evidence_reuse"]);
expect(prepareReleaseCandidate.with?.prepare_only).toBe(true);
expect(jobNeeds(releaseChecksParent)).toEqual([
"resolve_target",
"evidence_reuse",
"prepare_release_candidate",
]);
expect(jobNeeds(workflowJob(FULL_RELEASE_VALIDATION_WORKFLOW, "summary"))).toContain(
"release_checks",
);
const fullParentPath = [
timeoutForProfile(fullRelease.jobs?.resolve_target?.["timeout-minutes"], "full"),
timeoutForProfile(fullRelease.jobs?.evidence_reuse?.["timeout-minutes"], "full"),
timeoutForProfile(liveE2e.jobs?.validate_selected_ref?.["timeout-minutes"], "full"),
timeoutForProfile(liveE2e.jobs?.prepare_docker_e2e_image?.["timeout-minutes"], "full"),
timeoutForProfile(liveE2e.jobs?.docker_e2e_image_ready?.["timeout-minutes"], "full"),
timeoutForProfile(releaseChecksParent["timeout-minutes"], "full"),
timeoutForProfile(fullRelease.jobs?.summary?.["timeout-minutes"], "full"),
];
expect(fullParentPath).toEqual([10, 10, 30, 90, 5, 420, 5]);
const fullParentTimeoutFloor = fullParentPath.reduce((total, timeout) => total + timeout, 0);
expect(fullParentTimeoutFloor).toBe(570);
expect(FULL_RELEASE_WAIT_TIMEOUT_MINUTES - fullParentTimeoutFloor).toBeGreaterThanOrEqual(60);
});
it("bounds every direct job in nested release workflows", () => {
const boundedWorkflowPaths = [
RELEASE_CHECKS_WORKFLOW,
INSTALL_SMOKE_REUSABLE_WORKFLOW,
CROSS_OS_RELEASE_CHECKS_REUSABLE_WORKFLOW,
LIVE_E2E_WORKFLOW,
PACKAGE_ACCEPTANCE_WORKFLOW,
QA_LIVE_TRANSPORTS_WORKFLOW,
RELEASE_TELEGRAM_QA_WORKFLOW,
NPM_TELEGRAM_WORKFLOW,
];
for (const path of boundedWorkflowPaths) {
const jobs = readWorkflow(path).jobs ?? {};
expect(Object.keys(jobs).length, path).toBeGreaterThan(0);
for (const [jobName, job] of Object.entries(jobs)) {
if (job.uses) {
// GitHub does not allow timeout-minutes on reusable-workflow caller jobs.
expect(job["timeout-minutes"], `${path}:${jobName}`).toBeUndefined();
continue;
}
const evaluatedTimeouts = evaluatedJobTimeouts(path, jobName, job);
expect(evaluatedTimeouts.length, `${path}:${jobName}`).toBeGreaterThan(0);
for (const timeout of evaluatedTimeouts) {
expect(Number.isFinite(timeout), `${path}:${jobName}`).toBe(true);
expect(timeout, `${path}:${jobName}`).toBeGreaterThan(0);
}
}
}
});
it("pins every documented raw Full Release Validation caller to one exact SHA", () => {
const nightly = readFileSync(".agents/skills/release-openclaw-nightly/SKILL.md", "utf8");
const liveUpdater = readFileSync(".agents/skills/openclaw-live-updater/SKILL.md", "utf8");
const releaseCi = readFileSync(".agents/skills/release-openclaw-ci/SKILL.md", "utf8");
const releaseCiNotes = readFileSync(
".agents/skills/release-openclaw-ci/references/release-ci-notes.md",
"utf8",
);
const testing = readFileSync(".agents/skills/openclaw-testing/SKILL.md", "utf8");
const parallels = readFileSync(".agents/skills/openclaw-parallels-smoke/SKILL.md", "utf8");
const maintainer = readFileSync(RELEASE_MAINTAINER_SKILL, "utf8");
const ciDocs = readFileSync("docs/ci.md", "utf8");
const fullReleaseDocs = readFileSync("docs/reference/full-release-validation.md", "utf8");
const releasingDocs = readFileSync("docs/reference/RELEASING.md", "utf8");
expect(nightly).toContain('-f expected_sha="$SHA"');
expectTextToIncludeAll(liveUpdater, [
'MAIN_SHA="<exact-main-sha>"',
'-f ref="$MAIN_SHA"',
'-f expected_sha="$MAIN_SHA"',
]);
for (const text of [releaseCi, fullReleaseDocs, releasingDocs]) {
expectTextToIncludeAll(text, [
'RELEASE_SHA="$(git rev-parse HEAD)"',
"-f ref=extended-stable/YYYY.M.33",
'-f expected_sha="$RELEASE_SHA"',
]);
}
expectTextToIncludeAll(ciDocs, [
'VALIDATION_SHA="<full-commit-sha>"',
'-f ref="$VALIDATION_SHA"',
'-f expected_sha="$VALIDATION_SHA"',
]);
for (const text of [releaseCi, releaseCiNotes, testing, parallels, ciDocs, maintainer]) {
expect(text).toContain("Validation SHA + Tooling SHA");
}
expect(releaseCi).toContain("release lifecycle ledger: Code SHA, Release SHA, and Tooling SHA");
});
it("executes shared release candidate identity validation with its JSON input", () => {
const selectedSha = "a".repeat(40);
const candidate = {
test/scripts/plugin-prerelease-test-plan.test.ts
+108 -11
View File
@@ -38,6 +38,59 @@ function readPluginPrereleaseWorkflow() {
return parse(readFileSync(".github/workflows/plugin-prerelease.yml", "utf8"));
}
function readLiveE2eWorkflow() {
return parse(readFileSync(".github/workflows/openclaw-live-and-e2e-checks-reusable.yml", "utf8"));
}
function jobNeeds(job: { needs?: string | string[] }): string[] {
return Array.isArray(job.needs) ? job.needs : job.needs ? [job.needs] : [];
}
function timeoutForProfile(
timeout: number | string | undefined,
profile: "beta" | "stable" | "full",
): number {
if (typeof timeout === "number") {
return timeout;
}
const match = timeout?.match(
/^\$\{\{ inputs\.(?:release_profile|release_test_profile) == 'full' && ([0-9]+) \|\| ([0-9]+) \}\}$/u,
);
if (!match) {
throw new Error(`Unsupported release timeout expression: ${String(timeout)}`);
}
return Number(profile === "full" ? match[1] : match[2]);
}
function pluginPrereleaseTimeoutFloor(profile: "beta" | "stable" | "full"): number {
const plugin = readPluginPrereleaseWorkflow();
const liveE2e = readLiveE2eWorkflow();
const preflight = plugin.jobs.preflight;
const dockerSuite = plugin.jobs["plugin-prerelease-docker-suite"];
const suite = plugin.jobs["plugin-prerelease-suite"];
const validateSelectedRef = liveE2e.jobs.validate_selected_ref;
const prepareImage = liveE2e.jobs.prepare_docker_e2e_image;
const imageReady = liveE2e.jobs.docker_e2e_image_ready;
const dockerLanes = liveE2e.jobs.validate_docker_lanes;
expect(jobNeeds(dockerSuite)).toContain("preflight");
expect(jobNeeds(prepareImage)).toEqual(["validate_selected_ref"]);
expect(jobNeeds(imageReady)).toEqual(["prepare_docker_e2e_image"]);
expect(jobNeeds(dockerLanes)).toEqual(
expect.arrayContaining(["prepare_docker_e2e_image", "docker_e2e_image_ready"]),
);
expect(jobNeeds(suite)).toContain("plugin-prerelease-docker-suite");
return [
timeoutForProfile(preflight["timeout-minutes"], profile),
timeoutForProfile(validateSelectedRef["timeout-minutes"], profile),
timeoutForProfile(prepareImage["timeout-minutes"], profile),
timeoutForProfile(imageReady["timeout-minutes"], profile),
timeoutForProfile(dockerLanes["timeout-minutes"], profile),
timeoutForProfile(suite["timeout-minutes"], profile),
].reduce((total, value) => total + value, 0);
}
function getDockerLane(name: string) {
const lane = findLaneByName(name);
if (!lane) {
@@ -610,7 +663,7 @@ describe("scripts/lib/plugin-prerelease-test-plan.mts", () => {
).toContain("plugin-prerelease-inspector advisory result");
});
it("keeps release-check reruns independent while cancelling superseded umbrella runs", () => {
it("keeps exact release tuples independent without cancelling adopted children", () => {
const releaseChecksWorkflow = parse(
readFileSync(".github/workflows/openclaw-release-checks.yml", "utf8"),
);
@@ -622,10 +675,26 @@ describe("scripts/lib/plugin-prerelease-test-plan.mts", () => {
"cancel-in-progress": "${{ startsWith(github.ref, 'refs/heads/tideclaw/alpha/') }}",
});
expect(fullReleaseWorkflow.concurrency).toEqual({
group: "full-release-validation-${{ inputs.ref }}-${{ inputs.rerun_group }}",
"cancel-in-progress":
"${{ (inputs.ref == 'main' && inputs.rerun_group == 'all') || startsWith(inputs.ref, 'tideclaw/alpha/') || startsWith(inputs.ref, 'release/') }}",
group:
"full-release-validation-${{ inputs.expected_sha || inputs.ref }}-${{ github.sha }}-${{ inputs.rerun_group }}",
"cancel-in-progress": false,
});
expect(fullReleaseWorkflow.on.workflow_dispatch.inputs.expected_sha).toEqual({
description: "Optional full Validation SHA that ref must resolve to",
required: false,
default: "",
type: "string",
});
const resolveTargetStep = fullReleaseWorkflow.jobs.resolve_target.steps.find(
(step: WorkflowStep) => step.name === "Resolve target SHA",
);
const targetSummaryStep = fullReleaseWorkflow.jobs.resolve_target.steps.find(
(step: WorkflowStep) => step.name === "Summarize target",
);
expect(resolveTargetStep.env?.EXPECTED_SHA).toBe("${{ inputs.expected_sha }}");
expect(resolveTargetStep.run).toContain('--expected-sha "$EXPECTED_SHA"');
expect(targetSummaryStep.run).toContain("- Validation SHA:");
expect(targetSummaryStep.run).not.toContain("- Code SHA:");
expect(releaseChecksWorkflow.jobs.resolve_target["runs-on"]).toBe("ubuntu-24.04");
expect(releaseChecksWorkflow.jobs.prepare_release_package["runs-on"]).toBe("ubuntu-24.04");
expect(releaseChecksWorkflow.jobs.summary["runs-on"]).toBe("ubuntu-24.04");
@@ -634,16 +703,14 @@ describe("scripts/lib/plugin-prerelease-test-plan.mts", () => {
"docker_runtime_assets_preflight",
"normal_ci",
"plugin_prerelease",
"release_checks",
"npm_telegram",
"summary",
]) {
expect(fullReleaseWorkflow.jobs[jobName]["runs-on"]).toBe("ubuntu-24.04");
}
expect(fullReleaseWorkflow.jobs.release_checks["runs-on"]).toBe("blacksmith-4vcpu-ubuntu-2404");
expect(fullReleaseWorkflow.jobs.performance["runs-on"]).toBe("blacksmith-4vcpu-ubuntu-2404");
expect(fullReleaseWorkflow.jobs.normal_ci["timeout-minutes"]).toBe(
"${{ inputs.release_profile != 'beta' && 240 || 60 }}",
);
expect(fullReleaseWorkflow.jobs.normal_ci["timeout-minutes"]).toBe(240);
expect(fullReleaseWorkflow.jobs.normal_ci.needs).toEqual(["resolve_target", "evidence_reuse"]);
expect(fullReleaseWorkflow.jobs.normal_ci.if).toContain(
"needs.resolve_target.result == 'success'",
@@ -670,10 +737,29 @@ describe("scripts/lib/plugin-prerelease-test-plan.mts", () => {
(step: WorkflowStep) => step.name === "Build and smoke test final Docker runtime image",
),
).toBe(false);
expect(fullReleaseWorkflow.jobs.plugin_prerelease["timeout-minutes"]).toBe(
"${{ inputs.release_profile == 'full' && 300 || inputs.release_profile == 'stable' && 240 || 60 }}",
const pluginMonitorTimeout = fullReleaseWorkflow.jobs.plugin_prerelease["timeout-minutes"];
const childTimeoutFloors = {
beta: pluginPrereleaseTimeoutFloor("beta"),
stable: pluginPrereleaseTimeoutFloor("stable"),
full: pluginPrereleaseTimeoutFloor("full"),
};
const parentTimeouts = {
beta: timeoutForProfile(pluginMonitorTimeout, "beta"),
stable: timeoutForProfile(pluginMonitorTimeout, "stable"),
full: timeoutForProfile(pluginMonitorTimeout, "full"),
};
expect(childTimeoutFloors).toEqual({ beta: 175, stable: 175, full: 205 });
expect(parentTimeouts).toEqual({ beta: 240, stable: 240, full: 300 });
for (const profile of ["beta", "stable", "full"] as const) {
expect(parentTimeouts[profile] - childTimeoutFloors[profile], profile).toBeGreaterThanOrEqual(
60,
);
}
expect(fullReleaseWorkflow.jobs.release_checks["timeout-minutes"]).toBe(420);
expect(fullReleaseWorkflow.jobs.npm_telegram["timeout-minutes"]).toBe(
"${{ inputs.release_profile == 'full' && 360 || 120 }}",
);
expect(fullReleaseWorkflow.jobs.release_checks["timeout-minutes"]).toBe(240);
expect(fullReleaseWorkflow.jobs.performance["timeout-minutes"]).toBe(360);
const fullReleaseSource = readFileSync(".github/workflows/full-release-validation.yml", "utf8");
expect(fullReleaseWorkflow.on.workflow_dispatch.inputs.fail_fast).toEqual({
description:
@@ -699,6 +785,17 @@ describe("scripts/lib/plugin-prerelease-test-plan.mts", () => {
expect(fullReleaseSource).toContain(
"npm-telegram-beta-e2e.yml has failed child jobs before the workflow completed; cancelling the remaining run.",
);
expect(fullReleaseSource).not.toContain("trap cancel_child");
expect(fullReleaseSource).not.toContain("cancel_child_on_failure");
expect(fullReleaseSource).not.toContain("exit_on_parent_signal");
expect(fullReleaseSource).not.toContain("disable_child_cleanup");
expect(fullReleaseSource).toContain(
"Parent cancellation leaves this child running; cancel it explicitly if no longer needed.",
);
expect(fullReleaseSource).toContain(
'if [[ "$child_head_sha" != "$PARENT_WORKFLOW_SHA" ]]; then',
);
expect(fullReleaseSource).toContain("cancel_child\n exit 1");
expect(releaseChecksWorkflow.on.workflow_dispatch.inputs.fail_fast).toEqual({
description: "Stop the Matrix QA lane after its first failed check or scenario",
required: false,
test/scripts/release-candidate-checklist.test.ts
+74
View File
@@ -578,6 +578,79 @@ describe("release candidate checklist", () => {
).toBe("full");
});
it("defaults beta and alpha Parallels to postpublish confidence", () => {
const beta = parseArgs(["--tag", "v2026.5.14-beta.3"]);
const alpha = parseArgs([
"--tag",
"v2026.5.14-alpha.2",
"--workflow-ref",
"tideclaw/alpha/2026-07-10-1200Z",
"--npm-dist-tag",
"alpha",
]);
for (const options of [beta, alpha]) {
expect(options.releaseProfile).toBe("beta");
expect(options.parallelsMode).toBe("auto");
expect(options.skipParallels).toBe(true);
expect(options.parallelsSkipReason).toBe("deferred to postpublish release:beta-smoke");
}
});
it("supports explicit and profile-default Parallels execution", () => {
const beta = parseArgs(["--tag", "v2026.5.14-beta.3", "--run-parallels"]);
const stable = parseArgs(["--tag", "v2026.5.14", "--windows-node-tag", "v0.6.3"]);
const full = parseArgs([
"--tag",
"v2026.5.14",
"--windows-node-tag",
"v0.6.3",
"--release-profile",
"full",
]);
expect(beta).toMatchObject({
parallelsMode: "run",
parallelsSkipReason: "",
skipParallels: false,
});
for (const options of [stable, full]) {
expect(options.parallelsMode).toBe("auto");
expect(options.skipParallels).toBe(false);
expect(options.parallelsSkipReason).toBe("");
}
});
it("supports an explicit Parallels skip without changing persisted state shape", () => {
const options = parseArgs([
"--tag",
"v2026.5.14",
"--windows-node-tag",
"v0.6.3",
"--skip-parallels",
]);
const state = buildReleaseCandidateState(options, {
targetSha: "a".repeat(40),
toolingSha: "b".repeat(40),
});
expect(options).toMatchObject({
parallelsMode: "skip",
parallelsSkipReason: "operator skipped --skip-parallels",
skipParallels: true,
});
expect(state.skipParallels).toBe(true);
expect(state).not.toHaveProperty("parallelsMode");
expect(state).not.toHaveProperty("parallelsSkipReason");
expect(state).not.toHaveProperty("runParallels");
});
it("rejects conflicting Parallels modes", () => {
expect(() =>
parseArgs(["--tag", "v2026.5.14-beta.3", "--run-parallels", "--skip-parallels"]),
).toThrow("--run-parallels and --skip-parallels cannot be combined");
});
it("runs Parallels against the exact prepared candidate tarball", () => {
expect(candidateParallelsArgs(".artifacts/preflight/openclaw.tgz", [], "/trusted")).toEqual([
"exec",
@@ -937,6 +1010,7 @@ describe("release candidate checklist", () => {
duplicateOption("--windows-node-tag", "v0.6.3", "v0.6.4"),
duplicateFlag("--skip-dispatch"),
duplicateFlag("--skip-local-generated-check"),
duplicateFlag("--run-parallels"),
duplicateFlag("--skip-parallels"),
duplicateFlag("--skip-telegram"),
duplicateOption("--telegram-provider-mode", "mock-openai", "live-frontier"),
test/scripts/release-no-push-workflow.test.ts
+15 -11
View File
@@ -309,7 +309,7 @@ describe("release validation no-push transport", () => {
expect(releaseHelper.with?.["persist-credentials"]).toBe(false);
});
it("owns identified children before rejecting a mismatched workflow SHA", () => {
it("records adopted children before monitoring and cancels only a mismatched workflow SHA", () => {
const full = readWorkflow(FULL_RELEASE);
for (const [jobName, stepName] of [
["normal_ci", "Dispatch and monitor CI"],
@@ -326,24 +326,28 @@ describe("release validation no-push transport", () => {
);
expect(dispatchRun.match(/\.head_sha == \$head_sha/gu), jobName).toBeNull();
expect(dispatchRun, jobName).toContain('run_json="$(validate_child_run "$run_id")"');
expect(dispatchRun, jobName).toContain('active_child_run_id="$run_id"');
expect(dispatchRun, jobName).toContain("trap cancel_child EXIT INT TERM");
expect(dispatchRun, jobName).not.toContain("trap cancel_child");
expect(dispatchRun, jobName).not.toContain("cancel_child_on_failure");
expect(dispatchRun, jobName).not.toContain("exit_on_parent_signal");
expect(dispatchRun, jobName).not.toContain("disable_child_cleanup");
expect(
dispatchRun.indexOf('run_json="$(validate_child_run "$run_id")"'),
jobName,
).toBeLessThan(dispatchRun.indexOf('active_child_run_id="$run_id"'));
expect(dispatchRun.indexOf('active_child_run_id="$run_id"'), jobName).toBeLessThan(
dispatchRun.indexOf("trap cancel_child EXIT INT TERM"),
);
expect(dispatchRun.indexOf("trap cancel_child EXIT INT TERM"), jobName).toBeLessThan(
dispatchRun.indexOf('if [[ "$child_head_sha" != "$PARENT_WORKFLOW_SHA" ]]'),
);
).toBeLessThan(dispatchRun.indexOf('echo "run_id=${run_id}" >> "$GITHUB_OUTPUT"'));
expect(
dispatchRun.indexOf('echo "run_id=${run_id}" >> "$GITHUB_OUTPUT"'),
jobName,
).toBeLessThan(dispatchRun.indexOf("poll_count=0"));
expect(
dispatchRun.indexOf('run_json="$(validate_child_run "$run_id")"'),
jobName,
).toBeLessThan(dispatchRun.indexOf('if [[ "$child_head_sha" != "$PARENT_WORKFLOW_SHA" ]]'));
const shaMismatch = dispatchRun.slice(
dispatchRun.indexOf('if [[ "$child_head_sha" != "$PARENT_WORKFLOW_SHA" ]]'),
dispatchRun.indexOf("fail_fast_failed_jobs()"),
);
expect(shaMismatch, jobName).toContain("cancel_child");
expect(shaMismatch, jobName).toContain("trap - EXIT INT TERM");
expect(shaMismatch, jobName).not.toContain("trap");
}
const verify = step(job(full, "summary"), "Verify child workflow results");
test/scripts/resolve-openclaw-ref.test.ts
+16
View File
@@ -143,4 +143,20 @@ describe("scripts/github/resolve-openclaw-ref.sh", () => {
expect(result.stderr).toContain("Ref resolved ambiguously as both branch and tag: ambiguous");
expect(result.stdout).toBe("");
});
it("rejects a branch whose resolved SHA differs from expected_sha before emitting outputs", () => {
const expectedSha = "b".repeat(40);
const result = runResolver(remoteRepo, [
"--ref",
"release/test",
"--expected-sha",
expectedSha,
]);
expect(result.status).toBe(1);
expect(result.stdout).toBe("");
expect(result.stderr).toBe(
`Ref release/test resolved to ${remoteSha}, expected ${expectedSha}.\n`,
);
});
});