From dfdc95e755717e3d87f8185334c86e71e2081ddf Mon Sep 17 00:00:00 2001 From: Dallin Romney Date: Wed, 12 Aug 2026 01:20:18 +0700 Subject: [PATCH] fix(docs): format full release validation reference --- docs/reference/full-release-validation.md | 64 +++++++++++------------ 1 file changed, 32 insertions(+), 32 deletions(-) diff --git a/docs/reference/full-release-validation.md b/docs/reference/full-release-validation.md index 643a8bd1854e..cb5596ebf693 100644 --- a/docs/reference/full-release-validation.md +++ b/docs/reference/full-release-validation.md @@ -144,17 +144,17 @@ the `runtime-assets` Docker target with other stages and is enforced by the umbrella verifier; lanes no longer wait for it before dispatching. A narrower `rerun_group` skips this preflight. -| Stage | Details | -| ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Target resolution | **Job:** `Resolve target ref`
**Child workflow:** none
**Proves:** resolves the release branch, tag, or full commit SHA and records selected inputs.
**Rerun:** rerun the umbrella if this fails. | -| Shared candidate | **Job:** `Prepare shared release candidate`
**Child workflow:** `OpenClaw Live And E2E Checks (Reusable)`
**Proves:** packs and validates one exact-SHA package, builds one functional Docker image, and records immutable package and image artifact tuples for both package-facing child workflows.
**Rerun:** rerun the affected package, plugin-prerelease, cross-OS, or live/E2E group. | -| Docker assets preflight | **Job:** `Verify Docker runtime image assets`
**Child workflow:** none
**Proves:** the `runtime-assets` Docker build target still succeeds before any other stage dispatches. Runs only for `rerun_group=all`.
**Rerun:** rerun the umbrella with `rerun_group=all`. | -| Vitest and normal CI | **Job:** `Run normal full CI`
**Child workflow:** `CI`
**Proves:** manual full CI graph against the target ref, including Linux Node lanes, bundled plugin shards, plugin and channel contract shards, Node 22 compatibility, `check-*`, `check-additional-*`, built-artifact smoke checks, docs checks, Python skills, Windows, macOS, Control UI i18n, and Android via the umbrella.
**Rerun:** `rerun_group=ci`. | -| Plugin prerelease | **Job:** `Run plugin prerelease validation`
**Child workflow:** `Plugin Prerelease`
**Proves:** release-only plugin static checks, agentic plugin coverage, full plugin batch shards, plugin prerelease Docker lanes, and a non-blocking `plugin-inspector-advisory` artifact for compatibility triage.
**Rerun:** `rerun_group=plugin-prerelease`. | +| Stage | Details | +| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Target resolution | **Job:** `Resolve target ref`
**Child workflow:** none
**Proves:** resolves the release branch, tag, or full commit SHA and records selected inputs.
**Rerun:** rerun the umbrella if this fails. | +| Shared candidate | **Job:** `Prepare shared release candidate`
**Child workflow:** `OpenClaw Live And E2E Checks (Reusable)`
**Proves:** packs and validates one exact-SHA package, builds one functional Docker image, and records immutable package and image artifact tuples for both package-facing child workflows.
**Rerun:** rerun the affected package, plugin-prerelease, cross-OS, or live/E2E group. | +| Docker assets preflight | **Job:** `Verify Docker runtime image assets`
**Child workflow:** none
**Proves:** the `runtime-assets` Docker build target still succeeds before any other stage dispatches. Runs only for `rerun_group=all`.
**Rerun:** rerun the umbrella with `rerun_group=all`. | +| Vitest and normal CI | **Job:** `Run normal full CI`
**Child workflow:** `CI`
**Proves:** manual full CI graph against the target ref, including Linux Node lanes, bundled plugin shards, plugin and channel contract shards, Node 22 compatibility, `check-*`, `check-additional-*`, built-artifact smoke checks, docs checks, Python skills, Windows, macOS, Control UI i18n, and Android via the umbrella.
**Rerun:** `rerun_group=ci`. | +| Plugin prerelease | **Job:** `Run plugin prerelease validation`
**Child workflow:** `Plugin Prerelease`
**Proves:** release-only plugin static checks, agentic plugin coverage, full plugin batch shards, plugin prerelease Docker lanes, and a non-blocking `plugin-inspector-advisory` artifact for compatibility triage.
**Rerun:** `rerun_group=plugin-prerelease`. | | Release checks | **Job:** `Run release/live/Docker/QA validation`
**Child workflow:** `OpenClaw Release Checks`
**Proves:** install smoke, cross-OS package checks, stable and prerelease Windows-node release-artifact E2E plus an immutable Windows-node `main` source E2E against the exact release tarball, Package Acceptance, and QA Lab parity. QA-live Matrix, Buzz, and Telegram plus gated advisory Discord, WhatsApp, and Slack run for stable/full, beta with `run_release_soak=true`, or explicit `qa`/`qa-live` groups. Stable and full profiles also run exhaustive live/E2E suites and Docker release-path chunks.
**Rerun:** `rerun_group=release-checks` or a narrower release-checks handle. | -| Package Telegram | **Job:** `Run package Telegram E2E`
**Child workflow:** `NPM Telegram Beta E2E`
**Proves:** a focused published-package Telegram E2E when `release_package_spec` or `npm_telegram_package_spec` is set. Full candidate validation uses the canonical Package Acceptance Telegram E2E instead.
**Rerun:** `rerun_group=npm-telegram` with `release_package_spec` or `npm_telegram_package_spec`. | -| Product performance | **Job:** `Run product performance evidence`
**Child workflow:** `OpenClaw Performance`
**Proves:** release-profile performance run (`profile=release`, `repeat=3`, `fail_on_regression=true`, `publish_reports=false`) against the target SHA. Kova output stays in workflow artifacts and the child must prove its report publisher was skipped. Required (blocking) only for `rerun_group=all` or `rerun_group=performance`; not required for narrower rerun groups.
**Rerun:** `rerun_group=performance`. | -| Umbrella verifier | **Job:** `Verify full validation`
**Child workflow:** none
**Proves:** re-checks recorded child run conclusions and appends slowest-job tables from child workflows.
**Rerun:** rerun only this job after rerunning a failed child to green. | +| Package Telegram | **Job:** `Run package Telegram E2E`
**Child workflow:** `NPM Telegram Beta E2E`
**Proves:** a focused published-package Telegram E2E when `release_package_spec` or `npm_telegram_package_spec` is set. Full candidate validation uses the canonical Package Acceptance Telegram E2E instead.
**Rerun:** `rerun_group=npm-telegram` with `release_package_spec` or `npm_telegram_package_spec`. | +| Product performance | **Job:** `Run product performance evidence`
**Child workflow:** `OpenClaw Performance`
**Proves:** release-profile performance run (`profile=release`, `repeat=3`, `fail_on_regression=true`, `publish_reports=false`) against the target SHA. Kova output stays in workflow artifacts and the child must prove its report publisher was skipped. Required (blocking) only for `rerun_group=all` or `rerun_group=performance`; not required for narrower rerun groups.
**Rerun:** `rerun_group=performance`. | +| Umbrella verifier | **Job:** `Verify full validation`
**Child workflow:** none
**Proves:** re-checks recorded child run conclusions and appends slowest-job tables from child workflows.
**Rerun:** rerun only this job after rerunning a failed child to green. | The umbrella always dispatches product performance in artifact-only mode. `OpenClaw Performance` permits report publication only for scheduled runs or a @@ -187,29 +187,29 @@ once and validates the umbrella's shared package artifact when available. A direct or focused dispatch prepares its own `release-package-under-test` artifact when package or Docker-facing stages need it. -| Stage | Details | -| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Release target | **Job:** `Resolve target ref`
**Backing workflow:** none
**Tests:** selected ref, optional expected Validation SHA, profile, rerun group, and focused live suite filter.
**Rerun:** `rerun_group=release-checks`. | -| Package artifact | **Job:** `Prepare release package artifact`
**Backing workflow:** none
**Tests:** validates the umbrella's immutable package tuple, or packs one candidate tarball for a direct/focused Release Checks dispatch, then exposes it to downstream package-facing checks.
**Rerun:** the affected package, cross-OS, or live/E2E group. | -| Install smoke | **Job:** `Run install smoke`
**Backing workflow:** `Install Smoke`
**Tests:** full install path with root Dockerfile smoke image reuse, QR package install, root and gateway Docker smokes, installer Docker tests, and Bun global install image-provider smoke.
**Rerun:** `rerun_group=install-smoke`. | -| Cross-OS | **Job:** `cross_os_release_checks`
**Backing workflow:** `OpenClaw Cross-OS Release Checks (Reusable)`
**Tests:** fresh and upgrade lanes on Linux, Windows, and macOS for the selected provider and mode, using the candidate tarball plus a baseline package.
**Rerun:** `rerun_group=cross-os`. | -| Windows-node prerelease E2E | **Job:** `windows_node_prerelease_e2e`
**Backing workflow:** `openclaw/openclaw-windows-node` release-candidate E2E
**Tests:** resolves the current Windows-node production prerelease, carries its exact name and SHA-256 through the reusable boundary, binds the OpenClaw candidate to its producer run, and proves setup, Tray connection, Windows-node capability registration, and MCP execution against the exact OpenClaw tarball.
**Runs:** every full release validation (`rerun_group=all`) or focused Windows validation (`rerun_group=windows-node`); success is required. | -| Windows-node stable E2E | **Job:** `windows_node_stable_e2e`
**Backing workflow:** `openclaw/openclaw-windows-node` release-candidate E2E
**Tests:** runs the same producer-run-bound E2E against the exact selected published stable Windows-node artifact. A failure passes only when its evidence contains the Gateway's explicit `PROTOCOL_MISMATCH` code; install, network, pairing, auth, and all other failures remain blocking.
**Runs:** every full release validation (`rerun_group=all`) or focused Windows validation (`rerun_group=windows-node`). | +| Stage | Details | +| --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Release target | **Job:** `Resolve target ref`
**Backing workflow:** none
**Tests:** selected ref, optional expected Validation SHA, profile, rerun group, and focused live suite filter.
**Rerun:** `rerun_group=release-checks`. | +| Package artifact | **Job:** `Prepare release package artifact`
**Backing workflow:** none
**Tests:** validates the umbrella's immutable package tuple, or packs one candidate tarball for a direct/focused Release Checks dispatch, then exposes it to downstream package-facing checks.
**Rerun:** the affected package, cross-OS, or live/E2E group. | +| Install smoke | **Job:** `Run install smoke`
**Backing workflow:** `Install Smoke`
**Tests:** full install path with root Dockerfile smoke image reuse, QR package install, root and gateway Docker smokes, installer Docker tests, and Bun global install image-provider smoke.
**Rerun:** `rerun_group=install-smoke`. | +| Cross-OS | **Job:** `cross_os_release_checks`
**Backing workflow:** `OpenClaw Cross-OS Release Checks (Reusable)`
**Tests:** fresh and upgrade lanes on Linux, Windows, and macOS for the selected provider and mode, using the candidate tarball plus a baseline package.
**Rerun:** `rerun_group=cross-os`. | +| Windows-node prerelease E2E | **Job:** `windows_node_prerelease_e2e`
**Backing workflow:** `openclaw/openclaw-windows-node` release-candidate E2E
**Tests:** resolves the current Windows-node production prerelease, carries its exact name and SHA-256 through the reusable boundary, binds the OpenClaw candidate to its producer run, and proves setup, Tray connection, Windows-node capability registration, and MCP execution against the exact OpenClaw tarball.
**Runs:** every full release validation (`rerun_group=all`) or focused Windows validation (`rerun_group=windows-node`); success is required. | +| Windows-node stable E2E | **Job:** `windows_node_stable_e2e`
**Backing workflow:** `openclaw/openclaw-windows-node` release-candidate E2E
**Tests:** runs the same producer-run-bound E2E against the exact selected published stable Windows-node artifact. A failure passes only when its evidence contains the Gateway's explicit `PROTOCOL_MISMATCH` code; install, network, pairing, auth, and all other failures remain blocking.
**Runs:** every full release validation (`rerun_group=all`) or focused Windows validation (`rerun_group=windows-node`). | | Windows-node main E2E | **Job:** `windows_node_main_e2e`
**Backing workflow:** `openclaw/openclaw-windows-node` release-candidate E2E
**Tests:** resolves `openclaw/openclaw-windows-node` `main` to a full commit SHA, checks out that exact source separately from the pinned reusable-workflow revision, builds exactly one x64 Tray executable, and runs the same candidate Gateway setup, connection, capability, and MCP E2E. Main source never substitutes for a release artifact.
**Runs:** every full release validation (`rerun_group=all`) or focused Windows validation (`rerun_group=windows-node`); success is required. | -| Repo and live E2E | **Job:** `Run repo/live E2E validation`
**Backing workflow:** `OpenClaw Live And E2E Checks (Reusable)`
**Tests:** repository E2E, live cache, OpenAI websocket streaming, native live provider and plugin shards, and Docker-backed live model/backend/gateway harnesses selected by `release_profile`.
**Runs:** `run_release_soak=true`, `release_profile=full`, or focused `rerun_group=live-e2e`.
**Rerun:** `rerun_group=live-e2e`, optionally with `live_suite_filter`. | -| Docker release path | **Job:** `Run Docker release-path validation`
**Backing workflow:** `OpenClaw Live And E2E Checks (Reusable)`
**Tests:** release-path Docker chunks against the shared package artifact.
**Runs:** `run_release_soak=true`, `release_profile=full`, or focused `rerun_group=live-e2e`.
**Rerun:** `rerun_group=live-e2e`. | -| Package Acceptance | **Job:** `Run package acceptance`
**Backing workflow:** `Package Acceptance`
**Tests:** offline plugin package fixtures, plugin update, the canonical mock-OpenAI Telegram package E2E, and published-upgrade survivor checks against the same tarball. Blocking release checks use the default latest published baseline; soak checks (`run_release_soak=true`) expand to the last 4 stable npm releases plus 3 pinned historical versions (`2026.4.23`, `2026.5.2`, `2026.4.15`), run against reported-issue upgrade fixtures.
**Rerun:** `rerun_group=package`. | -| Maturity scorecard | **Job:** `Render maturity scorecard release docs`
**Backing workflow:** `maturity-scorecard.yml`
**Tests:** renders the advisory maturity scorecard docs against the target ref. Only runs when `run_maturity_scorecard=true` is passed.
**Rerun:** `rerun_group=qa` with `run_maturity_scorecard=true`. | -| QA parity | **Job:** `Run QA Lab parity lane` and `Run QA Lab parity report`
**Backing workflow:** direct jobs
**Tests:** candidate and baseline agentic parity packs, then the parity report.
**Rerun:** `rerun_group=qa-parity` or `rerun_group=qa`. | -| QA runtime parity | **Job:** `Verify QA Lab runtime-pair lanes`
**Backing workflow:** direct job
**Tests:** the canonical core `openclaw`/`codex` lane (`pnpm openclaw qa suite --runtime-pair openclaw,codex --runtime-pair-lane core`) and, with `run_release_soak=true`, the soak lane. Advisory: individual lane jobs do not block the release-check verifier.
**Rerun:** `rerun_group=qa-parity` or `rerun_group=qa`. | -| QA runtime tool coverage | **Job:** `Enforce QA Lab runtime tool coverage`
**Backing workflow:** direct job
**Tests:** dynamic tool drift between `openclaw` and `codex` in the canonical core runtime-pair lane (`pnpm openclaw qa coverage --tools`), using that lane's output. Blocking: this job is not advisory-overridable.
**Rerun:** `rerun_group=qa-parity` or `rerun_group=qa`. | -| QA live Matrix | **Job:** `Run QA Live Matrix catalog`
**Backing workflow:** `QA-Lab - All Lanes` reusable workflow
**Tests:** catalog-derived YAML scenarios through the shared Matrix live adapter in the `qa-live-shared` environment, distributed across deterministic shards.
**Rerun:** `rerun_group=qa-live` or `rerun_group=qa`; use `live_suite_filter=qa-live-matrix` for a focused Matrix rerun. | -| QA live Buzz | **Job:** `Run QA Lab live Buzz lane`
**Backing workflow:** `QA-Lab - All Lanes` reusable workflow
**Tests:** signed canary and mention-gating round trips through the real Buzz plugin using dedicated Convex-leased identities and a hosted relay room.
**Rerun:** `rerun_group=qa-live` or `rerun_group=qa`; use `live_suite_filter=qa-live-buzz` for a focused Buzz rerun. | -| QA live Telegram | **Job:** `Run QA Lab live Telegram lane`
**Backing workflow:** trusted `OpenClaw Release Telegram QA` dispatch
**Tests:** live Telegram QA with Convex CI credential leases.
**Rerun:** `rerun_group=qa-live` or `rerun_group=qa`. | -| QA live Discord | **Job:** `Run QA Lab live Discord lane`
**Backing workflow:** direct advisory job
**Tests:** live Discord QA with Convex CI credential leases when `OPENCLAW_RELEASE_QA_DISCORD_LIVE_CI_ENABLED` is enabled.
**Rerun:** `rerun_group=qa-live` with `live_suite_filter=qa-live-discord`. | -| QA live WhatsApp | **Job:** `Run QA Lab live WhatsApp lane`
**Backing workflow:** direct advisory job
**Tests:** live WhatsApp QA with Convex CI credential leases when `OPENCLAW_RELEASE_QA_WHATSAPP_LIVE_CI_ENABLED` is enabled.
**Rerun:** `rerun_group=qa-live` with `live_suite_filter=qa-live-whatsapp`. | -| QA live Slack | **Job:** `Run QA Lab live Slack lane`
**Backing workflow:** direct advisory job
**Tests:** live Slack QA with Convex CI credential leases when `OPENCLAW_RELEASE_QA_SLACK_LIVE_CI_ENABLED` is enabled.
**Rerun:** `rerun_group=qa-live` with `live_suite_filter=qa-live-slack`. | -| Release verifier | **Job:** `Verify release checks`
**Backing workflow:** none
**Tests:** required release-check jobs for the selected rerun group.
**Rerun:** rerun after focused child jobs pass. | +| Repo and live E2E | **Job:** `Run repo/live E2E validation`
**Backing workflow:** `OpenClaw Live And E2E Checks (Reusable)`
**Tests:** repository E2E, live cache, OpenAI websocket streaming, native live provider and plugin shards, and Docker-backed live model/backend/gateway harnesses selected by `release_profile`.
**Runs:** `run_release_soak=true`, `release_profile=full`, or focused `rerun_group=live-e2e`.
**Rerun:** `rerun_group=live-e2e`, optionally with `live_suite_filter`. | +| Docker release path | **Job:** `Run Docker release-path validation`
**Backing workflow:** `OpenClaw Live And E2E Checks (Reusable)`
**Tests:** release-path Docker chunks against the shared package artifact.
**Runs:** `run_release_soak=true`, `release_profile=full`, or focused `rerun_group=live-e2e`.
**Rerun:** `rerun_group=live-e2e`. | +| Package Acceptance | **Job:** `Run package acceptance`
**Backing workflow:** `Package Acceptance`
**Tests:** offline plugin package fixtures, plugin update, the canonical mock-OpenAI Telegram package E2E, and published-upgrade survivor checks against the same tarball. Blocking release checks use the default latest published baseline; soak checks (`run_release_soak=true`) expand to the last 4 stable npm releases plus 3 pinned historical versions (`2026.4.23`, `2026.5.2`, `2026.4.15`), run against reported-issue upgrade fixtures.
**Rerun:** `rerun_group=package`. | +| Maturity scorecard | **Job:** `Render maturity scorecard release docs`
**Backing workflow:** `maturity-scorecard.yml`
**Tests:** renders the advisory maturity scorecard docs against the target ref. Only runs when `run_maturity_scorecard=true` is passed.
**Rerun:** `rerun_group=qa` with `run_maturity_scorecard=true`. | +| QA parity | **Job:** `Run QA Lab parity lane` and `Run QA Lab parity report`
**Backing workflow:** direct jobs
**Tests:** candidate and baseline agentic parity packs, then the parity report.
**Rerun:** `rerun_group=qa-parity` or `rerun_group=qa`. | +| QA runtime parity | **Job:** `Verify QA Lab runtime-pair lanes`
**Backing workflow:** direct job
**Tests:** the canonical core `openclaw`/`codex` lane (`pnpm openclaw qa suite --runtime-pair openclaw,codex --runtime-pair-lane core`) and, with `run_release_soak=true`, the soak lane. Advisory: individual lane jobs do not block the release-check verifier.
**Rerun:** `rerun_group=qa-parity` or `rerun_group=qa`. | +| QA runtime tool coverage | **Job:** `Enforce QA Lab runtime tool coverage`
**Backing workflow:** direct job
**Tests:** dynamic tool drift between `openclaw` and `codex` in the canonical core runtime-pair lane (`pnpm openclaw qa coverage --tools`), using that lane's output. Blocking: this job is not advisory-overridable.
**Rerun:** `rerun_group=qa-parity` or `rerun_group=qa`. | +| QA live Matrix | **Job:** `Run QA Live Matrix catalog`
**Backing workflow:** `QA-Lab - All Lanes` reusable workflow
**Tests:** catalog-derived YAML scenarios through the shared Matrix live adapter in the `qa-live-shared` environment, distributed across deterministic shards.
**Rerun:** `rerun_group=qa-live` or `rerun_group=qa`; use `live_suite_filter=qa-live-matrix` for a focused Matrix rerun. | +| QA live Buzz | **Job:** `Run QA Lab live Buzz lane`
**Backing workflow:** `QA-Lab - All Lanes` reusable workflow
**Tests:** signed canary and mention-gating round trips through the real Buzz plugin using dedicated Convex-leased identities and a hosted relay room.
**Rerun:** `rerun_group=qa-live` or `rerun_group=qa`; use `live_suite_filter=qa-live-buzz` for a focused Buzz rerun. | +| QA live Telegram | **Job:** `Run QA Lab live Telegram lane`
**Backing workflow:** trusted `OpenClaw Release Telegram QA` dispatch
**Tests:** live Telegram QA with Convex CI credential leases.
**Rerun:** `rerun_group=qa-live` or `rerun_group=qa`. | +| QA live Discord | **Job:** `Run QA Lab live Discord lane`
**Backing workflow:** direct advisory job
**Tests:** live Discord QA with Convex CI credential leases when `OPENCLAW_RELEASE_QA_DISCORD_LIVE_CI_ENABLED` is enabled.
**Rerun:** `rerun_group=qa-live` with `live_suite_filter=qa-live-discord`. | +| QA live WhatsApp | **Job:** `Run QA Lab live WhatsApp lane`
**Backing workflow:** direct advisory job
**Tests:** live WhatsApp QA with Convex CI credential leases when `OPENCLAW_RELEASE_QA_WHATSAPP_LIVE_CI_ENABLED` is enabled.
**Rerun:** `rerun_group=qa-live` with `live_suite_filter=qa-live-whatsapp`. | +| QA live Slack | **Job:** `Run QA Lab live Slack lane`
**Backing workflow:** direct advisory job
**Tests:** live Slack QA with Convex CI credential leases when `OPENCLAW_RELEASE_QA_SLACK_LIVE_CI_ENABLED` is enabled.
**Rerun:** `rerun_group=qa-live` with `live_suite_filter=qa-live-slack`. | +| Release verifier | **Job:** `Verify release checks`
**Backing workflow:** none
**Tests:** required release-check jobs for the selected rerun group.
**Rerun:** rerun after focused child jobs pass. | ## Docker release-path chunks