diff --git a/.github/labeler.yml b/.github/labeler.yml index 25328ebdde7a..14728d04b1f3 100644 --- a/.github/labeler.yml +++ b/.github/labeler.yml @@ -118,6 +118,7 @@ - any-glob-to-any-file: - "extensions/qa-lab/**" - "qa/scenarios/**" + - "docs/maturity/**" - "docs/concepts/qa-e2e-automation.md" - "docs/concepts/personal-agent-benchmark-pack.md" - "docs/channels/qa-channel.md" diff --git a/docs/AGENTS.md b/docs/AGENTS.md index fa5ceca3f908..4954077257b9 100644 --- a/docs/AGENTS.md +++ b/docs/AGENTS.md @@ -24,6 +24,14 @@ This directory owns docs authoring, Mintlify link rules, and docs i18n policy. - `scripts/docs-sync-publish.mjs` excludes and prunes `docs/internal/**` from the public `openclaw/docs` publish repo if a page is force-added later. - Internal docs may mention repo paths, private app names, 1Password item names, and runbooks, but never include secret values. +## Maturity Scorecard Editing + +`taxonomy.yaml` and `qa/maturity-scores.yaml` are the source inputs; generated maturity docs under `docs/maturity/` are projections and should not be hand-edited for score, LTS, taxonomy, QA profile, or evidence tables. +`scripts/qa/render-maturity-docs.ts` owns generation; use `pnpm maturity:render` to refresh committed docs and `pnpm maturity:check` to verify them. +`.github/workflows/maturity-scorecard.yml` renders artifact previews and can open generated-doc PRs; `.github/workflows/openclaw-release-checks.yml` dispatches it for release QA. +Keep deterministic `qa-evidence.json.scorecard` data in GitHub Actions artifacts unless a maintainer explicitly asks for a sanitized committed projection. +Human overrides must change source state in a PR and explain the reason plus public or redacted evidence. + ## Docs i18n - Foreign-language docs are not maintained in this repo. The generated publish output lives in the separate `openclaw/docs` repo (often cloned locally as `../openclaw-docs`). diff --git a/docs/concepts/qa-e2e-automation.md b/docs/concepts/qa-e2e-automation.md index 9bc5cb555757..68aede00fda5 100644 --- a/docs/concepts/qa-e2e-automation.md +++ b/docs/concepts/qa-e2e-automation.md @@ -966,6 +966,7 @@ output and whose artifact paths are resolved relative to that producer `qa run --qa-profile`, the same `qa-evidence.json` also includes the profile scorecard summary for the selected taxonomy categories. Treat it as a discovery aid, not a gate replacement; the selected scenario still needs the right provider mode, live transport, Multipass, Testbox, or release lane for the behavior under test. +For scorecard context, see [Maturity scorecard](/maturity/scorecard). For character and style checks, run the same scenario across multiple live model refs and write a judged Markdown report: @@ -1023,6 +1024,7 @@ When no `--judge-model` is passed, the judges default to ## Related docs - [Matrix QA](/concepts/qa-matrix) +- [Maturity scorecard](/maturity/scorecard) - [Personal agent benchmark pack](/concepts/personal-agent-benchmark-pack) - [QA Channel](/channels/qa-channel) - [Testing](/help/testing) diff --git a/docs/help/testing.md b/docs/help/testing.md index 32521725e082..d7edb2f9ad13 100644 --- a/docs/help/testing.md +++ b/docs/help/testing.md @@ -20,6 +20,7 @@ of Docker runners. This doc is a "how we test" guide: - [QA overview](/concepts/qa-e2e-automation) - architecture, command surface, scenario authoring. - [Matrix QA](/concepts/qa-matrix) - reference for `pnpm openclaw qa matrix`. +- [Maturity scorecard](/maturity/scorecard) - how release QA evidence supports stability and LTS decisions. - [QA channel](/channels/qa-channel) - the synthetic transport plugin used by repo-backed scenarios. This page covers running the regular test suites and Docker/Parallels runners. The QA-specific runners section below ([QA-specific runners](#qa-specific-runners)) lists the concrete `qa` invocations and points back at the references above.