--- summary: "How to run tests locally (vitest) and when to use force/coverage modes" read_when: - Running or fixing tests title: "Tests" --- - Full testing kit (suites, live, Docker): [Testing](/help/testing) - Update and plugin package validation: [Testing updates and plugins](/help/testing-updates-plugins) ## Agent default Agent sessions run trusted development tests, changed gates, typecheck/lint, and builds locally by default, broadening only when the touched contract requires it. Never execute untrusted repository tooling locally. Use Crabbox when the environment is part of the proof: clean-machine, install/package, Docker, E2E, live, desktop, or cross-platform work, or when the operator explicitly requests remote proof. Do not use Crabbox merely as generic compute offload. The configured Testbox workflow hydrates credentials, so untrusted contributor or fork code must use secretless fork CI or sanitized direct AWS Crabbox instead. Do not pre-warm for anticipated work. Acquire the backend lazily when the first environment-sensitive command is ready, reuse the returned `tbx_...` id for later remote commands, sync the current checkout on every run, and stop it before handoff. After the first successful reuse, the wrapper records the lease's base, dependency, and Testbox workflow fingerprint under `.crabbox/testbox-leases/`. Source-only edits keep reusing the warmed box. A changed merge base, lockfile, package-manager input, wrapper, or Testbox workflow fails closed and requires a fresh lease. Every run still syncs the current checkout. `OPENCLAW_TESTBOX_ALLOW_STALE=1` is only for intentional diagnostics, not release proof. Local test commands below are the normal trusted development path. Keep proof proportional to the touched contract. For untrusted proof, lazily warm with `--provider aws`. Every run must set `CRABBOX_ENV_ALLOW=CI`, pass `--provider aws --no-hydrate`, and use a fresh temporary remote `HOME` before installing dependencies or running tests. Use a newly warmed lease dedicated to that untrusted source; never reuse a trusted or previously hydrated lease. Launch an installed trusted Crabbox binary from a clean trusted `main` checkout and fetch only the remote PR with `--fresh-pr`; never execute the untrusted checkout's wrapper or config locally. Unset `CRABBOX_AWS_INSTANCE_PROFILE` and fail closed unless resolved `aws.instanceProfile` is empty. Before any install/test, use trusted absolute-path tools to require an IMDSv2 token, prove the IAM credentials endpoint returns 404, and verify remote `git rev-parse HEAD` equals the full reviewed PR head SHA. Bind the lease to that SHA and stop/rewarm when the head changes. Upload trusted `scripts/crabbox-untrusted-bootstrap.sh` from clean `main` alongside `--fresh-pr`; it installs pinned Node/pnpm, verifies the SHA and package-manager pin, isolates `HOME`, installs dependencies, then executes the requested test. If the broker cannot prove no role or no remote PR exists, use secretless fork CI. Do not use `hydrate-github`, `--no-sync`, or a credential-hydrated Testbox workflow. Unset all `CRABBOX_TAILSCALE*` overrides, force `--network public --tailscale=false`, clear exit-node/LAN flags, and require `crabbox inspect` to report public networking with no Tailscale state before uploading any script. ## Routine local order 1. `pnpm test:changed` for changed-scope Vitest proof. 2. `pnpm test ` for one file, directory, or explicit target. 3. `pnpm test` only when you intentionally need the full local Vitest suite. Codex and other linked/sparse worktrees can run local tests and checks. When the dependency install is ready, use the normal commands above. If pnpm would reconcile a shared install, use the direct Node harnesses to bypass that package-manager preflight: - Bounded focused proof with ready dependencies: `node scripts/run-vitest.mjs `. - Changed typecheck/lint/guard proof: `node scripts/check-changed.mjs`. For remote-environment proof, invoke `node scripts/crabbox-wrapper.mjs` directly. Avoid local `pnpm crabbox:run` in linked worktrees because pnpm may reconcile dependencies before the remote wrapper starts. ## Core commands Test wrapper runs end with a short `[test] passed|failed|skipped ... in ...` summary; Vitest's own duration line stays the per-shard detail. | Command | What it does | | ------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `pnpm test` | Explicit file/directory targets route through scoped Vitest lanes. Untargeted runs are full-suite proof: fixed shard groups expand to leaf configs for local parallel execution, with the expected shard fanout printed before starting. The extension group always expands to per-extension shard configs instead of one giant root-project process. | | `pnpm test:changed` | Cheap smart changed-test run: precise targets from direct test edits, sibling `*.test.ts` files, explicit source mappings, and the local import graph. Broad/config/package changes are skipped unless they map to precise tests. | | `OPENCLAW_TEST_CHANGED_BROAD=1 pnpm test:changed` | Explicit broad changed-test run; use when a test harness/config/package edit should fall back to Vitest's broader changed-test behavior. | | `pnpm test:force` | Frees the configured OpenClaw gateway port (default `18789`), then runs the full suite with an isolated gateway port so server tests do not collide with a running instance. | | `pnpm test:coverage` | Emits an informational V8 coverage report for the default unit lane (`vitest.unit.config.ts`); no coverage thresholds are enforced. | | `pnpm test:coverage:changed` | Unit coverage only for files changed since `origin/main`. | | `pnpm changed:lanes` | Shows the architectural lanes triggered by the diff against `origin/main`. | | `pnpm check:changed` | Classifies and runs the local changed formatting/typecheck/lint/guard plan. Does not run Vitest; use `pnpm test:changed` or `pnpm test ` for test proof. | ## Shared test state and process helpers - `src/test-utils/openclaw-test-state.ts`: use from Vitest when a test needs an isolated `HOME`, `OPENCLAW_STATE_DIR`, `OPENCLAW_CONFIG_PATH`, config fixture, workspace, agent dir, or auth-profile store. - `pnpm test:env-mutations:report`: non-blocking report of tests/harnesses that mutate `HOME`, `OPENCLAW_STATE_DIR`, `OPENCLAW_CONFIG_PATH`, `OPENCLAW_WORKSPACE_DIR`, or related env keys directly. Use it to find migration candidates for the shared test-state helper. - `test/helpers/openclaw-test-instance.ts`: process-level E2E tests needing a running Gateway, CLI env, log capture, and cleanup in one place. - Docker/Bash E2E lanes that source `scripts/lib/docker-e2e-image.sh` can pass `docker_e2e_test_state_shell_b64