ewilliams28/openclaw
1
0
Fork 0
mirror of https://github.com/openclaw/openclaw.git synced 2026-08-12 21:53:00 -06:00
Code Issues Packages Projects Releases Wiki Activity

docs(agents): require screenshot/video proof for UI-visible and gateway-behavior changes (#122920)

Browse Source 
Adds hard policy: UI-visible changes need before/after screenshots or a
short video as PR evidence; gateway-behavior changes provable in the
Control UI need an isolated dev-gateway live run with a recorded video.
Documents the verified video upload flow (user-attachments endpoint,
mp4/webm both served) and the bare-URL embed rule for GitHub's player.
This commit is contained in:
Peter Steinberger
2026-08-12 19:32:43 -07:00
committed by GitHub
parent cba8aff63f
commit e0cd23d81f
1 changed files with 3 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+3
View File
@@ -210,6 +210,8 @@ Skills own workflows; root owns hard policy and routing. Product direction and m
- Testbox mechanics: warm from the task checkout; ownership is checkout-path scoped; `--reclaim` only for intentional transfer, and it does not retarget the remote checkout — never cross repos. One lease, one active command; never sync/reclaim during a run; base/head changed means stop and rewarm — never override stale lease checks. Warmup must print a lease id; silent success is unusable — verify before reuse, else fall back to one-shot `run`. Wrapper reuse requires its local SSH key; missing after restart/handoff means warm fresh. Direct lease: `blacksmith testbox run`; Crabbox wrapper reuse needs a wrapper-created lease. Status/stop: `blacksmith testbox status|stop --id <tbx_id>` — id is not positional, no status `--json` flag. Delegated runs reject `--fresh-pr` and `--stop-after`; sync current checkout, workflow owns lifecycle. Compound commands: `bash -lc`, never `sh -lc`; job env uses Bash `declare`. Testbox owns Chromium; never pass Crabbox `--browser` to `provider=blacksmith-testbox`. - Testbox mechanics: warm from the task checkout; ownership is checkout-path scoped; `--reclaim` only for intentional transfer, and it does not retarget the remote checkout — never cross repos. One lease, one active command; never sync/reclaim during a run; base/head changed means stop and rewarm — never override stale lease checks. Warmup must print a lease id; silent success is unusable — verify before reuse, else fall back to one-shot `run`. Wrapper reuse requires its local SSH key; missing after restart/handoff means warm fresh. Direct lease: `blacksmith testbox run`; Crabbox wrapper reuse needs a wrapper-created lease. Status/stop: `blacksmith testbox status|stop --id <tbx_id>` — id is not positional, no status `--json` flag. Delegated runs reject `--fresh-pr` and `--stop-after`; sync current checkout, workflow owns lifecycle. Compound commands: `bash -lc`, never `sh -lc`; job env uses Bash `declare`. Testbox owns Chromium; never pass Crabbox `--browser` to `provider=blacksmith-testbox`.
- Crabbox mechanics: a Crabbox request means real scenario proof — install/update/call/repro the user path, not just copied tests run remotely. Final timing JSON = proof complete; if portal sync hangs after it, interrupt the wrapper only. Wrapper `stop` has no `--timing-json`; use `node scripts/crabbox-wrapper.mjs stop --provider <provider> --id <id>`. Sparse-sync temp checkout may claim a kept Testbox; repo-path reuse needs `--reclaim`. Dirty-sync generator proof: compare hashes before/after; `git diff` includes the synced patch. - Crabbox mechanics: a Crabbox request means real scenario proof — install/update/call/repro the user path, not just copied tests run remotely. Final timing JSON = proof complete; if portal sync hangs after it, interrupt the wrapper only. Wrapper `stop` has no `--timing-json`; use `node scripts/crabbox-wrapper.mjs stop --provider <provider> --id <id>`. Sparse-sync temp checkout may claim a kept Testbox; repo-path reuse needs `--reclaim`. Dirty-sync generator proof: compare hashes before/after; `git diff` includes the synced patch.
- Visual proof: use Crabbox, set up like a user, then screenshot-verify. No harness/bypass/shortcut unless explicitly asked. - Visual proof: use Crabbox, set up like a user, then screenshot-verify. No harness/bypass/shortcut unless explicitly asked.
- UI-visible change (Control UI, native app, or user-visible chat/session behavior): before/after screenshots or a short video are mandatory PR evidence, captured from a real running surface and sanitized. UI proof infeasible: state the exact blocker in the PR.
- Gateway-behavior change provable in the Control UI (session lifecycle, steering/queue, subagent flows, delivery states): prove on a live dev gateway — isolated `OPENCLAW_STATE_DIR`, own port, never the operator's gateway — and attach a video of the flow. Default recorder: Playwright `recordVideo` against the dashboard URL; keep the driving script's waits on asserted UI states, not sleeps.
- In Codex or linked worktrees, direct local `pnpm test*`, `pnpm check*`, and `pnpm crabbox:run` can trigger pnpm dependency reconciliation or install prompts. Prefer `node` wrappers locally and Crabbox/Testbox for pnpm-gated proof. - In Codex or linked worktrees, direct local `pnpm test*`, `pnpm check*`, and `pnpm crabbox:run` can trigger pnpm dependency reconciliation or install prompts. Prefer `node` wrappers locally and Crabbox/Testbox for pnpm-gated proof.
- Repo-native PR worktrees may omit `node_modules`; prove remotely, then use `git commit --no-verify`. - Repo-native PR worktrees may omit `node_modules`; prove remotely, then use `git commit --no-verify`.
- Release-branch formatting: Testbox or existing binary; never local `pnpm exec` reconciliation. Targeted local format/lint: existing `./node_modules/.bin/*`; never `pnpm exec` reconciliation. - Release-branch formatting: Testbox or existing binary; never local `pnpm exec` reconciliation. Targeted local format/lint: existing `./node_modules/.bin/*`; never `pnpm exec` reconciliation.
@@ -259,6 +261,7 @@ Skills own workflows; root owns hard policy and routing. Product direction and m
- GitHub issue/PR create: read `$agent-transcript`; ask about sanitized transcript logs when available. - GitHub issue/PR create: read `$agent-transcript`; ask about sanitized transcript logs when available.
- Contributor PRs: parsed context requires authored `What Problem This Solves` and `Evidence` sections. Do not require field-level proof forms; reviewers inspect code, tests, and CI for correctness. - Contributor PRs: parsed context requires authored `What Problem This Solves` and `Evidence` sections. Do not require field-level proof forms; reviewers inspect code, tests, and CI for correctness.
- PR/issue images/video: `curl -s "https://uploads.github.com/user-attachments/assets?name=<f>&content_type=<mime>&repository_id=<id>" -X POST -H "Authorization: Bearer $(gh auth token)" -H "Accept: application/json" --data-binary @<f>`; embed returned `.url` as markdown. Same CDN as drag-drop; inherits repo visibility; no browser/computer use. 422 = unsupported type; 404 = bad repo id/no push. Non-media artifacts or endpoint failure: Crabbox artifact publishing plus the manifest URL. Never push proof assets to any product repo branch; do not commit `.github/pr-assets`. - PR/issue images/video: `curl -s "https://uploads.github.com/user-attachments/assets?name=<f>&content_type=<mime>&repository_id=<id>" -X POST -H "Authorization: Bearer $(gh auth token)" -H "Accept: application/json" --data-binary @<f>`; embed returned `.url` as markdown. Same CDN as drag-drop; inherits repo visibility; no browser/computer use. 422 = unsupported type; 404 = bad repo id/no push. Non-media artifacts or endpoint failure: Crabbox artifact publishing plus the manifest URL. Never push proof assets to any product repo branch; do not commit `.github/pr-assets`.
- Video proof upload: same endpoint, `content_type` `video/mp4` or `video/webm` (both verified served). Embed as the returned URL on its own bare line — GitHub renders a player; `![]()` image syntax does not. Playwright records webm; transcode `ffmpeg -i in.webm -c:v libx264 -pix_fmt yuv420p out.mp4` before upload for broad playback.
- CI polling: exact SHA, relevant checks only, minimal fields. Skip routine noise (`Auto response`, `Labeler`, docs agents, performance/stale). Logs only after failure/completion or concrete need. Never `gh run watch`; its 3s polling exhausts API quota. Use sparse GraphQL rollups. Filter `gh run list` by workflow/branch/commit; broad JSON lists can exceed relay caps. Exact-SHA fallback dispatches require the full 40-character SHA. - CI polling: exact SHA, relevant checks only, minimal fields. Skip routine noise (`Auto response`, `Labeler`, docs agents, performance/stale). Logs only after failure/completion or concrete need. Never `gh run watch`; its 3s polling exhausts API quota. Use sparse GraphQL rollups. Filter `gh run list` by workflow/branch/commit; broad JSON lists can exceed relay caps. Exact-SHA fallback dispatches require the full 40-character SHA.
- CI waits: `node scripts/watch-pr-ci.mjs <pr> <head-sha>` — prechecks mergeable (CONFLICTING = pull_request CI cannot attach) and run attachment before polling; watchers emit every terminal state; no unbounded polls. - CI waits: `node scripts/watch-pr-ci.mjs <pr> <head-sha>` — prechecks mergeable (CONFLICTING = pull_request CI cannot attach) and run attachment before polling; watchers emit every terminal state; no unbounded polls.
- Trusted-workflow release-branch CI: pass `target_ref` + `release_candidate_ref`; never `release_gate` (requires workflow head == target). - Trusted-workflow release-branch CI: pass `target_ref` + `release_candidate_ref`; never `release_gate` (requires workflow head == target).