* fix(mantis): enforce verdict-expectation coherence and publish agent analysis files
Mantis run 32619081130 on #127989 published an overall `pass` while its own
manifest recorded that the candidate expectation was not observed: per-lane
`status` was mechanical capture success and the agent's judgment lived only
in `expected` prose, so nothing reconciled the two before publication.
- `mantis-evidence.json` schemaVersion 2: each comparison lane carries a
required boolean `expectationMet`. The desktop agent sets it in the same
manifest edit as `expected`; mechanical producers (Telegram live, web UI,
Slack, Discord) derive it from lane status.
- `scripts/mantis/publish-pr-evidence.mjs` is the single enforcement owner:
it requires the booleans, recomputes `pass`/`outcome`, downgrades a
contradictory pass claim to `fail`, and renders a visible "verdict
downgraded" note. The desktop workflow invokes it with `--validate-only`
before upload or comment.
- Agent top-level `*.json`/`*.md` analysis files (assertions, comparisons,
recipe suggestion) now survive the quarantine rebuild and upload, so cited
evidence actually exists in the artifact.
* fix(mantis): derive expectations from trusted lane facts
Capture the proven mechanics from proof run 32616859410 (PR #127950) as a
reusable recipe: dual 600s timeout config patch, Code Mode exec events file,
streamed chunkDelayMs hold, and the observe-loop pattern around the 60s cap.
Require agents to record recipe-suggestion.md after scenario-mechanics
iteration so future discoveries persist.
* feat(mantis): add exec and restart lane primitives
Give the proof agent a developer shell inside each SUT container and an
in-container gateway restart so it can design scenarios like a local
developer: patch openclaw.json and restart, stage plugins and fixtures,
run node/tsx against the read-only repo root, inspect SQLite state.
- container script: exec (docker exec as mantis-sut, bounded by timeout),
restart (request file + TERM), sut_command becomes a relaunch supervisor
- lane CLI: exec returns bounded stdout/stderr/exitCode and records a
redacted invocation; restart waits for a fresh [gateway] ready marker
- MAX_SENDS 12 -> 40 (shared-QA-bot flood safety, not a scenario bound)
- runtime root chown root:mantis-proof, mode 1770 so the agent can stage
files while root-owned attestation stays unreplaceable
* docs(mantis): let the proof agent design scenarios like a local developer
Lead with developer-shell parity, allow reading whatever code the scenario
needs (PR text still untrusted, PR code only inside SUT lanes), document
exec/restart shapes, and reserve block for hard impossibilities.
* fix(mantis): keep the SUT exec result type local
* fix(mantis): resume the agent when it ends without a manifest
Run 32615428295 (exec branch on #127950) hit Codex context compaction at
03:52:11 and the model answered with a confabulated "handoff" message instead
of continuing; codex exec exited 0 with no mantis-evidence.json and the
trusted-evidence step failed the run with no verdict.
The agent step now checks for the manifest after codex exits and, when it is
missing, resumes the same thread (`codex exec ... resume --last -`, verified
against codex-rs/exec/src/lib.rs at rust-v0.149.0: cwd-matched latest thread,
`-` reads the prompt from stdin) with a short correction prompt, bounded to
three resumes. The main prompt states that a handoff/summary is never an
acceptable final message.
The pdf tool auto-resolves its model from plugin manifest defaults
(openai gpt-5.5), which the Mantis SUT catalog does not define, so every
pdf round trip died with "Unknown model: openai/gpt-5.5" before
native-vs-fallback dispatch (run 32572580656). Pin
agents.defaults.pdfModel to the only harness model and account for the
pdf tool's own provider request in the staged-media recipe wait/pins.
* docs(mantis): script catalog-tool turns through Code Mode exec
Run 32570733988 blocked both lanes with 'Tool pdf not found': the proof
agent scripted a top-level pdf function_call, but the SUT agent runs Code
Mode where catalog tools execute from JavaScript inside exec. Document the
canonical route in the proof prompt, add a complete exec(pdf) scripted-event
example plus the document-extract configPatch to the staged-media recipe,
and pin the new guidance in the workflow test.
* docs(mantis): wait for the exec round trip before finish
ClawSweeper P2: send releases the mock response once the outbound message
is visible, and finish tears the lane down, so a slow exec/pdf round trip
could lose its function_call_output follow-up. Observe until the cumulative
provider-request count covers the follow-up and assert the recorded
function_call_output before finish; pin the contract in the workflow test.
* feat(e2e): record provider media content facts
* docs(mantis): list staged-media recipe in recipe index
* fix(mantis): isolate mock provider evidence
* fix(mantis): state provider facts as tamper-evident, not provenance-trusted
The mock sidecar makes provider request facts tamper-evident: candidate
code cannot alter or remove a recorded request. It cannot make them
provenance-authenticated — every process in the SUT container shares the
internal network, UID, and gateway config, so nothing distinguishes the
gateway flow from any other candidate-originated request. The proof
prompt and busy-queue recipe now state exactly that property instead of
calling the facts trusted.
* fix(mantis): redact unparseable provider request bodies
Media redaction walks the parsed JSON body, so a request that fails
JSON.parse fell back to logging the raw text — leaking base64 payloads
the redactor exists to strip. Unparseable bodies now log a bounded
byte-count marker instead; regression posts a malformed body carrying a
data URL and asserts the payload never reaches the record.
* fix(mantis): expose newest provider records through a seq-stamped tail
The lane's requests surface kept the first 100 provider records, so a
session longer than the window hid exactly the newest requests a proof
asserts on. The mock server now stamps each record with a producer-owned
absolute seq ordinal, and the lane reads a bounded 128-record tail —
mirroring the sibling botApiRequests window. Regression writes 130
records and asserts the tail keeps seq 3..130; it fails pre-fix.
* refactor(mantis): reuse authorized desktop captures
* fix(mantis): budget desktop authorization failures
* chore(mantis): bound desktop proof retries
* fix(e2e): drop unused recorder failure fact type export
* fix(ci): route Mantis desktop teardown through the recorder wrapper
Cleanup invoked the internal recorder executable as mantis-sut, which is
deliberately kept out of the docker group and cannot read the
recorder-owned session file; teardown therefore failed and blocked
safe_to_release. The cleanup step already runs as the recorder user, so
call the public wrapper whose exec shim cds into the session root.
* fix(e2e): make recorder failure fact lane-readable; document v2 lifecycle
The Mantis workflow runs the recorder as the desktop user while the lane
reads the authorization-failure fact as mantis-sut; 0600 made that read
fail EACCES and silently disabled the two-attempt retry budget. Write the
fact 0644 — the 0770 attempt directory bounds visibility.
Update the mantis doc's recorder section for the v2 session lifecycle:
required --session handle with healthy-session reuse, capture-only stop,
and teardown owning authorization termination and lease release.
Makes the Mantis Telegram Desktop proof agent programmable at its trust boundaries: declarative Bot API fault rules (drop/status per method), per-request scripted mock-provider responses, observe-until predicates (post-cursor events/text, cumulative provider count), and bounded proxy-side recording of outbound Bot API requests as trusted lane facts. Adds a reusable recipe library under .github/codex/prompts/mantis-recipes/ and raises the proof agent's reasoning effort to high.
Security: the SUT container shadows proxy-control with an inaccessible tmpfs so candidate PR code sharing the mantis-sut uid cannot read or rewrite the proxy's recorded evidence; unmount is blocked by cap-drop/no-new-privileges. Proof doctrine now treats proxy-recorded Bot API facts as trusted comparison evidence and provider request logs as diagnostics.
Follow-up named in PR: move the mock OpenAI server out of the SUT container so provider request facts also become candidate-tamper-proof.
Make long, free-form Telegram proof runs truthful and resilient. Keep the trusted mock harness current across historical SUTs, preserve intentional silence and blocked outcomes, remove fixed attempt/lifetime caps, and export cropped motion proof without the prior memory spike.
Co-authored-by: Ayaan Zaidi <hi@obviy.us>
Remove the premature visibility classifier and let one proof agent configure and exercise the disposable Telegram gateway. Align mock response timing with the 15-minute lane budget while preserving credential isolation through the alias-token proxy.
Preserve honest blocked proof outcomes and publish visible stop-reports without marking them passed. Serialize burst runs through the authoritative Telegram-user lease while reserving time for proof and cleanup.
Add trusted ClawSweeper-label and maintainer-comment dispatch for Mantis Telegram proof. Short-circuit non-visible PRs before desktop setup while preserving exact-head, fork, credential, and comment-ownership boundaries.
Move Mantis Telegram Desktop proof from the remote AWS/Crabbox lane to a recorder-driven local Docker desktop. Keep proof scenarios agent-authored, cache trusted build outputs, and publish exact visible Telegram evidence without writing the QA bot token to artifacts.
Co-authored-by: Ayaan Zaidi <hi@obviy.us>
* feat(ci): weekly Codex sweep for date-carrying to-dos
Mondays 06:23 UTC (plus default-branch-only manual dispatch with
dry_run): a deterministic prefilter collects candidate lines where
to-do markers co-occur with date tokens, plus every deprecated compat
record from the plugin registry; a pinned openai/codex-action step
judges each candidate in context (genuine dated commitment vs
historical date or fixture) and writes an OVERDUE / DUE-30-days /
FUTURE report; a separate privileged job validates the report (tracked
file:line locations, date shape, inert text) and upserts one
marker-tagged tracking issue via the Barnacle app token, commenting
only when items newly become due.
The app token is minted on a fresh runner and checkout — never beside
Codex or its child processes; only the validated report artifact
crosses jobs. No permission-* subsets on token minting (installations
reject explicit subsets; see pr-ci-sweeper).
* chore(ci): localize upsert helpers with no external consumers
Patrick Erichsen