Commit Graph

18 Commits

Author SHA1 Message Date
Ayaan Zaidi d9d44981fb feat(e2e): record provider media content facts for Mantis proofs (#127830)
* feat(e2e): record provider media content facts

* docs(mantis): list staged-media recipe in recipe index

* fix(mantis): isolate mock provider evidence

* fix(mantis): state provider facts as tamper-evident, not provenance-trusted

The mock sidecar makes provider request facts tamper-evident: candidate
code cannot alter or remove a recorded request. It cannot make them
provenance-authenticated — every process in the SUT container shares the
internal network, UID, and gateway config, so nothing distinguishes the
gateway flow from any other candidate-originated request. The proof
prompt and busy-queue recipe now state exactly that property instead of
calling the facts trusted.

* fix(mantis): redact unparseable provider request bodies

Media redaction walks the parsed JSON body, so a request that fails
JSON.parse fell back to logging the raw text — leaking base64 payloads
the redactor exists to strip. Unparseable bodies now log a bounded
byte-count marker instead; regression posts a malformed body carrying a
data URL and asserts the payload never reaches the record.

* fix(mantis): expose newest provider records through a seq-stamped tail

The lane's requests surface kept the first 100 provider records, so a
session longer than the window hid exactly the newest requests a proof
asserts on. The mock server now stamps each record with a producer-owned
absolute seq ordinal, and the lane reads a bounded 128-record tail —
mirroring the sibling botApiRequests window. Regression writes 130
records and asserts the tail keeps seq 3..130; it fails pre-fix.
2026-08-22 16:30:30 +05:30
Ayaan Zaidi 10e0e690df feat(mantis): programmable proof boundaries with recipe library (#127782)
Makes the Mantis Telegram Desktop proof agent programmable at its trust boundaries: declarative Bot API fault rules (drop/status per method), per-request scripted mock-provider responses, observe-until predicates (post-cursor events/text, cumulative provider count), and bounded proxy-side recording of outbound Bot API requests as trusted lane facts. Adds a reusable recipe library under .github/codex/prompts/mantis-recipes/ and raises the proof agent's reasoning effort to high.

Security: the SUT container shadows proxy-control with an inaccessible tmpfs so candidate PR code sharing the mantis-sut uid cannot read or rewrite the proxy's recorded evidence; unmount is blocked by cap-drop/no-new-privileges. Proof doctrine now treats proxy-recorded Bot API facts as trusted comparison evidence and provider request logs as diagnostics.

Follow-up named in PR: move the mock OpenAI server out of the SUT container so provider request facts also become candidate-tamper-proof.
2026-08-22 11:06:06 +05:30
Ayaan Zaidi 190f2edd7f fix(qa): stop Mantis skipping visible Telegram changes (#127032)
Remove the premature visibility classifier and let one proof agent configure and exercise the disposable Telegram gateway. Align mock response timing with the 15-minute lane budget while preserving credential isolation through the alias-token proxy.
2026-08-21 12:33:38 +05:30
Ayaan Zaidi b7b7eec5ee feat(qa): run Mantis Telegram proof on local desktop (#126220)
Move Mantis Telegram Desktop proof from the remote AWS/Crabbox lane to a recorder-driven local Docker desktop. Keep proof scenarios agent-authored, cache trusted build outputs, and publish exact visible Telegram evidence without writing the QA bot token to artifacts.

Co-authored-by: Ayaan Zaidi <hi@obviy.us>
2026-08-20 11:12:51 +00:00
Peter Steinberger 980b07f4a2 refactor(agents): replace tool-failure warning heuristics with a two-rule policy (#126542)
* refactor(agents): reduce tool failure warnings to two rules

* test(agents): remove obsolete tool recovery receipt proof

* refactor(agents): drop unused meta param from buildToolMutationState

* test(agents): remove stale tool warning assertions

* test: export runtime source snapshot from closed runtime-snapshot mocks

Heal the main breakage introduced by 9441e3fe6e / #126531, which added a runtime source-snapshot read to provider model route resolution. Closed Vitest factories now return null for that source snapshot, preserving their pre-projection behavior.

* test(gateway): make compaction read-error faults order-immune

Generation-2 CI failure in run 32342180898, job 96343444772 showed that the mock factory initialized while shared gateway-server importers remained bound to the real transcript reader.

The dedicated isolated project fixes normal shards. Complete its ownership by adding it to the root project matrix and excluding the test from the non-isolated OPENCLAW_GATEWAY_PROJECT_SHARDS=0 fallback.
2026-08-20 01:11:05 -07:00
Ayaan Zaidi 8c6976e1fc fix(agents): report recovered tool retries (#125078)
Preserve attempt-local recovery state so successful mutation retries emit a redacted terminal receipt without stale failure warnings.

Co-authored-by: Ayaan Zaidi <hi@obviy.us>
2026-08-17 14:35:00 +05:30
Peter Steinberger 2fa9a29c77 test(plugins): add Agent Plugins gateway e2e with model-driven MCP tool call (#120303)
* test(plugins): add Agent Plugins gateway e2e with model-driven MCP tool call

* fix(test): wire Agent Plugins gateway E2E and assert activation
2026-08-07 11:47:07 -07:00
Peter Steinberger 45bad50e26 fix(qa): harden Mantis Telegram proof harness (#114429)
* fix(qa): harden Mantis Telegram proof harness

* fix(qa): require explicit Mantis dispatch
2026-07-27 04:33:06 -04:00
Jason (Json) baa8b9b24f test(telegram): add MCP App Funnel proof fixture (#111238) 2026-07-19 00:14:36 -06:00
Peter Steinberger fecd11fdc9 perf(test): unref process watchdogs (#108643) 2026-07-16 00:39:08 -07:00
Vincent Koc 796fc409f6 fix(e2e): echo dynamic mobile markers 2026-07-12 15:22:19 +02:00
Vincent Koc a2c2c5576e fix(qa): restore OpenAI web search smoke 2026-07-12 07:17:48 +02:00
Dallin Romney 7b2c51573f refactor: consolidate free-port test helpers (#99771) 2026-07-03 19:53:43 -07:00
Vincent Koc 317919ec52 fix(e2e): reject invalid mock fixture ports 2026-06-18 20:28:20 +02:00
Vincent Koc 5893758957 test(scripts): wait for mock stderr flush 2026-06-17 22:55:29 +02:00
Peter Steinberger 58c663920d docs: document script tests 2026-06-04 20:49:50 -04:00
Vincent Koc 61d16dd173 fix(e2e): fail mock openai request log errors 2026-06-04 06:37:24 +02:00
Vincent Koc c903b271cf fix(e2e): reject loose mock config limits 2026-05-29 02:38:29 +02:00