Commit Graph

45 Commits

Author SHA1 Message Date
Ayaan Zaidi d9d44981fb feat(e2e): record provider media content facts for Mantis proofs (#127830)
* feat(e2e): record provider media content facts

* docs(mantis): list staged-media recipe in recipe index

* fix(mantis): isolate mock provider evidence

* fix(mantis): state provider facts as tamper-evident, not provenance-trusted

The mock sidecar makes provider request facts tamper-evident: candidate
code cannot alter or remove a recorded request. It cannot make them
provenance-authenticated — every process in the SUT container shares the
internal network, UID, and gateway config, so nothing distinguishes the
gateway flow from any other candidate-originated request. The proof
prompt and busy-queue recipe now state exactly that property instead of
calling the facts trusted.

* fix(mantis): redact unparseable provider request bodies

Media redaction walks the parsed JSON body, so a request that fails
JSON.parse fell back to logging the raw text — leaking base64 payloads
the redactor exists to strip. Unparseable bodies now log a bounded
byte-count marker instead; regression posts a malformed body carrying a
data URL and asserts the payload never reaches the record.

* fix(mantis): expose newest provider records through a seq-stamped tail

The lane's requests surface kept the first 100 provider records, so a
session longer than the window hid exactly the newest requests a proof
asserts on. The mock server now stamps each record with a producer-owned
absolute seq ordinal, and the lane reads a bounded 128-record tail —
mirroring the sibling botApiRequests window. Regression writes 130
records and asserts the tail keeps seq 3..130; it fails pre-fix.
2026-08-22 16:30:30 +05:30
Ayaan Zaidi 190f2edd7f fix(qa): stop Mantis skipping visible Telegram changes (#127032)
Remove the premature visibility classifier and let one proof agent configure and exercise the disposable Telegram gateway. Align mock response timing with the 15-minute lane budget while preserving credential isolation through the alias-token proxy.
2026-08-21 12:33:38 +05:30
Ayaan Zaidi b7b7eec5ee feat(qa): run Mantis Telegram proof on local desktop (#126220)
Move Mantis Telegram Desktop proof from the remote AWS/Crabbox lane to a recorder-driven local Docker desktop. Keep proof scenarios agent-authored, cache trusted build outputs, and publish exact visible Telegram evidence without writing the QA bot token to artifacts.

Co-authored-by: Ayaan Zaidi <hi@obviy.us>
2026-08-20 11:12:51 +00:00
Peter Steinberger 448eb41653 refactor(tooling): unify benchmark and process helpers (#125491)
* refactor(tooling): unify benchmark and process helpers

* test(tooling): harden process fixture readiness

* fix(tooling): keep benchmark harness types private
2026-08-17 19:08:22 -07:00
Pavan Kumar Gondhi 6e026c2fe3 fix(gateway): reject unattributable loopback proxy traffic [AI] (#119950)
* fix(gateway): bind auth limits to ingress attribution

* fix(gateway): close remaining ingress auth gaps

* fix(gateway): carry attribution into new ingress paths

* fix(gateway): close ingress ownership gaps

* fix(gateway): complete proxy ingress hardening

* fix(gateway): stabilize managed Tailscale ingress

* fix(gateway): make Tailscale cleanup ownership-safe

Refuse reset-on-exit publication until Tailscale exposes an atomic owner-bound cleanup operation, and migrate legacy configs with Doctor.

* fix(gateway): finish ingress ownership repair

* fix(gateway): own managed Tailscale route lifetime

Run managed Serve and Funnel routes as foreground claims tied to the Gateway lifecycle. Retire named Service config through Doctor because Tailscale Services cannot run in foreground mode.

Co-authored-by: Pavan Kumar Gondhi <pavangondhi@gmail.com>

* fix(gateway): align Tailscale consumers and build guards

Remove the retired named-service config from Telegram Mini App URL resolution and register the lifecycle worker as an explicit production entry.

Co-authored-by: Pavan Kumar Gondhi <pavangondhi@gmail.com>

* fix(gateway): preserve retired Tailscale inputs

Keep Funnel enabled when removing an ignored named-Service setting and accept the legacy positive reset flag as a no-op now that managed routes always follow Gateway lifetime.

Co-authored-by: Pavan Kumar Gondhi <pavangondhi@gmail.com>

* fix(gateway): preserve Tailscale route diagnostics

Prefer the actionable foreground CLI failure captured during timeout cleanup, and cover the original delayed-failure ordering.

Co-authored-by: Pavan Kumar Gondhi <pavangondhi@gmail.com>

* fix(gateway): reconcile Tailscale ingress with main

Preserve current ingress ownership contracts after the rebase, retire the obsolete device-auth migration check, validate route-owner IPC, and move Tailscale auth coverage onto the managed listener.

Co-authored-by: Pavan Kumar Gondhi <pavangondhi@gmail.com>

* fix(gateway): finish ingress rebase coverage

Unify the rebased net imports and let module-reset WebSocket tests prepare attribution through the same fresh module instance as the handler.

Co-authored-by: Pavan Kumar Gondhi <pavangondhi@gmail.com>

* test(gateway): align run-loop server fixture

---------

Co-authored-by: joshavant <830519+joshavant@users.noreply.github.com>
2026-08-16 21:01:20 -07:00
Josh Avant 97a53a9b35 feat: audit admitted channel participant identity (#122863)
* feat: audit admitted channel participant identity

* fix: preserve Telegram identity through thread recovery

* fix: signal held gateway process groups

* fix: keep audit evidence passive in collect routing

* fix: validate copied channel participant evidence

* fix: bind channel participant evidence to host ingress

* fix: honor Telegram proof credential roles

* fix: restart held Telegram proof through gateway

* fix: repair channel identity CI regressions

* test(matrix): bind thread routing owner

* fix: preserve direct DM SDK compatibility

* fix: bind channel provenance at host runtime

* test(feishu): provide channel context builder

* fix: defer record-bound channel runtime resolution

* fix: keep channel admission evidence core-private

* fix(audit): bind channel admission to plugin lifecycle

* fix(audit): bind ingress provenance to final context

* refactor(audit): split admission scope keys

* test(queue): cover combined metadata carriers

* refactor(audit): keep lifecycle helpers private

* fix(queue): preserve combined turn authority

* test(channels): provide ingress context builders

* test(channels): align integrated CI fixtures

* test(clickclack): resolve model-loop ingress

* docs: preserve channel participant evidence invariant
2026-08-14 08:57:01 -05:00
Vincent Koc 282e6a47ae test(telegram): support fixed-delay Mantis proof (#122642)
Punchcard-Session: amber-harbor-timber-mb
2026-08-12 23:20:34 +08:00
wangmiao0668000666 c719cbbfe9 fix(e2e): preserve Unicode in Telegram proof tails (#109940)
Punchcard-Session: ember-brook-workshop-qc

Co-authored-by: wangmiao0668000666 <290215524+wangmiao0668000666@users.noreply.github.com>
2026-08-11 18:09:29 +08:00
Peter Steinberger 8616c0c374 refactor: finish shared test helper migrations (#120996)
* test: finish shared helper migrations

* test: fix helper migration CI

* style: fix test import ordering

* test(acpx): restore deferred void types

* test: fix helper migrations after rebase
2026-08-09 06:00:06 -07:00
Peter Steinberger 0487b6812c fix(qa): honor Crabbox SSH fallback ports (#120539) 2026-08-08 12:01:30 -07:00
Peter Steinberger 7ecdef6e2e fix: prefer Crabbox inspect SSH host (#120611) 2026-08-08 08:30:07 -07:00
Peter Steinberger 4232126bba chore: update dependencies across workspace (#115677)
* chore(deps): update dependencies

* fix(deps): restore CI compatibility
2026-07-29 05:16:42 -04:00
Peter Steinberger 1695854877 fix(qa): isolate Mantis candidate execution (#114465)
* fix(qa): allow restricted Mantis worktree traversal

* fix(qa): pin Mantis harness checkout revision

* fix(qa): isolate Mantis candidate execution

* fix(qa): harden Mantis teardown
2026-07-27 12:11:34 -04:00
Peter Steinberger 45bad50e26 fix(qa): harden Mantis Telegram proof harness (#114429)
* fix(qa): harden Mantis Telegram proof harness

* fix(qa): require explicit Mantis dispatch
2026-07-27 04:33:06 -04:00
Jason (Json) baa8b9b24f test(telegram): add MCP App Funnel proof fixture (#111238) 2026-07-19 00:14:36 -06:00
Peter Steinberger 51fa35c94d test: fix three pid/signal races behind CI flakes (batch 3) (#109877)
A. check-extension-package-tsc-boundary "force-kills aborted async node
step process groups" (run 29568640449): two mechanisms behind the
expected-false-to-be-true at the child-alive assertion.
- Pid-file existence polls can observe writeFileSync's open-truncate
  0-byte window, parse NaN, and process.kill(NaN, 0) throws, so
  isProcessAlive reads false (the #109140 class, same as tsdown-build's
  b1e6c6883b). Added waitForPidFile (poll until content parses to a
  positive pid) and applied it to all three pid-read sites in the file.
- The fail-fast step exited on a 150ms fuse after the sibling's pid file
  appeared, racing the test's aliveness assertion: a worker descheduled
  >150ms observes the abort chain's SIGKILL first. The step now exits
  only after the test writes abort.ack, which happens strictly after the
  assertion (event-driven; the step's 5s timeout bounds a wedged run).
  Forced proof: injecting a 300ms stall between pid-file wait and the
  assertion reproduced the exact CI failure, and passed post-fix with
  the same stall in place.

B. telegram-user-crabbox-proof "stops Crabbox recording when the desktop
probe fails" (waitFor timeout on recorder.term): the fake recorder
published its pid file before installing its SIGTERM handler. The probe
throws as soon as the pid file exists and recordProbeVideo SIGTERMs the
recorder in its finally, so a recorder preempted between publish and
process.on dies to the default disposition and never writes
recorder.term. Handler now arms before the pid publish. Forced proof: a
100ms spin between publish and process.on reproduced the exact CI
failure deterministically; moving the spin after the reordered handler
passes. Applied the same handler-before-readiness ordering to the three
sibling fakes in the file, and made the gateway-grandchild pid read
parse-validated inside its poll (a NaN pid skipped both the dead-check
and the finally SIGKILL cleanup, leaking the grandchild).

C. openclaw-live-updater "treats an owner file creation race as a normal
overlap" (run 29569279237): prod bug in acquireMaintenanceLock. Only
ENOENT got the bounded creation-window retry; a reader hitting the
racing writer's open-truncate window read an empty owner.json, JSON.parse
threw SyntaxError, and the script escalated to invalid_lock instead of
overlap. Any owner read/parse failure now re-reads within the same
bounded budget before declaring the lock invalid. New regression test
freezes the window (pre-created empty owner.json, delayed writer):
fails with the exact CI error pre-fix, passes post-fix.

Proof: focused runs green; 10x per-file stress at
OPENCLAW_VITEST_MAX_WORKERS=6 green for all three files (30/30).
2026-07-17 11:01:38 +01:00
Peter Steinberger fecd11fdc9 perf(test): unref process watchdogs (#108643) 2026-07-16 00:39:08 -07:00
Peter Steinberger 506bb14d22 test: tighten more async polling (#108385) 2026-07-15 10:49:43 -07:00
Peter Steinberger b163dbb97d ci: harden current and frozen release checks (#104956)
* test(qa): wait for complete process pid fixture

* style(qa): format pid readiness predicate

* test(sqlite): allow schema version pragma

* ci: install target playwright for frozen UI checks
2026-07-11 22:08:49 -07:00
Peter Steinberger fe261b0f59 chore(tooling): typecheck root test/** with a dedicated tsgo lane (#104475)
* chore(types): add declaration files for scripts/lib and scripts/e2e modules

* chore(types): add declaration files for top-level script modules (a-m)

* chore(types): add declaration files for top-level script modules (n-z)

* test: use a non-secret-shaped gateway token fixture

* test: type ci workflow guard helpers for the root test lane

* chore(tooling): typecheck root test/** with a dedicated tsgo lane

- test/tsconfig/tsconfig.test.root.json: root-test program (strict unused checks,
  fixtures excluded; two Docker E2E clients that import built dist/** stay out,
  same rationale as the scripts/e2e exclusion in tsconfig.scripts.json)
- tsgo:test:root wired into tsgo:test, check:test-types, scripts/check.mjs, and
  the ci.yml test-types shard, mirroring the tsgo:scripts lane (#104348)
- changed-lane routing: test/**/*.ts (excluding fixtures) and the lane tsconfig
  now trigger 'typecheck test root' in check:changed; previously test/ paths ran
  lint only, so harness type errors surfaced first in CI (#104287 envDir case)
- burn down all 1071 latent type errors in the program: precise param/local
  types across test/scripts, test/vitest, test/e2e, and transitive scripts/e2e
  program members; 205 sibling .d.mts declaration files for imported .mjs
  modules (committed separately); zero any, zero ts-expect-error
- resolve the pre-existing testing star-export ambiguity in
  scripts/e2e/parallels/common.ts with an explicit re-export

Closes #104388

* chore(types): correct declaration fidelity per structured review

- re-derive 51 .d.mts files from implementation data flow instead of
  initializers: fix a wrong never return (runTestProjectsDelegation returns
  the child), add encoding-sensitive exec/spawn overloads (plain-gh), restore
  the full release profile union, make parsed paths string | null, add missing
  parseArgs fields via help/non-help unions, add a missing sibling declaration
  (budget-number-args), drop 15 unused lint directives
- precise install-record/tuple typing removes the type-aware oxlint
  regressions the first declarations caused in scripts/e2e implementations
- route .mts declaration edits under test/ to the testRoot lane and reference
  the test-root project from tsconfig.projects.json so tsgo:all covers it
  (closes both review findings against the lane wiring)

* chore(scripts): keep telegram runner dist typing structural for the boundary guard

* chore(types): declare runtime pack and gateway readiness exports added on main

* test: pin the importTargetPlan form of the plugin-contract plan import

The guard expectation still referenced the raw await import( form that
7ae5996bb3 (#103975) replaced with the importTargetPlan fallback helper;
the assertion fails on current main.
2026-07-11 06:15:41 -07:00
Peter Steinberger 1f484a8dbd test: speed up and stabilize full suite 2026-07-05 08:00:23 -04:00
Vincent Koc 2cbb4e70cc fix(qa): reject duplicate telegram proof controls 2026-06-23 16:54:31 +02:00
Vincent Koc 720e295cff fix(qa): require Telegram proof report before publish 2026-06-23 15:09:04 +02:00
Vincent Koc 04575a97b6 fix(qa): avoid telegram proof artifact collisions 2026-06-23 11:57:34 +02:00
Vincent Koc c578608b78 fix(scripts): clamp telegram proof timeouts 2026-06-22 02:13:45 +02:00
Vincent Koc b95b725c83 fix(e2e): reject help flag Crabbox proof values 2026-06-21 23:13:12 +02:00
Vincent Koc 9940110b88 fix(telegram): resolve taskkill in crabbox proof 2026-06-21 10:10:27 +02:00
Vincent Koc 140a2fa520 fix(crabbox): force taskkill telegram proof trees on windows 2026-06-21 07:46:57 +02:00
Vincent Koc 38c8b0c196 fix(crabbox): kill telegram proof trees on windows 2026-06-21 06:47:25 +02:00
Vincent Koc 2f9f45f734 fix(telegram): include session probe artifacts 2026-06-20 18:51:20 +02:00
Vincent Koc 32cbaecd09 fix(telegram): stage full proof artifacts safely 2026-06-20 18:47:12 +02:00
Vincent Koc a1d278b174 fix(crabbox): preserve telegram proof kill grace 2026-06-20 12:25:03 +02:00
Vincent Koc 138ffa2992 fix(e2e): keep telegram proof command groups tracked 2026-06-20 11:14:05 +02:00
Vincent Koc 4e7a717868 fix(e2e): reap exited gateway process groups 2026-06-20 09:19:59 +02:00
Vincent Koc 00c2dc66b1 fix(e2e): avoid stale Crabbox recorder waits 2026-06-20 09:15:37 +02:00
Vincent Koc 3285a10c7f fix(scripts): shell quote Telegram Crabbox remotes 2026-06-20 07:31:21 +02:00
Vincent Koc d2e847e8cf fix(e2e): reject invalid telegram proof ports 2026-06-18 20:23:01 +02:00
Vincent Koc 677358f4a9 fix(e2e): bound telegram desktop proof logs 2026-06-07 10:42:28 +02:00
Peter Steinberger 58c663920d docs: document script tests 2026-06-04 20:49:50 -04:00
Peter Steinberger cb7a4239ef fix: stabilize full-suite regressions 2026-06-01 10:26:08 -04:00
Vincent Koc 75645aec08 fix(e2e): clean Telegram proof child processes 2026-06-01 13:20:03 +02:00
Vincent Koc ec6ad888a4 fix(e2e): bound telegram proof commands 2026-06-01 06:26:44 +02:00
Vincent Koc e449392c4f fix(e2e): route telegram proof through pnpm runner 2026-05-30 15:33:38 +02:00
Vincent Koc 6d43200248 fix(e2e): reject loose Telegram proof log limits 2026-05-30 14:23:40 +02:00
Vincent Koc a2f714cd44 fix(e2e): bound Telegram proof log polling 2026-05-27 16:59:35 +02:00