Commit Graph

57 Commits

Author SHA1 Message Date
Ayaan Zaidi da0cb592dd perf(e2e): reuse one authorized Telegram Desktop session per Mantis run (#127835)
* refactor(mantis): reuse authorized desktop captures

* fix(mantis): budget desktop authorization failures

* chore(mantis): bound desktop proof retries

* fix(e2e): drop unused recorder failure fact type export

* fix(ci): route Mantis desktop teardown through the recorder wrapper

Cleanup invoked the internal recorder executable as mantis-sut, which is
deliberately kept out of the docker group and cannot read the
recorder-owned session file; teardown therefore failed and blocked
safe_to_release. The cleanup step already runs as the recorder user, so
call the public wrapper whose exec shim cds into the session root.

* fix(e2e): make recorder failure fact lane-readable; document v2 lifecycle

The Mantis workflow runs the recorder as the desktop user while the lane
reads the authorization-failure fact as mantis-sut; 0600 made that read
fail EACCES and silently disabled the two-attempt retry budget. Write the
fact 0644 — the 0770 attempt directory bounds visibility.

Update the mantis doc's recorder section for the v2 session lifecycle:
required --session handle with healthy-session reuse, capture-only stop,
and teardown owning authorization termination and lease release.
2026-08-22 15:58:12 +05:30
Ayaan Zaidi 10c774cf38 feat(mantis): keep QA leases alive and stage fixture plugins (#127804)
* feat(mantis): keep leases alive and stage fixture plugins

* refactor(mantis): deduplicate lease keepalive guard

* fix(mantis): retry keepalive on unexpected broker errors

* fix(mantis): fence active proof on terminal lease loss
2026-08-22 14:09:28 +05:30
Ayaan Zaidi 460f5b6c61 perf(mantis): parallelize proof builds and warm caches (#127798)
* perf(mantis): parallelize proof builds and warm caches

* fix(mantis): size build image for the copied pnpm store

* chore(ui): refresh startup JS gzip baseline after streamed-markdown perf work

Identical source measures 345034-345058 B across builds while the
committed baseline left only a 9 B margin under the 512 B ratchet
tolerance, so build-artifacts flips on gzip nondeterminism (green on
main run 32559609413, red on PR run 32559442295, red locally).
Regenerated with scripts/check-control-ui-performance.mts
--update-baseline; the 350 KiB hard ceiling still bounds creep.
2026-08-22 07:53:36 +00:00
Ayaan Zaidi 0933263f76 fix(mantis): publish proof comments on auto-triggered runs (#127787)
## What Problem This Solves

Auto-triggered Mantis proof runs (label/`clawsweeper_label` and other non-comment request sources) end with no PR comment at all. The durable evidence publisher runs with `--create-missing false` and only edits an existing marker comment, but the inline status comment carrying that marker was only created when `request_source == 'issue_comment'`. Label-triggered runs therefore published nothing and logged the misleading "Skipped stale Mantis QA evidence comment because its status is no longer active" — observed on PR #127735. This is the silent-failure class: a Mantis run completes and the PR shows no visible outcome.

## Why This Change Was Made

- `.github/workflows/mantis-telegram-desktop-proof.yml`: the status ack comment (👀 + active-job link + run-scoped marker) is now created for every request source that resolves to a PR (`pr_number != ''`), not only `issue_comment`. The 👀 *reaction* stays `issue_comment`-only (it lives in `mantis-resolve-request.yml`, untouched — there is no triggering comment to react to on label runs).
- The start-failure fallback comment and the existing-artifact republish path now use the same run-scoped marker `<!-- mantis-telegram-desktop-proof:${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT} -->` as the status comment and the main publisher, so every publisher edits the single run-owned comment (ack → progress → final proof; no comment spam). The republish path gets an explicit `--create-missing false` to match. The design invariant that makes `false` safe: the fallback status-comment step is not `continue-on-error`, so a run in which no marker comment could be created fails `resolve_request` and never reaches publish.
- `scripts/mantis/publish-pr-evidence.mjs`: the two skip cases now log honestly — "no existing comment found" vs "could not update existing comment" — instead of one misleading stale-status message.

## User Impact

Operators triggering Mantis via labels (ClawSweeper flows) now get the same single evolving PR comment as comment-triggered runs: an immediate 👀 ack with the running job link, edited in place into the final proof evidence. No more runs that finish invisibly.

## Evidence

- Focused suite: `node scripts/run-vitest.mjs test/scripts/mantis-telegram-desktop-proof-workflow.test.ts` — 28/28, including new assertions that the status/failure comment gates use `pr_number != ''` (and not `request_source`) and that both publishers pass the run-scoped marker with `--create-missing false`.
- `node scripts/check-changed.mjs -- <touched files>` green; `git diff --check` clean.
- Marker alignment verified across all five sites in the workflow (status comment, prior-attempt cleanup regex, fallback comment, failure report, both publisher invocations): all use `mantis-telegram-desktop-proof:${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}`.
- Live-run proof of the label-triggered path requires a merged workflow (GitHub runs the workflow from the default branch for these triggers), so the first post-merge label-triggered Mantis run is the live verification; stated here as the known evidence gap.

Production LOC delta: −4 (workflow/tooling); tests +13.
2026-08-22 11:10:11 +05:30
Ayaan Zaidi 10e0e690df feat(mantis): programmable proof boundaries with recipe library (#127782)
Makes the Mantis Telegram Desktop proof agent programmable at its trust boundaries: declarative Bot API fault rules (drop/status per method), per-request scripted mock-provider responses, observe-until predicates (post-cursor events/text, cumulative provider count), and bounded proxy-side recording of outbound Bot API requests as trusted lane facts. Adds a reusable recipe library under .github/codex/prompts/mantis-recipes/ and raises the proof agent's reasoning effort to high.

Security: the SUT container shadows proxy-control with an inaccessible tmpfs so candidate PR code sharing the mantis-sut uid cannot read or rewrite the proxy's recorded evidence; unmount is blocked by cap-drop/no-new-privileges. Proof doctrine now treats proxy-recorded Bot API facts as trusted comparison evidence and provider request logs as diagnostics.

Follow-up named in PR: move the mock OpenAI server out of the SUT container so provider request facts also become candidate-tamper-proof.
2026-08-22 11:06:06 +05:30
Peter Steinberger 89bb601b91 fix(qa): replace Linux-only /proc probes in the Telegram proof harness (#127718)
The Mantis lane and the Telegram user driver reached for /proc to enforce
media containment and to prove a pid was still alive. Off Linux those paths
do not exist, so the containment check compared a literal "/proc/self/fd/N"
string against the output root and refused every file, while both liveness
probes silently reported "gone": the lane stole a held harness lock and the
driver deleted the observer pid file without ever signalling the process.

Containment now uses each runtime's strongest portable construct. Python
descends the media path one component at a time from an open directory
descriptor (O_DIRECTORY|O_NOFOLLOW, dir_fd), which is race-free and also
covers the intermediate directories the old leaf-only O_NOFOLLOW did not.
Node has no openat(2), so publicRelativePath re-walks the resolved
components after the descriptor is open and refuses any that became a
symlink. Liveness moves to process.kill(pid, 0) and to ps state/args, which
keeps the pid-reuse and zombie handling the /proc reads provided.

The compound-command approval binding test asserted an operand count that
only held where the host interpreters live in a root-owned prefix; a
Homebrew python3 is writable, so it correctly binds as a third executable
operand. Assert the script operands by path instead.
2026-08-21 17:45:12 -07:00
Ayaan Zaidi e8d2b8cb18 fix(mantis): keep Telegram proof running and honest (#127108)
Make long, free-form Telegram proof runs truthful and resilient. Keep the trusted mock harness current across historical SUTs, preserve intentional silence and blocked outcomes, remove fixed attempt/lifetime caps, and export cropped motion proof without the prior memory spike.

Co-authored-by: Ayaan Zaidi <hi@obviy.us>
2026-08-21 17:53:54 +05:30
Vincent Koc 225aa5a178 fix(ci): isolate Actions cache writes (#127107) 2026-08-21 02:45:06 -07:00
Ayaan Zaidi 190f2edd7f fix(qa): stop Mantis skipping visible Telegram changes (#127032)
Remove the premature visibility classifier and let one proof agent configure and exercise the disposable Telegram gateway. Align mock response timing with the 15-minute lane budget while preserving credential isolation through the alias-token proxy.
2026-08-21 12:33:38 +05:30
Ayaan Zaidi 1aa211be4a fix(qa): make Mantis backfills reliable (#126980)
Preserve honest blocked proof outcomes and publish visible stop-reports without marking them passed. Serialize burst runs through the authoritative Telegram-user lease while reserving time for proof and cleanup.
2026-08-21 10:01:58 +05:30
Peter Steinberger ab2bbd42df test: trim UI and tooling test seams (#126937) 2026-08-20 19:04:18 -07:00
Ayaan Zaidi 4768ac53c5 feat(qa): acknowledge Mantis PR requests (#126702)
Give maintainers immediate visibility when Mantis is requested. Bare mentions now react, link the active run, and keep one run-owned status comment through proof, short-circuit, or failure.

Co-authored-by: Ayaan Zaidi <hi@obviy.us>
2026-08-20 21:19:40 +05:30
Ayaan Zaidi 8458777e6a feat(qa): trigger Mantis proof from pull requests (#126658)
Add trusted ClawSweeper-label and maintainer-comment dispatch for Mantis Telegram proof. Short-circuit non-visible PRs before desktop setup while preserving exact-head, fork, credential, and comment-ownership boundaries.
2026-08-20 20:05:51 +05:30
Ayaan Zaidi b7b7eec5ee feat(qa): run Mantis Telegram proof on local desktop (#126220)
Move Mantis Telegram Desktop proof from the remote AWS/Crabbox lane to a recorder-driven local Docker desktop. Keep proof scenarios agent-authored, cache trusted build outputs, and publish exact visible Telegram evidence without writing the QA bot token to artifacts.

Co-authored-by: Ayaan Zaidi <hi@obviy.us>
2026-08-20 11:12:51 +00:00
Ayaan Zaidi 97e3136b9e feat(qa): add a standalone Telegram Desktop recorder with a prebaked desktop image (#125186)
Records native Telegram Desktop from a digest-pinned prebaked image: a ready desktop in ~12s with no per-lease apt installs or downloads, which also removes the dpkg-lock failures that killed recent runs.

The recorder only records; callers drive the turn and supply the TDLib driver used for QR authorization. Nothing in this repository invokes it yet - routing the Mantis Telegram Desktop Proof workflow through it is a follow-up.
2026-08-19 11:23:48 +05:30
Peter Steinberger 3224005c82 fix(ci): bound Telegram Mantis proof cleanup (#115253)
* test(mantis): bound Telegram proof cleanup

* test(mantis): enforce cleanup deadline in wrapper

* test(mantis): supervise cleanup through bash
2026-07-28 11:44:28 -04:00
Peter Steinberger 1695854877 fix(qa): isolate Mantis candidate execution (#114465)
* fix(qa): allow restricted Mantis worktree traversal

* fix(qa): pin Mantis harness checkout revision

* fix(qa): isolate Mantis candidate execution

* fix(qa): harden Mantis teardown
2026-07-27 12:11:34 -04:00
Peter Steinberger 45bad50e26 fix(qa): harden Mantis Telegram proof harness (#114429)
* fix(qa): harden Mantis Telegram proof harness

* fix(qa): require explicit Mantis dispatch
2026-07-27 04:33:06 -04:00
Jason (Json) baa8b9b24f test(telegram): add MCP App Funnel proof fixture (#111238) 2026-07-19 00:14:36 -06:00
Peter Steinberger 6b6bf3a34b fix(ci): repair Mantis desktop scenario dispatch (#109505) 2026-07-16 18:50:57 -07:00
Peter Steinberger fe261b0f59 chore(tooling): typecheck root test/** with a dedicated tsgo lane (#104475)
* chore(types): add declaration files for scripts/lib and scripts/e2e modules

* chore(types): add declaration files for top-level script modules (a-m)

* chore(types): add declaration files for top-level script modules (n-z)

* test: use a non-secret-shaped gateway token fixture

* test: type ci workflow guard helpers for the root test lane

* chore(tooling): typecheck root test/** with a dedicated tsgo lane

- test/tsconfig/tsconfig.test.root.json: root-test program (strict unused checks,
  fixtures excluded; two Docker E2E clients that import built dist/** stay out,
  same rationale as the scripts/e2e exclusion in tsconfig.scripts.json)
- tsgo:test:root wired into tsgo:test, check:test-types, scripts/check.mjs, and
  the ci.yml test-types shard, mirroring the tsgo:scripts lane (#104348)
- changed-lane routing: test/**/*.ts (excluding fixtures) and the lane tsconfig
  now trigger 'typecheck test root' in check:changed; previously test/ paths ran
  lint only, so harness type errors surfaced first in CI (#104287 envDir case)
- burn down all 1071 latent type errors in the program: precise param/local
  types across test/scripts, test/vitest, test/e2e, and transitive scripts/e2e
  program members; 205 sibling .d.mts declaration files for imported .mjs
  modules (committed separately); zero any, zero ts-expect-error
- resolve the pre-existing testing star-export ambiguity in
  scripts/e2e/parallels/common.ts with an explicit re-export

Closes #104388

* chore(types): correct declaration fidelity per structured review

- re-derive 51 .d.mts files from implementation data flow instead of
  initializers: fix a wrong never return (runTestProjectsDelegation returns
  the child), add encoding-sensitive exec/spawn overloads (plain-gh), restore
  the full release profile union, make parsed paths string | null, add missing
  parseArgs fields via help/non-help unions, add a missing sibling declaration
  (budget-number-args), drop 15 unused lint directives
- precise install-record/tuple typing removes the type-aware oxlint
  regressions the first declarations caused in scripts/e2e implementations
- route .mts declaration edits under test/ to the testRoot lane and reference
  the test-root project from tsconfig.projects.json so tsgo:all covers it
  (closes both review findings against the lane wiring)

* chore(scripts): keep telegram runner dist typing structural for the boundary guard

* chore(types): declare runtime pack and gateway readiness exports added on main

* test: pin the importTargetPlan form of the plugin-contract plan import

The guard expectation still referenced the raw await import( form that
7ae5996bb3 (#103975) replaced with the importTargetPlan fallback helper;
the assertion fails on current main.
2026-07-11 06:15:41 -07:00
joshavant 75e3aa293a fix(qa): use valid Convex module filename 2026-07-07 15:04:53 -05:00
Vincent Koc 8c8857c3ef fix(qa): keep telegram credential tests sparse safe 2026-06-20 22:45:25 +02:00
Peter Steinberger 58c663920d docs: document script tests 2026-06-04 20:49:50 -04:00
Ayaan Zaidi 51d7f3c143 ci(mantis): route telegram proof runs to us-east-1 2026-05-28 10:10:32 +05:30
Vincent Koc 0cca7861c1 fix(e2e): kill timed setup process groups 2026-05-26 15:31:56 +02:00
Ayaan Zaidi 4fdf61753a ci(mantis): pass crabbox capacity regions 2026-05-26 09:02:11 +05:30
Ayaan Zaidi cf275676f3 fix(mantis): release telegram user leases on startup failure 2026-05-25 23:34:35 +05:30
Peter Steinberger a0702e195d build(pnpm): use packageManager as pnpm source
Recreated from #85108 because the original branch could not be updated by maintainers.

Preserves current-main pnpm install hardening while switching workflow pnpm setup to packageManager, and adds exact version-scoped release-age exclusions for already-locked packages that pnpm 11.2.2 audits during install.

Co-authored-by: Altay <altay@hey.com>
2026-05-22 19:17:43 +01:00
Ayaan Zaidi edcf862da5 fix(mantis): finish interrupted telegram proof sessions 2026-05-19 22:59:50 +05:30
Ayaan Zaidi f0a86450b1 fix(mantis): release interrupted telegram proof leases 2026-05-19 09:58:34 +05:30
Ayaan Zaidi 6da73ac90f fix(mantis): wait for telegram desktop bootstrap 2026-05-19 09:15:15 +05:30
Ayaan Zaidi 1f794d2816 fix(mantis): skip zombie telegram queue runs 2026-05-19 08:45:31 +05:30
Ayaan Zaidi d75e16a1b9 fix(mantis): ignore stale telegram queue blockers 2026-05-19 08:32:26 +05:30
Ayaan Zaidi 131577a4dc fix(mantis): preserve telegram account queue 2026-05-19 08:15:52 +05:30
Ayaan Zaidi 5fb9c0c937 fix(mantis): crop telegram proof chat pane 2026-05-18 18:30:36 +05:30
Ayaan Zaidi 125ebd0987 fix(mantis): load telegram credential validator 2026-05-18 18:01:03 +05:30
Ayaan Zaidi 2fa86c6a42 fix(mantis): point telegram proof skill at workflow command 2026-05-18 17:36:12 +05:30
Ayaan Zaidi cd15ce35a0 fix(qa): keep telegram user creds mantis-only 2026-05-18 10:04:58 +05:30
Ayaan Zaidi 1c778f7afb fix(telegram): repair desktop proof login 2026-05-18 09:49:21 +05:30
Tak Hoffman 0dc04fb926 ci(mantis): allow ClawSweeper telegram proof agent (#83243) 2026-05-17 14:26:15 -05:00
Tak Hoffman 0f1f9525f3 fix(ci): clear Mantis command reactions (#83194)
* fix(ci): clear mantis command reactions

* fix(ci): clear Mantis command reactions

---------

Co-authored-by: clawsweeper <274271284+clawsweeper[bot]@users.noreply.github.com>
2026-05-17 12:22:01 -05:00
Ayaan Zaidi 59efd95669 ci(mantis): add telegram proof label trigger 2026-05-17 21:16:00 +05:30
Ayaan Zaidi b2d04646c1 ci(mantis): run telegram proof agent faster 2026-05-15 16:54:58 +05:30
Ayaan Zaidi 16cfc83643 ci(mantis): fix telegram run wait before checkout 2026-05-12 10:39:27 +05:30
Ayaan Zaidi 9c16388177 ci(mantis): serialize telegram account proofs 2026-05-11 20:42:00 +05:30
Ayaan Zaidi 663206aac4 ci(mantis): derive telegram proof refs from pr 2026-05-11 17:50:23 +05:30
Ayaan Zaidi d4c7519989 ci(mantis): allow fork telegram proof 2026-05-11 17:33:29 +05:30
Ayaan Zaidi becb6e8bbb fix(mantis): route telegram proof through trusted harness 2026-05-11 15:02:36 +05:30
Altay 7084c7d43f test(pnpm): check workflow pnpm major 2026-05-11 10:13:49 +01:00