Commit Graph

36 Commits

Author SHA1 Message Date
Peter Steinberger c70aee247e refactor(scripts): migrate JavaScript tools to TypeScript (#121005)
* refactor(scripts): migrate JavaScript tools to TypeScript

* fix(ci): keep changed-scope preflight zero-install

* fix(ci): preserve zero-install script owners

* fix(ci): complete script migration follow-through

* fix(release): keep stable closeout zero-install

* fix(scripts): preserve standalone execution boundaries

* fix(scripts): repair standalone loader boundaries

* fix(scripts): normalize gateway observation ids

* fix(scripts): keep Docker packager standalone

* test(scripts): preserve rebase cleanup helpers

* test(sessions): use tracked temp directory
2026-08-09 07:21:35 -07:00
Peter Steinberger 0463d1bd83 test(qa): derive UX producer aggregate status (#120418) 2026-08-07 18:55:19 -07:00
Peter Steinberger cd50780d6a fix(qa): provision ffmpeg for Playwright scenarios (#119064) 2026-08-03 19:40:39 -07:00
Dallin Romney 652b0b96e2 qa: deduplicate producer and fallback evidence (#117806) 2026-08-02 17:20:32 +08:00
Peter Steinberger 80bbdeda1c fix(qa): verify requested scenarios and reject silent character runs (#116912)
* fix(qa): authenticate requested scenario test execution

* fix(qa): reject unanswered character evaluation runs

* refactor(qa): extract native vitest report validation

* fix(qa): require requested native test file to exist

---------

Co-authored-by: Peter Steinberger <steipete@macos.shared>
2026-07-31 09:18:05 -07:00
Peter Steinberger 366a686cba fix(qa): authenticate producer and generated-media evidence (#116828)
Co-authored-by: Peter Steinberger <steipete@macos.shared>
2026-07-31 05:24:06 -07:00
Peter Steinberger 7ea4129227 fix(qa): authenticate summaries and bound native scenario commands (#116748)
Co-authored-by: Peter Steinberger <steipete@macos.shared>
2026-07-31 03:12:48 -07:00
EricKwan e00ef16ee7 fix(xai): follow OAuth default model automatically (#115617)
* fix(xai): default OAuth login to Grok 4.5

* fix(xai): follow OAuth default model automatically

* fix(xai): send OAuth proxy request headers

* fix(xai): avoid provider API shadowing

* fix(qa): restore current main validation gates

---------

Co-authored-by: echeung <echeung@agentsonly.com>
Co-authored-by: fuller-stack-dev <263060202+fuller-stack-dev@users.noreply.github.com>
2026-07-29 23:05:29 +00:00
Peter Steinberger d16e33e08e fix: restore trustworthy end-to-end QA and cross-channel delivery (#115404)
* fix(qa): repair verified end-to-end and channel regressions

* fix(gateway): make interrupted restart recovery lifecycle-safe

* test(heartbeat): target the canonical recovery session store

* fix(gateway): prioritize durable restart recovery before heartbeat

* fix(qa): preserve safe restart recovery and channel expiry

* fix(qa): fail closed and fence restart recovery

* test(agents): isolate restart recovery timing

* test(agents): prove actual restart retry timing

* fix(qa): report incompatible profile scenarios

* fix(scripts): resolve symlinked docker scheduler entrypoints

* fix(qa): require fresh native test evidence

* fix(heartbeat): fence active restart recovery delivery

* fix(gateway): consume untargeted restart acknowledgements

* fix(qa): satisfy exhaustive hosted validation gates

* fix(agents): fence stopped restart recovery dispatch

* style(agents): format restart recovery lifecycle regression

* test(gateway): isolate context prewarm sidecar lifecycle

* test(qa): make scenario process timeout cleanup deterministic

* fix(qa): stamp synthetic gateway configs with current version

* fix(openai): preserve vision capabilities in stale model catalogs

* test(qa): align profile channel rejection with current main

* fix(openai): forward supported moderation for image edits

* fix: restore latest-main CI and image edit documentation

* fix(qa): retain relocated code-mode evidence validation

* fix(openai): expose GPT-5.4 vision in static catalog

* fix(pricing): honor explicit model cost overrides

* test(pricing): keep isolated provider regressions deterministic

* fix(openai): inherit transport for discovered static models

* fix(gateway): honor agent-owned static image capabilities

* test(gateway): preserve prepared-snapshot attachment races

* test(gateway): isolate subagent persistence failure injection

* test(gateway): exercise concurrent voice replay admission

* fix(gateway): restore stale model image capabilities

* fix(agents): publish configured model vision capabilities

* fix(agents): isolate detached media transcript ownership

* test(agents): preserve generic transcript lock regression

* fix(gateway): require proven static model route identity

* fix(qa): accept bounded full-size generated image attachments

* fix(qa): require fresh script producer evidence

* test(qa): prove native E2E scenario execution
2026-07-29 12:45:27 -04:00
Dallin Romney 92f6289cce fix(qa): run maturity Docker scenarios through the aggregate scheduler (#114447)
* fix(qa): batch Docker scorecard scenarios

* fix(qa): keep Docker batch types private
2026-07-28 22:41:35 +08:00
Peter Steinberger d8f099ebed fix(qa): honor fail-fast across unified suite runners (#113843) 2026-07-25 13:26:51 -07:00
Dallin Romney 132d91e427 refactor(qa): make taxonomy coverage ids unique (#110280) 2026-07-22 16:28:44 +09:00
Dallin Romney 8924e27bb6 refactor(qa): keep synthetic UX evidence out of scenario catalog (#109315)
* refactor(qa): keep UX evidence producer out of catalog

* test(qa): preserve UX producer runner integration
2026-07-16 13:51:18 -07:00
Peter Steinberger ff8a015981 test: tighten additional async polling (#108450) 2026-07-15 13:14:35 -07:00
Peter Steinberger f81f9d8570 chore: enforce max-lines suppression ratchet (#107315)
* ci: enforce max-lines suppression ratchet

* chore: prune stale max-lines suppression

* fix: close max-lines ratchet enforcement gaps

* fix: harden max-lines ratchet checks

* fix(ci): satisfy max-lines ratchet checks

* style: format max-lines declarations

* fix(ci): match oxlint suppression grammar

* test: isolate max-lines git fixtures

* chore: prune resolved max-lines debt

* test: skip newline path fixture on Windows

* fix: harden max-lines suppression ratchet

* chore: refresh max-lines baseline

* fix: close max-lines ratchet bypasses

* fix: derive ratchet base from PR merge tree

* fix: support older Git in staged ratchet

* fix: align max-lines declarations and baseline

* chore: refresh max-lines baseline for current main

* fix: exclude generated wizard locales from max-lines

* chore: prune resolved max-lines debt
2026-07-14 09:27:02 -07:00
Peter Steinberger fa77fe10d5 chore: migrate active GPT-5.5 references to GPT-5.6 (#104452)
* chore(models): migrate active GPT-5.5 references

* test(workboard): expect GPT-5.6 Sol default

* chore: keep release notes in PR body

* test(models): align picker fixtures with Sol default

* test: update PDF default model expectation

* test(qa): migrate thinking smoke to Luna

* test(gateway): align mock catalog with Sol default

* ci: retrigger exact-head PR checks

* test(gateway): document default catalog invariant
2026-07-11 06:30:57 -07:00
Dallin Romney 0b9c2e6f95 fix(qa): simplify smoke lanes and isolate fanout (#102262)
* fix(qa): simplify smoke lanes and isolate fanout

* ci: re-enable repaired QA smoke lanes

* ci: split Crabline smoke into three shards

* ci: eliminate repeated QA smoke builds

* fix(qa): focus control ui smoke scenario

* ci(qa): remove smoke worker launch delay

* ci(qa): bound automatic smoke coverage
2026-07-10 09:48:40 -07:00
Peter Steinberger 1f484a8dbd test: speed up and stabilize full suite 2026-07-05 08:00:23 -04:00
Shakker 65e12328aa feat: refactor the Control UI architecture
Refactor the Control UI around route-owned page lifecycle and state while preserving existing behavior and design.

Prepared head SHA: bd51b6fa76
Co-authored-by: Shakker <165377636+shakkernerd@users.noreply.github.com>
Reviewed-by: @shakkernerd
2026-07-04 23:19:38 +01:00
Vincent Koc bc802ceee1 test(qa): loosen mac script timeout harness 2026-07-04 03:06:25 +02:00
Dallin Romney 80a3c515fd test: add ClawHub release candidate QA proof (#97896)
* test: add ClawHub release candidate QA proof

* test: harden ClawHub release candidate evidence
2026-06-29 11:14:09 -07:00
Vincent Koc fd66b44f5e fix(qa): recover Playwright Chromium on Ubuntu 26 2026-06-24 13:24:43 +08:00
Dallin Romney 32dc664b4b fix(qa-lab): avoid duplicate child evidence files (#96030) 2026-06-23 01:18:24 -07:00
Vincent Koc 7cc21ef59d fix(qa): stabilize smoke-ci scenarios 2026-06-22 15:41:53 +08:00
Vincent Koc 735d70d9db test(extensions): pin Windows taskkill test roots 2026-06-21 11:37:59 +02:00
Vincent Koc 61d1fd1f72 fix(qa-lab): resolve Windows taskkill path 2026-06-21 11:26:30 +02:00
Vincent Koc c43822077a fix(qa-lab): force taskkill scenario trees on windows 2026-06-21 07:56:03 +02:00
Vincent Koc 1ede829fbf fix(qa-lab): leave vitest timeout cleanup to wrapper 2026-06-20 12:53:39 +02:00
Vincent Koc b93b07ee1b test(qa-lab): use temp harness in scenario runner tests 2026-06-20 12:53:39 +02:00
Vincent Koc 405e5072fd fix(qa-lab): bound test file scenario commands 2026-06-20 12:53:39 +02:00
Dallin Romney e12cf72b17 Standardize QA coverage IDs on dotted names (#94702)
* fix: standardize qa coverage ids

* test: avoid qa coverage id assertion spread
2026-06-18 17:25:26 -07:00
Colin Johnson d5a27b0b96 test: add QA Lab UX Matrix evidence scenario (#94306)
* test: add qa lab ux matrix script scenario

* fix(qa-lab): annotate UX Matrix producer catch callback as unknown for oxlint

---------

Co-authored-by: Dallin Romney <dallinromney@gmail.com>
2026-06-18 14:10:06 -07:00
Colin Johnson 591313e80a qa-lab: support script-backed evidence scenarios (#94276)
* qa: add script scenario execution kind

* fix(qa-lab): carry suite profile into script producer evidence and simplify artifact path resolution

* fix(qa-lab): keep out-of-repo producer artifacts absolute to avoid ../ traversal refs

---------

Co-authored-by: Dallin Romney <dallinromney@gmail.com>
2026-06-17 15:09:25 -07:00
Dallin Romney 1f8c4d3958 simplify QA evidence profile and mappings/coverage shape (#93153)
* test(qa): simplify evidence coverage shape

* test(qa): collapse evidence scorecard metadata

* test(qa): document evidence schema version
2026-06-14 22:26:58 -07:00
Dallin Romney e8db9c3bc0 test(qa): add qa run --profile and unified output summary/evidence (#91587)
* test(qa): add mapped qa run profiles

* test(qa): document mapped profile runner

* test(qa): validate run profiles from mapping

* test(qa): preserve root profile parsing

* test(qa): simplify taxonomy profile dispatch

* test(qa): align tool coverage CLI expectation

* test(qa): fix profile dispatch fixture type

* test(qa): share profile runner option types

* test(qa): split shared cli runner options

* test(qa): unify profile suite artifacts

* fix(qa): filter profile scenarios by provider lane

* test(qa): drop native scenario subreports

* fix(qa): keep native log refs repo-relative

* fix(cli): preserve qa run root profile parsing

* fix(qa): avoid qa profile flag collision

* fix(qa): reject profile flags without qa profile
2026-06-14 18:08:42 -07:00
Dallin Romney 561b293c7a Run Vitest and Playwright scenarios from qa suite (#92606)
* test(qa): run vitest and playwright scenarios from qa suite

* fix(qa): harden scenario suite dispatch

* refactor(qa): share scenario path utilities

* refactor(qa): share test file scenario runner

* refactor(qa): route test file scenarios through suite runtime

* refactor(qa): use explicit suite runtime result kind

* test(qa): write suite evidence artifact

* refactor(qa): clarify suite execution dispatch

* fix(qa): keep test-file scenarios out of flow-only runners

* refactor(qa): export mixed scenario suite runner
2026-06-13 01:06:10 -07:00