licheer-zte a32e81c8e8 fix(model-fallback): treat empty non-GPT completions as failed candidates (#120132) (#120148) ...

* fix(model-fallback): treat empty non-GPT completions as failed candidates (#120132)

Empty and whitespace-only completions from non-GPT models were counted as
candidate_succeeded, silently dropping the turn on visible channels. Apply
the empty/reasoning-only classification to every model; deliberate silent
replies and committed outbound deliveries remain successful.

* fix(model-fallback): classify mixed reasoning-plus-blank completions as failed (#120148)

A completion like [{ isReasoning: true, text: "thinking" }, { text: " " }]
carries no user-visible reply: reasoning text is invisible to the shared
visibility test (includeReasoningPayloads: false), so counting it as visible
made the run look successful and silently ended visible-channel turns.

Filter reasoning payloads out of the empty/whitespace predicate so mixed
reasoning-plus-blank results classify as empty_result (fallback-worthy),
while mixed reasoning-plus-visible-text results stay successful.

Regression tests: mixed reasoning+blank -> empty_result; mixed
reasoning+visible -> success.

* fix(model-fallback): require deliverable assistant results

Use one owner-boundary deliverability predicate for fallback classification, preserve intentional terminal outcomes, and add a mock-channel Gateway scenario for mixed reasoning-plus-blank recovery.\n\nCo-authored-by: 李琪0668001400 <li.qi16@xydigit.com>

* chore: preserve contributor credit

Co-authored-by: 李琪0668001400 <li.qi16@xydigit.com>

* test(qa): cover default model fallback scenario

Make the mixed reasoning-plus-blank fixture recover through both the catalog default alternate and the explicit proof model.

Co-authored-by: 李琪0668001400 <li.qi16@xydigit.com>

---------

Co-authored-by: licheer-zte <licheer-zte@users.noreply.github.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>

2026-08-08 07:48:17 -07:00

..

live-transports

fix(qa): settle Unix commands after PGID cleanup (#120010)

2026-08-08 15:16:44 +08:00

mantis

refactor(qa): remove live-channel scenario registries and wrappers (#115752)

2026-07-30 13:56:34 +08:00

providers

fix(model-fallback): treat empty non-GPT completions as failed candidates (#120132) (#120148)

2026-08-08 07:48:17 -07:00

agentic-parity-cache-usage.ts

improve: reduce GPT-5.6 coding harness overhead (#114574)

2026-07-27 19:24:52 -04:00

agentic-parity-report-test-helpers.ts

improve: reduce GPT-5.6 coding harness overhead (#114574)

2026-07-27 19:24:52 -04:00

agentic-parity-report.cache.test.ts

fix(codex): stabilize prompts and expose real cache regressions (#115238)

2026-07-28 13:26:04 -04:00

agentic-parity-report.test.ts

refactor(test): table-drive exact-duplicate test blocks, batch 2 (#120071)

2026-08-06 17:00:28 -07:00

agentic-parity-report.timing.test.ts

improve: reduce GPT-5.6 coding harness overhead (#114574)

2026-07-27 19:24:52 -04:00

agentic-parity-report.ts

fix(codex): stabilize prompts and expose real cache regressions (#115238)

2026-07-28 13:26:04 -04:00

agentic-parity-runtime-markdown.ts

fix(codex): stabilize prompts and expose real cache regressions (#115238)

2026-07-28 13:26:04 -04:00

agentic-parity-runtime-report-contract.ts

fix(codex): stabilize prompts and expose real cache regressions (#115238)

2026-07-28 13:26:04 -04:00

agentic-parity.ts

refactor(qa-lab): trim dead export surface (#107016)

2026-07-13 19:32:13 -07:00

artifact-assertion.ts

…

artifact-run-id.ts

…

auth-profile.fixture.ts

refactor(auth): finish SQLite-only auth profile cutover (#114033)

2026-07-26 07:27:54 -04:00

browser-runtime.test.ts

…

browser-runtime.ts

…

bundled-plugin-staging.ts

fix(qa): clean partial gateway startup and staged plugin roots (#119494)

2026-08-04 22:48:25 -07:00

bus-queries.ts

fix(qa): preserve scoped native message and command identity (#119499)

2026-08-05 05:34:43 -07:00

bus-server.test.ts

fix: restore trustworthy end-to-end QA and cross-channel delivery (#115404)

2026-07-29 12:45:27 -04:00

bus-server.ts

fix: restore trustworthy end-to-end QA and cross-channel delivery (#115404)

2026-07-29 12:45:27 -04:00

bus-state.test.ts

fix(qa): reject deleted replies and failed streaming outcomes (#119660)

2026-08-05 09:30:42 -07:00

bus-state.ts

fix(qa): preserve scoped native message and command identity (#119499)

2026-08-05 05:34:43 -07:00

bus-waiters.ts

fix(qa): reject deleted replies and failed streaming outcomes (#119660)

2026-08-05 09:30:42 -07:00

channel-driver-lifecycle.test.ts

fix(qa): serialize racing channel driver lifecycle transitions (#113855)

2026-07-25 14:01:42 -07:00

channel-driver-lifecycle.ts

fix(qa): serialize racing channel driver lifecycle transitions (#113855)

2026-07-25 14:01:42 -07:00

character-eval.test.ts

refactor(test): table-drive exact-duplicate test blocks, batch 2 (#120071)

2026-08-06 17:00:28 -07:00

character-eval.ts

fix(qa): reject incomplete character judge rankings (#118423)

2026-08-02 21:25:19 -07:00

child-output.test.ts

fix(qa-lab): keep child output UTF-8 safe when capped (#108156)

2026-07-16 08:35:42 -07:00

child-output.ts

fix(qa-lab): keep child output UTF-8 safe when capped (#108156)

2026-07-16 08:35:42 -07:00

ci-smoke-plan.test.ts

fix(ui): make Control UI ownership and permissions reliable (#115364)

2026-07-28 16:42:05 -04:00

ci-smoke-plan.ts

refactor(qa): make scenario selection taxonomy-owned (#114055)

2026-07-28 22:23:41 +08:00

cli-options.ts

…

cli-paths.ts

…

cli.runtime.test.ts

fix(qa): attest realized profile executions (#119761)

2026-08-06 08:33:02 +08:00

cli.runtime.ts

refactor(qa): make module flows implementation-aware (#118069)

2026-08-06 19:51:28 +08:00

cli.test.ts

refactor(qa): make scenario selection taxonomy-owned (#114055)

2026-07-28 22:23:41 +08:00

cli.ts

refactor(qa): make scenario selection taxonomy-owned (#114055)

2026-07-28 22:23:41 +08:00

code-mode-model-matrix.test.ts

refactor: deduplicate extension normalization primitives (#115650)

2026-07-29 04:10:06 -04:00

codex-app-server-args.test.ts

fix(codex): isolate QA sandbox from production policy (#114830)

2026-07-27 21:05:41 -04:00

codex-app-server-args.ts

fix(codex): isolate QA sandbox from production policy (#114830)

2026-07-27 21:05:41 -04:00

codex-plugin-lifecycle.test.ts

fix(qa): replace false Codex pin compatibility evidence (#118012)

2026-08-07 08:45:26 +08:00

codex-plugin.fixture.ts

fix(qa): replace false Codex pin compatibility evidence (#118012)

2026-08-07 08:45:26 +08:00

confidence-report.test.ts

fix(security): harden CodeQL-reported boundaries (#115055)

2026-07-28 16:47:12 +08:00

confidence-report.ts

fix(security): harden CodeQL-reported boundaries (#115055)

2026-07-28 16:47:12 +08:00

coverage-id.ts

refactor(qa): make taxonomy coverage ids unique (#110280)

2026-07-22 16:28:44 +09:00

coverage-report.test.ts

fix(ui): make Control UI ownership and permissions reliable (#115364)

2026-07-28 16:42:05 -04:00

coverage-report.ts

refactor(qa): make scenario selection taxonomy-owned (#114055)

2026-07-28 22:23:41 +08:00

crabline-artifacts.ts

fix(qa-lab): report authoritative Crabline readiness artifact (#106039)

2026-07-13 01:55:12 -07:00

crabline-provider-targets.ts

refactor(qa): share Matrix flows across drivers (#101253)

2026-07-14 01:48:30 -07:00

crabline-transport-message-id.test.ts

fix(qa): preserve scoped native message and command identity (#119499)

2026-08-05 05:34:43 -07:00

crabline-transport-native-command.test.ts

fix(qa): preserve scoped native message and command identity (#119499)

2026-08-05 05:34:43 -07:00

crabline-transport-readiness.test.ts

fix(qa): require ready transport accounts (#119431)

2026-08-04 20:23:31 -07:00

crabline-transport.test.ts

improve(qa): align Telegram coverage with startup behavior (#113527)

2026-07-29 12:59:42 +08:00

crabline-transport.ts

refactor(qa): consolidate scenario runtime helpers (#119623)

2026-08-05 08:01:58 -07:00

cron-run-wait.test.ts

fix(cron): enforce caps across CLI backends

2026-07-24 01:24:03 -05:00

cron-run-wait.ts

fix(cron): enforce caps across CLI backends

2026-07-24 01:24:03 -05:00

discovery-eval.test.ts

…

discovery-eval.ts

…

docker-harness.test.ts

fix(qa-lab): unify runner control plane (#112838)

2026-07-27 14:05:14 +08:00

docker-harness.ts

fix(qa-lab): unify runner control plane (#112838)

2026-07-27 14:05:14 +08:00

docker-runtime.ts

…

docker-up.runtime.test.ts

fix(qa): stop Docker startup hangs on stalled health checks (#108992)

2026-07-16 05:51:38 -07:00

docker-up.runtime.ts

fix(qa): stop Docker startup hangs on stalled health checks (#108992)

2026-07-16 05:51:38 -07:00

errors.ts

refactor(qa): consolidate scenario runtime helpers (#119623)

2026-08-05 08:01:58 -07:00

evidence-environment.test.ts

fix(qa-lab): bound evidence checkout ref git probe with timeout (#111448)

2026-07-19 15:48:53 -07:00

evidence-environment.ts

fix(qa-lab): bound evidence checkout ref git probe with timeout (#111448)

2026-07-19 15:48:53 -07:00

evidence-gallery.test.ts

ci(release): restore full validation after config migrations (#112740)

2026-07-23 08:46:55 -04:00

evidence-gallery.ts

fix(qa-lab): preserve Unicode in capped evidence previews (#108545)

2026-07-16 03:50:30 -07:00

evidence-summary.test.ts

fix(qa): attest realized profile executions (#119761)

2026-08-06 08:33:02 +08:00

evidence-summary.ts

fix(qa): attest realized profile executions (#119761)

2026-08-06 08:33:02 +08:00

extract-tool-payload.ts

…

fixture-utils.ts

feat(tools): support batched tool search queries (#118623)

2026-08-05 16:28:10 +10:00

gateway-child.test.ts

fix(qa): bound Matrix scenario preparation separately (#119996)

2026-08-07 13:52:59 +08:00

gateway-child.ts

fix(qa): settle Unix commands after PGID cleanup (#120010)

2026-08-08 15:16:44 +08:00

gateway-log-redaction.test.ts

test(buzz): add live QA Lab channel lane (#116298)

2026-07-30 17:44:27 -07:00

gateway-log-redaction.ts

test(buzz): add live QA Lab channel lane (#116298)

2026-07-30 17:44:27 -07:00

gateway-log-sentinel.test.ts

chore: migrate active GPT-5.5 references to GPT-5.6 (#104452)

2026-07-11 06:30:57 -07:00

gateway-log-sentinel.ts

refactor(extensions): remove dead QA and utility exports (#105914)

2026-07-13 00:34:46 -07:00

gateway-process-boundary.test.ts

refactor: privatize diagnostics, Meet, and QA test seams (#107869)

2026-07-14 17:42:34 -07:00

gateway-process-boundary.ts

refactor: deduplicate extension normalization primitives (#115650)

2026-07-29 04:10:06 -04:00

gateway-rpc-client.test.ts

fix(qa): bound Matrix scenario preparation separately (#119996)

2026-08-07 13:52:59 +08:00

gateway-rpc-client.ts

fix(qa): bound Matrix scenario preparation separately (#119996)

2026-08-07 13:52:59 +08:00

harness-parity.test.ts

refactor(qa-lab): trim dead export surface (#107016)

2026-07-13 19:32:13 -07:00

harness-parity.ts

refactor(qa-lab): trim dead export surface (#107016)

2026-07-13 19:32:13 -07:00

harness-runtime.integration.test.ts

fix(qa): align runner inbound dispatch contract (#111507)

2026-07-19 11:51:24 -07:00

harness-runtime.ts

fix(qa): align runner inbound dispatch contract (#111507)

2026-07-19 11:51:24 -07:00

hook-agent-account-routing.e2e.test.ts

test(qa): assert hook account rejection reasons

2026-07-30 02:59:06 -05:00

ignored-response-body.test.ts

fix(qa): discard ignored guarded response bodies (#110443)

2026-07-18 18:08:16 +01:00

ignored-response-body.ts

fix(qa): discard ignored guarded response bodies (#110443)

2026-07-18 18:08:16 +01:00

jsonl-replay.test.ts

fix(qa): reject runtime tool intent without result evidence (#113502)

2026-07-27 11:00:59 +08:00

jsonl-replay.ts

fix(qa): reject runtime tool intent without result evidence (#113502)

2026-07-27 11:00:59 +08:00

lab-server-capture.test.ts

chore: migrate active GPT-5.5 references to GPT-5.6 (#104452)

2026-07-11 06:30:57 -07:00

lab-server-capture.ts

…

lab-server-ui.test.ts

…

lab-server-ui.ts

feat(tooling): enforce noUncheckedIndexedAccess in the extensions lane (NUIA phase 4) (#105132)

2026-07-12 09:17:31 +01:00

lab-server.test.ts

fix(qa): attest realized profile executions (#119761)

2026-08-06 08:33:02 +08:00

lab-server.ts

refactor(qa): make module flows implementation-aware (#118069)

2026-08-06 19:51:28 +08:00

lab-server.types.ts

…

live-scenario-timeouts.test.ts

fix(commands): restore tool inventory for dynamic models (#119306)

2026-08-05 06:57:21 +08:00

live-timeout.test.ts

test: fix current main CI baselines (#120080)

2026-08-06 17:52:37 -07:00

live-timeout.ts

…

managed-gateway-service-lifecycle.e2e.test.ts

test(qa): cover managed gateway service lifecycle (#118855)

2026-08-04 07:07:27 +08:00

mantis-options.runtime.ts

…

mantis-phase-timer.runtime.ts

refactor(extensions): remove dead QA and utility exports (#105914)

2026-07-13 00:34:46 -07:00

manual-lane.runtime.test.ts

fix(qa): release transport leases after gateway stop

2026-07-18 08:23:34 +01:00

manual-lane.runtime.ts

fix(qa): release transport leases after gateway stop

2026-07-18 08:23:34 +01:00

matrix-channel-driver.lifecycle.live.test.ts

refactor(qa): drive Matrix lifecycle through channel drivers (#101055)

2026-07-06 23:13:21 -07:00

model-catalog.runtime.test.ts

test: tighten additional async polling (#108450)

2026-07-15 13:14:35 -07:00

model-catalog.runtime.ts

refactor(process): route bounded commands through Execa (#106495)

2026-07-13 11:07:35 -07:00

model-selection.runtime.test.ts

refactor(qa-lab): trim dead export surface (#107016)

2026-07-13 19:32:13 -07:00

model-selection.runtime.ts

refactor(qa-lab): trim dead export surface (#107016)

2026-07-13 19:32:13 -07:00

model-selection.ts

refactor(deadcode): tighten extension root modeling (#108538)

2026-07-15 18:18:23 -07:00

model-switch-eval.test.ts

…

model-switch-eval.ts

…

multipass.runtime.test.ts

fix(qa): honor fail-fast across unified suite runners (#113843)

2026-07-25 13:26:51 -07:00

multipass.runtime.ts

fix(qa): honor fail-fast across unified suite runners (#113843)

2026-07-25 13:26:51 -07:00

node-exec.test.ts

fix(qa-lab): bound Node binary lookup (#109428)

2026-07-18 09:35:16 +01:00

node-exec.ts

fix(qa-lab): bound Node binary lookup (#109428)

2026-07-18 09:35:16 +01:00

parity-shared.ts

fix(qa): separate provider plans from runtime calls (#113399)

2026-07-24 20:55:54 +00:00

plugin-subagent-current-requester.e2e.test.ts

fix(plugins): deliver subagent completion to current requester (#116091)

2026-07-30 07:06:17 +08:00

plugin-subagent-self-yield-followup.e2e.test.ts

fix(subagents): wake the parent when a follow-up finishes a yielded child (#120187)

2026-08-08 06:46:19 -07:00

posix-command-settlement.test.ts

fix(qa): settle Unix commands after PGID cleanup (#120010)

2026-08-08 15:16:44 +08:00

posix-command-settlement.ts

fix(qa): settle Unix commands after PGID cleanup (#120010)

2026-08-08 15:16:44 +08:00

posix-process-group.test.ts

fix(qa): settle Unix commands after PGID cleanup (#120010)

2026-08-08 15:16:44 +08:00

posix-process-group.ts

fix(qa): settle Unix commands after PGID cleanup (#120010)

2026-08-08 15:16:44 +08:00

process-tree-cpu.test.ts

fix(qa): bound process snapshot probes (#109207)

2026-07-16 16:54:54 -07:00

process-tree-cpu.ts

fix(qa): bound process snapshot probes (#109207)

2026-07-16 16:54:54 -07:00

process-tree-cpu.windows.test.ts

fix(qa): bound process snapshot probes (#109207)

2026-07-16 16:54:54 -07:00

profile-evidence-plan.test.ts

fix(qa): attest realized profile executions (#119761)

2026-08-06 08:33:02 +08:00

profile-evidence-plan.ts

fix(qa): attest realized profile executions (#119761)

2026-08-06 08:33:02 +08:00

profile-planning.ts

refactor(qa): make module flows implementation-aware (#118069)

2026-08-06 19:51:28 +08:00

profile-selection.test.ts

fix(qa): restore Telegram release validation (#118542)

2026-08-03 17:22:29 +08:00

progress-format.ts

…

protocol.ts

refactor(plugin-sdk): narrow wildcard barrels to explicit used exports (#108440)

2026-07-16 00:45:23 -07:00

qa-agent-bootstrap.ts

…

qa-agent-workspace.test.ts

refactor: privatize diagnostics, Meet, and QA test seams (#107869)

2026-07-14 17:42:34 -07:00

qa-agent-workspace.ts

refactor: privatize diagnostics, Meet, and QA test seams (#107869)

2026-07-14 17:42:34 -07:00

qa-bus-protocol.test.ts

fix(qa): repair package Telegram harness boundaries (#108499)

2026-07-15 19:24:55 -07:00

qa-bus-protocol.ts

fix(qa): repair package Telegram harness boundaries (#108499)

2026-07-15 19:24:55 -07:00

qa-channel-transport.test.ts

fix(qa): preserve scoped native message and command identity (#119499)

2026-08-05 05:34:43 -07:00

qa-channel-transport.ts

refactor(qa): consolidate scenario runtime helpers (#119623)

2026-08-05 08:01:58 -07:00

qa-cli-process.ts

fix(ui): keep task transcripts in the task sidebar (#120463)

2026-08-08 01:45:44 -07:00

qa-credentials-admin.runtime.test.ts

QA-lab credentials-admin response bound (#101131)

2026-07-07 04:56:07 +01:00

qa-credentials-admin.runtime.ts

QA-lab credentials-admin response bound (#101131)

2026-07-07 04:56:07 +01:00

qa-credentials-common.runtime.test.ts

…

qa-credentials-common.runtime.ts

…

qa-credentials-fingerprint.runtime.ts

…

qa-gateway-config.test.ts

fix(qa): repair release validation scenarios (#119150)

2026-08-05 09:56:06 +08:00

qa-gateway-config.ts

fix(qa): repair release validation scenarios (#119150)

2026-08-05 09:56:06 +08:00

qa-thinking.ts

…

qa-transport-registry.test.ts

refactor(qa): make module flows implementation-aware (#118069)

2026-08-06 19:51:28 +08:00

qa-transport-registry.ts

refactor(qa): make module flows implementation-aware (#118069)

2026-08-06 19:51:28 +08:00

qa-transport.test.ts

fix(qa): reject deleted replies and failed streaming outcomes (#119660)

2026-08-05 09:30:42 -07:00

qa-transport.ts

fix(qa): reject deleted replies and failed streaming outcomes (#119660)

2026-08-05 09:30:42 -07:00

qa-web-search-provider.shared.ts

refactor(extensions): remove unused Discord and QA Lab exports (#106286)

2026-07-13 09:25:45 -07:00

qa-web-search-provider.test.ts

fix(search): propagate cancellation through provider-owned requests (#118895)

2026-08-03 12:46:17 -07:00

qa-web-search-provider.ts

fix(search): propagate cancellation through provider-owned requests (#118895)

2026-08-03 12:46:17 -07:00

reply-failure.test.ts

fix(qa): restore full-stack scenarios and truthful suite exits (#115290)

2026-07-28 13:00:22 -04:00

reply-failure.ts

fix(qa): restore full-stack scenarios and truthful suite exits (#115290)

2026-07-28 13:00:22 -04:00

repo-path.ts

…

report.test.ts

fix(qa): preserve transport ownership and truthful reporting (#116911)

2026-07-31 09:00:44 -07:00

report.ts

fix(qa): preserve transport ownership and truthful reporting (#116911)

2026-07-31 09:00:44 -07:00

run-config.test.ts

refactor(qa): make module flows implementation-aware (#118069)

2026-08-06 19:51:28 +08:00

run-config.ts

refactor(qa): make module flows implementation-aware (#118069)

2026-08-06 19:51:28 +08:00

runtime-api.ts

fix(qa-channel): bind message actions to authorized targets (#108433)

2026-07-15 14:36:20 -07:00

runtime-pair-lane-selection.ts

refactor(qa): make module flows implementation-aware (#118069)

2026-08-06 19:51:28 +08:00

runtime-parity-cache-diagnostics.test.ts

fix(qa): preserve missing Ollama cache telemetry (#116424)

2026-07-30 16:37:54 +00:00

runtime-parity-cache-diagnostics.ts

fix(codex): stabilize prompts and expose real cache regressions (#115238)

2026-07-28 13:26:04 -04:00

runtime-parity-outcomes.test.ts

fix(qa): allow tracked runtime harness gaps (#114667)

2026-07-27 14:09:14 -04:00

runtime-parity-session-selection.test.ts

fix(qa): reject runtime tool intent without result evidence (#113502)

2026-07-27 11:00:59 +08:00

runtime-parity-timing.ts

improve: reduce GPT-5.6 coding harness overhead (#114574)

2026-07-27 19:24:52 -04:00

runtime-parity-usage.ts

fix(codex): stabilize prompts and expose real cache regressions (#115238)

2026-07-28 13:26:04 -04:00

runtime-parity.execution-order.test.ts

fix: make Codex harness caching and benchmark parity exact (#115663)

2026-07-29 03:33:36 -04:00

runtime-parity.test.ts

fix(qa): require a fresh reply and real tool use after switching models (#119662)

2026-08-08 03:05:32 +08:00

runtime-parity.ts

fix(qa): require a fresh reply and real tool use after switching models (#119662)

2026-08-08 03:05:32 +08:00

runtime-tool-fixture.test.ts

test(qa): deduplicate runtime tool fixtures (#117827)

2026-08-01 22:05:55 -07:00

runtime-tool-fixture.ts

fix(qa): avoid classifying fetched prose as tool failure (#116444)

2026-07-31 14:21:57 +08:00

runtime-tool-metadata.ts

refactor(qa): canonicalize runtime-pair lanes (#112603)

2026-07-23 10:06:12 +09:00

scenario-catalog-anthropic-live.test.ts

fix(qa): keep live provider scenarios out of mock lanes (#119498)

2026-08-04 22:52:57 -07:00

scenario-catalog-browser-coverage.test.ts

fix: restore trustworthy end-to-end QA and cross-channel delivery (#115404)

2026-07-29 12:45:27 -04:00

scenario-catalog-causality.test.ts

test(qa): isolate active memory request traces (#119502)

2026-08-04 23:08:31 -07:00

scenario-catalog-channels.test.ts

fix(qa): enforce scenario provider lane metadata (#119908)

2026-08-06 23:33:17 +08:00

scenario-catalog-compaction.test.ts

fix(qa): preserve profile execution contracts (#119519)

2026-08-05 15:07:40 +08:00

scenario-catalog-matrix-session-identity-proof.test.ts

fix(qa): verify Matrix DM rooms really share or isolate their sessions (#119663)

2026-08-05 08:36:32 -07:00

scenario-catalog-memory-ranking-proof.test.ts

fix(qa): prove newer session facts outrank competing durable memory (#119496)

2026-08-04 23:34:46 -07:00

scenario-catalog-memory-recall-proof.test.ts

fix(qa): require a fresh response for memory recall (#119493)

2026-08-04 22:56:58 -07:00

scenario-catalog-model-switch-follow-up-proof.test.ts

fix(qa): require a fresh reply and real tool use after switching models (#119662)

2026-08-08 03:05:32 +08:00

scenario-catalog-model-switch-tool-continuity-proof.test.ts

fix(qa): require a fresh reply and real tool use after switching models (#119662)

2026-08-08 03:05:32 +08:00

scenario-catalog.inference.test.ts

fix(inference): simplify lifecycle and preserve Codex tool identity (#117298)

2026-08-01 02:05:45 -07:00

scenario-catalog.no-meta.test.ts

fix(agents): observe native provider prompt egress (#119219)

2026-08-04 22:46:07 +08:00

scenario-catalog.openai-live.test.ts

fix(qa): keep live provider scenarios out of mock lanes (#119498)

2026-08-04 22:52:57 -07:00

scenario-catalog.runtime-pair.test.ts

fix(qa): replace false Codex pin compatibility evidence (#118012)

2026-08-07 08:45:26 +08:00

scenario-catalog.slack-blocked-lifecycle.test.ts

fix(qa): exercise blocked Slack lifecycle on selected account (#117805)

2026-08-02 18:22:44 +08:00

scenario-catalog.test-utils.ts

fix(qa): repair release validation scenarios (#119150)

2026-08-05 09:56:06 +08:00

scenario-catalog.test.ts

test(qa): add managed-worktrees CLI lifecycle scenario coverage (#120335)

2026-08-07 14:18:12 -07:00

scenario-catalog.ts

fix(qa): enforce scenario provider lane metadata (#119908)

2026-08-06 23:33:17 +08:00

scenario-flow-runner-character-safety.test.ts

fix(qa): reject silent and out-of-character persona conversations (#119497)

2026-08-05 05:37:54 -07:00

scenario-flow-runner-redaction-proof.test.ts

fix(qa): prove secret fixtures were read before redaction (#119495)

2026-08-04 23:00:32 -07:00

scenario-flow-runner.test-support.ts

test(qa): isolate voice talkback flow coverage

2026-08-02 18:35:48 +08:00

scenario-flow-runner.test.ts

fix(agents): observe native provider prompt egress (#119219)

2026-08-04 22:46:07 +08:00

scenario-flow-runner.ts

fix(qa): require a fresh reply and real tool use after switching models (#119662)

2026-08-08 03:05:32 +08:00

scenario-flow-runner.voice-talkback.test.ts

test(qa): enforce live voice model

2026-08-02 19:54:38 +08:00

scenario-lane.test.ts

fix(qa): enforce scenario provider lane metadata (#119908)

2026-08-06 23:33:17 +08:00

scenario-lane.ts

fix(qa): enforce scenario provider lane metadata (#119908)

2026-08-06 23:33:17 +08:00

scenario-module-flow.test.ts

refactor(qa): make module flows implementation-aware (#118069)

2026-08-06 19:51:28 +08:00

scenario-module-flow.ts

fix(qa): enforce scenario provider lane metadata (#119908)

2026-08-06 23:33:17 +08:00

scenario-runtime-api.test.ts

fix(qa): require a fresh reply and real tool use after switching models (#119662)

2026-08-08 03:05:32 +08:00

scenario-runtime-api.ts

fix(qa): require a fresh reply and real tool use after switching models (#119662)

2026-08-08 03:05:32 +08:00

scenario.ts

…

scorecard-evidence.test.ts

fix(qa): attest realized profile executions (#119761)

2026-08-06 08:33:02 +08:00

scorecard-evidence.ts

fix(qa): attest realized profile executions (#119761)

2026-08-06 08:33:02 +08:00

scorecard-taxonomy.ts

refactor(qa): make scenario selection taxonomy-owned (#114055)

2026-07-28 22:23:41 +08:00

self-check-runner.ts

…

self-check-scenario.ts

fix(qa-channel): bind message actions to authorized targets (#108433)

2026-07-15 14:36:20 -07:00

self-check.test.ts

fix(qa-channel): enforce message and thread lifecycle ownership (#116801)

2026-07-31 04:56:07 -07:00

self-check.ts

fix(qa): release transport leases after gateway stop

2026-07-18 08:23:34 +01:00

shell-quote.ts

…

static-ssh-worker-provider.test.ts

refactor(qa-lab): trim dead export surface (#107016)

2026-07-13 19:32:13 -07:00

static-ssh-worker-provider.ts

refactor(qa-lab): trim dead export surface (#107016)

2026-07-13 19:32:13 -07:00

suite-artifacts.ts

fix(qa): attest realized profile executions (#119761)

2026-08-06 08:33:02 +08:00

suite-launch.runtime.test.ts

fix(qa): attest realized profile executions (#119761)

2026-08-06 08:33:02 +08:00

suite-launch.runtime.ts

fix(qa): enforce scenario provider lane metadata (#119908)

2026-08-06 23:33:17 +08:00

suite-merge-patch.test.ts

chore: migrate active GPT-5.5 references to GPT-5.6 (#104452)

2026-07-11 06:30:57 -07:00

suite-merge-patch.ts

…

suite-model-selection.ts

fix(qa): enforce scenario provider lane metadata (#119908)

2026-08-06 23:33:17 +08:00

suite-planning-module-flows.test.ts

refactor(qa): make module flows implementation-aware (#118069)

2026-08-06 19:51:28 +08:00

suite-planning.test.ts

fix(qa): select the scenario's required provider before its models (#119491)

2026-08-05 05:53:02 -07:00

suite-planning.ts

refactor(qa): make module flows implementation-aware (#118069)

2026-08-06 19:51:28 +08:00

suite-progress.ts

fix(qa-lab): unify runner control plane (#112838)

2026-07-27 14:05:14 +08:00

suite-provider-selection.test.ts

fix(qa): enforce scenario provider lane metadata (#119908)

2026-08-06 23:33:17 +08:00

suite-round-trip.test.ts

refactor(qa): migrate Telegram scenarios into QA Lab (#108430)

2026-07-15 12:55:00 -07:00

suite-round-trip.ts

refactor(qa): migrate Telegram scenarios into QA Lab (#108430)

2026-07-15 12:55:00 -07:00

suite-run-isolated.cleanup.test.ts

fix(qa): attest realized profile executions (#119761)

2026-08-06 08:33:02 +08:00

suite-run-isolated.ts

fix(qa): attest realized profile executions (#119761)

2026-08-06 08:33:02 +08:00

suite-run-standard.parity-retry.test.ts

fix(qa): report cleanup failures truthfully (#119727)

2026-08-06 05:16:35 +08:00

suite-run-standard.ts

fix(qa): attest realized profile executions (#119761)

2026-08-06 08:33:02 +08:00

suite-run.runtime.ts

refactor(qa): make module flows implementation-aware (#118069)

2026-08-06 19:51:28 +08:00

suite-runtime-agent-common.ts

…

suite-runtime-agent-media.test.ts

fix(qa): generated-image evidence waits despite successful delivery (#117799)

2026-08-02 17:24:35 +08:00

suite-runtime-agent-media.ts

fix(qa): generated-image evidence waits despite successful delivery (#117799)

2026-08-02 17:24:35 +08:00

suite-runtime-agent-process.integration.test.ts

fix(ui): keep task transcripts in the task sidebar (#120463)

2026-08-08 01:45:44 -07:00

suite-runtime-agent-process.test.ts

fix(ui): keep task transcripts in the task sidebar (#120463)

2026-08-08 01:45:44 -07:00

suite-runtime-agent-process.ts

fix(ui): keep task transcripts in the task sidebar (#120463)

2026-08-08 01:45:44 -07:00

suite-runtime-agent-session.test.ts

fix(agents): recover from invalid compaction summaries (#119137)

2026-08-04 19:43:25 +08:00

suite-runtime-agent-session.ts

fix(agents): recover from invalid compaction summaries (#119137)

2026-08-04 19:43:25 +08:00

suite-runtime-agent-tools.test.ts

fix(qa): use built plugin tools server (#116442)

2026-07-31 14:49:24 +08:00

suite-runtime-agent-tools.ts

fix(qa): use built plugin tools server (#116442)

2026-07-31 14:49:24 +08:00

suite-runtime-agent.ts

fix(ui): keep task transcripts in the task sidebar (#120463)

2026-08-08 01:45:44 -07:00

suite-runtime-flow.test.ts

fix(qa): require a fresh reply and real tool use after switching models (#119662)

2026-08-08 03:05:32 +08:00

suite-runtime-flow.ts

fix(qa): require a fresh reply and real tool use after switching models (#119662)

2026-08-08 03:05:32 +08:00

suite-runtime-gateway.test.ts

fix: restore trustworthy end-to-end QA and cross-channel delivery (#115404)

2026-07-29 12:45:27 -04:00

suite-runtime-gateway.ts

refactor(qa): consolidate scenario runtime helpers (#119623)

2026-08-05 08:01:58 -07:00

suite-runtime-parity-result.ts

improve: reduce GPT-5.6 coding harness overhead (#114574)

2026-07-27 19:24:52 -04:00

suite-runtime-parity-runner.cleanup.test.ts

fix(qa): attest realized profile executions (#119761)

2026-08-06 08:33:02 +08:00

suite-runtime-parity-runner.control-ui.test.ts

test(qa): keep Control UI parity fixture typed (#120090)

2026-08-07 09:08:12 +08:00

suite-runtime-parity-runner.test.ts

fix(qa): allow tracked runtime harness gaps (#114667)

2026-07-27 14:09:14 -04:00

suite-runtime-parity-runner.ts

fix(qa): attest realized profile executions (#119761)

2026-08-06 08:33:02 +08:00

suite-runtime-transport.test.ts

fix(qa-channel): isolate account evidence and exclude deleted messages (#116744)

2026-07-31 04:19:41 -07:00

suite-runtime-transport.ts

fix(qa-channel): isolate account evidence and exclude deleted messages (#116744)

2026-07-31 04:19:41 -07:00

suite-runtime-types.ts

refactor(qa): route live channel selectors through suite host (#108429)

2026-07-15 13:35:34 -07:00

suite-summary.test.ts

fix(qa): authenticate actual scenario execution evidence (#117004)

2026-07-31 13:03:12 -07:00

suite-summary.ts

fix(qa): report failed evaluations and scenario runs accurately (#118120)

2026-08-02 12:38:35 -07:00

suite-support.ts

fix(qa): preserve control UI policy across all suite runtimes (#119656)

2026-08-05 08:24:38 -07:00

suite-test-helpers.ts

refactor(qa): make module flows implementation-aware (#118069)

2026-08-06 19:51:28 +08:00

suite-types.ts

fix(qa): attest realized profile executions (#119761)

2026-08-06 08:33:02 +08:00

suite.summary-json.test.ts

fix(qa): attest realized profile executions (#119761)

2026-08-06 08:33:02 +08:00

suite.test.ts

fix(qa): attest realized profile executions (#119761)

2026-08-06 08:33:02 +08:00

suite.ts

fix(qa): require a fresh reply and real tool use after switching models (#119662)

2026-08-08 03:05:32 +08:00

temp-dir.test-helper.ts

…

test-file-scenario-command-lifecycle.test.ts

fix(qa): settle Unix commands after PGID cleanup (#120010)

2026-08-08 15:16:44 +08:00

test-file-scenario-command-lifecycle.ts

fix(qa): settle Unix commands after PGID cleanup (#120010)

2026-08-08 15:16:44 +08:00

test-file-scenario-docker-batch.ts

fix(qa): run maturity Docker scenarios through the aggregate scheduler (#114447)

2026-07-28 22:41:35 +08:00

test-file-scenario-runner.e2e-routing.test.ts

fix(qa): verify requested scenarios and reject silent character runs (#116912)

2026-07-31 09:18:05 -07:00

test-file-scenario-runner.test.ts

test(qa): derive UX producer aggregate status (#120418)

2026-08-07 18:55:19 -07:00

test-file-scenario-runner.ts

fix(qa): provision ffmpeg for Playwright scenarios (#119064)

2026-08-03 19:40:39 -07:00

test-file-scenario-script-evidence.ts

fix(xai): follow OAuth default model automatically (#115617)

2026-07-29 23:05:29 +00:00

test-file-scenario-vitest-report.ts

fix(qa): verify requested scenarios and reject silent character runs (#116912)

2026-07-31 09:18:05 -07:00

timer-timeouts.test.ts

…

timer-timeouts.ts

…

token-efficiency-report.test.ts

fix(codex): stabilize prompts and expose real cache regressions (#115238)

2026-07-28 13:26:04 -04:00

token-efficiency-report.ts

fix(codex): stabilize prompts and expose real cache regressions (#115238)

2026-07-28 13:26:04 -04:00

tool-coverage-report.test.ts

fix(security): harden CodeQL-reported boundaries (#115055)

2026-07-28 16:47:12 +08:00

tool-coverage-report.ts

fix(security): harden CodeQL-reported boundaries (#115055)

2026-07-28 16:47:12 +08:00

tool-search-gateway.fixture.test.ts

feat(tools): support batched tool search queries (#118623)

2026-08-05 16:28:10 +10:00

tool-search-gateway.fixture.ts

feat(tools): support batched tool search queries (#118623)

2026-08-05 16:28:10 +10:00

voice-preflight.fixture.ts

fix(qa): share bundled voice preflight fixture

2026-08-02 19:33:28 +08:00

web-runtime.test.ts

fix(qa-lab): keep bounded web snapshots UTF-16 safe (#104249)

2026-07-11 04:49:00 -07:00

web-runtime.ts

fix(qa-lab): keep bounded web snapshots UTF-16 safe (#104249)

2026-07-11 04:49:00 -07:00

windows-system-tools.test.ts

refactor(qa-lab): trim dead export surface (#107016)

2026-07-13 19:32:13 -07:00

windows-system-tools.ts

refactor(qa-lab): trim dead export surface (#107016)

2026-07-13 19:32:13 -07:00