Commit Graph

15 Commits

Author SHA1 Message Date
Peter Steinberger fba9a094db test(runner): delete duplicate SQLite replay (#121983)
Co-authored-by: Amp <amp@ampcode.com>
2026-08-11 02:54:58 -07:00
Peter Steinberger 600787a0cb test(gateway): isolate live-event sweep assertion (#106216)
* test(gateway): isolate agent run registry state

* test(infra): reset agent event state between files
2026-08-09 05:06:40 -07:00
Vincent Koc 45643863d7 fix(test): isolate shared state between test files (#118781)
* fix(test): isolate session suspension state

* fix(test): isolate nonzero worker exit coverage

* fix(test): keep suspension reset generations monotonic

* fix(test): isolate archived session UI mocks

* fix(deps): patch fast-uri and undici advisories

* fix(test): isolate stateful UI suites
2026-08-04 15:26:22 +08:00
Peter Steinberger 5fcf7e75de test: clear plugin runtimes between shared files (#111556) 2026-07-19 15:28:46 -07:00
Peter Steinberger 2e70de9d32 fix(mcp): own sandbox CSP fail-closed validation in the decoder (#110267)
* fix(mcp): reject sandbox CSP metadata that normalizes to no policy

A present ?csp= value that decodes to valid JSON but is not a usable CSP
(e.g. null or a wrong-shaped object) normalized to undefined and was
indistinguishable from an absent parameter, so the gateway served proxy
HTML under the default policy. encodeCsp omits the query param entirely
for such values, so a present-but-empty policy is never legitimate;
throw so the sandbox endpoint fails closed with 400. Found by review on
the revert of 73685b4e7c946; the gap predates that commit.

* fix(mcp): treat empty csp query value as malformed, not absent

?csp= with an empty value passed the falsy absent-guard and served proxy
HTML under the default policy. Only a truly absent parameter (null) may
skip validation; an empty string now falls through to JSON.parse and
fails closed with 400.

* refactor(gateway): drop redundant sandbox CSP handler guard

decodeMcpAppSandboxCsp now throws for every present-but-unusable value
(1ca26c508d added the same fail-closed behavior at the handler seam),
so the handler-level 'present but falsy' check is unreachable. Keep the
invariant in the decoder, which owns policy decode semantics.

* fix(test): reset diagnostic listener-presence mirror between non-isolated files

resetOpenClawGlobalDiagnosticState clears the listener sets and deletes
the diagnostic-events state key, but the listener-presence counts live
under a separate globalThis record and survived, so
hasInternalDiagnosticEventListeners() stayed true for every later file
in the worker once any file leaked a registration (e.g. the import-time
listener in src/logging/diagnostic-run-activity.ts whose stop handle is
lost to the module-registry reset). Zero the counts to match the cleared
sets. Root cause of the model-call-diagnostics flake in CI run
29624203224; #110288 added the victim-side reset, this fixes the class.
2026-07-18 02:25:34 +01:00
Peter Steinberger b2e34b03b5 test: serialize Vitest mock resolution in isolate:false shared workers (#109874)
* test: serialize concurrent vitest mock resolution in the non-isolated runner

Root-causes the subagent-orphan-recovery flake (main run 29566318167, shard
agentic-agents-core-subagents): "includes last human message in resume when
available" and "adds config change hint..." received the bare resume template
because prod code called a different readSessionMessagesAsync mock instance
than the one the test configured.

Mechanism: vitest's BareModuleMocker.resolveMocks has no in-flight guard.
pendingIds is cleared only after all parallel resolveId RPCs settle, and every
registration re-invalidates the mock module node. In a shared isolate:false
worker, leaked async work from an earlier file (a real-timer dynamic import)
triggers a fetchModule while the next file's vi.mock registrations are still
pending, starting a second concurrent resolution pass over the same
pendingIds. The slower pass re-registers each manual mock and wipes the
already-evaluated mock module mid-import-chain: importers evaluated before the
wipe (the test file's own binding, subagent-orphan-recovery.test.ts:8) keep
factory instance A while later importers (subagent-orphan-recovery.ts via
test line 16) instantiate a fresh instance B. Only inline-factory mocks split;
vi.hoisted-stable mocks are immune, matching the CI failure signature exactly.

Fix: OpenClawNonIsolatedRunner installs a coalescing wrapper around
moduleRunner.mocker.resolveMocks (once per worker) so concurrent callers share
one pass and registration happens exactly once before any awaiting import
proceeds.

Proof:
- Deterministic repro (poison file leaving a 1ms recurring dynamic-import
  timer + resolveId jitter on session-transcript-readers) reproduced the exact
  CI assertion failures at lines 755/772 on the unfixed runner and passes 5/5
  with the fix.
- New unit tests cover coalescing, idempotent install, and fresh passes.
- Stress: 10x subagent-orphan-recovery.test.ts and 5x the full
  agentic-agents-core-subagents file set (63 files, 1220 tests) at
  OPENCLAW_VITEST_MAX_WORKERS=6, all green.

* test: requeue mock ids queued during an in-flight resolveMocks pass

Review follow-up to the resolveMocks serialization pin: upstream snapshots the
pendingIds contents at pass start and reassigns the static to [] at the end,
so ids queued during the pass's RPC window land in the abandoned array. Pure
coalescing would silently drop those registrations (upstream's racy second
pass previously processed them). The wrapper now captures the queue reference
before each pass, requeues ids pushed past the processed count, and drains
until the queue is empty, so every coalesced caller's vi.mock registration is
applied before its fetch proceeds.

Proof: new unit test "registers ids queued while a pass is in flight before
callers proceed"; deterministic zombie+jitter repro still passes 5/5; full
agentic-agents-core-subagents file set green at OPENCLAW_VITEST_MAX_WORKERS=6.

* test: chain resolveMocks passes per caller instead of sharing one pass

The coalescing pin regressed shard agentic-agents-core-auth
(models-config.providers.auth-provenance.test.ts, runs 29570918219 and
29571513381): sharing one in-flight pass broke the mocker's freshness
invariant. Upstream gives every resolveMocks caller a pass that starts at or
after its call, so ids the caller queued (vi.doUnmock before dynamic imports
in that file's beforeAll/beforeEach) are always registered before its fetch
proceeds. A coalesced caller could instead ride a pass snapshotted before its
ids were queued and import with mock state unresolved. Reproduced locally on a
cold transform cache: the file then loaded the real provider-auth warm worker
plus a live oauth-manager refresh guarded by a 120s withRefreshCallTimeout
(the 105-142s stalls and the "Test timed out in 120000ms" CI failure), and the
real resolveProviderSyntheticAuthWithPlugin returned undefined (the
mode:"none" assertion failure at line 348).

The wrapper now chains each caller onto its own sequential pass. This keeps
both invariants: serialization (a pass queued behind an in-flight one sees the
cleared queue and no-ops, so a snapshot is never registered or its mock
modules invalidated twice - the original subagent-orphan-recovery disease) and
freshness (each caller's pass starts at or after its call). Abandoned-id
requeue is preserved so ids pushed during a pass's RPC window are registered
by the next chained pass instead of dropped.

Proof:
- Cold-cache auth shard (37 files, 528 tests): failed 105-142s with the
  coalescing pin (also with a coalescing-only variant), passed 20s with the
  unfixed runner, passes 3/3 in ~20-30s with the chained pin.
- Long-timer instrumentation pinpointed the stall: provider-auth warm worker
  spawn plus oauth-manager withRefreshCallTimeout delay=120000.
- Original zombie+jitter orphan-recovery repro still passes 5/5.
- Warm agentic-agents-core-subagents (63 files, 1220 tests) and auth shard
  green at OPENCLAW_VITEST_MAX_WORKERS=6; wrapper unit tests updated to
  assert serialization, freshness, requeue, idempotent install.
2026-07-17 11:54:54 +01:00
Peter Steinberger 883995f08d test: stabilize flaky tests at root cause (batch 2) and fix the isolate:false cleanup hole (#109653)
* test: stabilize four flaky tests at root cause

- test/non-isolated-runner.ts: move shared-worker cleanup from onAfterRunSuite
  to onAfterRunFiles. Vitest early-returns runSuite for files that fail during
  collection, skipping onAfterRunSuite, so a crashed sibling left its evaluated
  real modules cached and the next file's vi.mock factories silently never
  applied — the "res.setHeader is not a function" failures in
  http-utils.authorize-request.test.ts. onAfterRunFiles fires per file
  regardless of collect/run outcome. Regression meta-test spawns a child
  vitest run (collect-crash sibling + mock-dependent file) and fails on the
  old runner with flavor:real vs flavor:mocked.
- src/gateway/http-utils.authorize-request.test.ts: export every http-common
  binding http-auth-utils imports so the factory stays isolate:false-safe.
- src/gateway/session-message-events.test.ts: drop the beforeAll 60s override;
  the suite harness cold-imports the full gateway server graph, which can
  legitimately exceed 60s under contention. Sibling suites use the shared
  project hookTimeout (120s/180s) for the same boot.
- src/gateway/server-methods/agent.sessions-and-models.test-utils.ts: the
  finalize-throw test polled a 2s waitForAssertion for an off-turn background
  run; signal finalize via a promise resolved from the spy (event-driven,
  bounded by the test timeout) and keep the bounded poll for the follow-up
  respond/warn observations.
- ui/src/pages/chat/chat-responsive.browser.test.ts: budget cold-app first
  renders at 30s (real-app cases boot a cold Vite dev server whose first
  transform can starve past 10s under 6-worker contention) and replace
  one-shot isVisible reads with bounded locator waits for the state-driven
  hover reveal.

Proof: focused runs green; 5x repeats of the three gateway files and 5x
repeats of chat-responsive with OPENCLAW_VITEST_MAX_WORKERS=6 all green;
runner regression test fails pre-fix, passes post-fix.

* test: isolate runner meta-test child env from GitHub Actions

The child vitest inherited GITHUB_ACTIONS/CI from the runner job, which
turned on ANSI colors (breaking the plain-substring assertions) and let
the child's github-actions reporter emit ::error annotations the parent
job rendered as its own failures. Drop GITHUB_ACTIONS/FORCE_COLOR and set
NO_COLOR, which overrides every tinyrainbow enable path.

* test: wait for parseable pid in tsdown-build pid-file polls

waitForFile existence polling can catch writeFileSync's open-truncate
0-byte window, yielding NaN pids and false isProcessAlive failures (the
same class as #109140). Poll until the file parses to a positive pid;
covers all three sibling pid-read sites in the suite.
2026-07-16 22:18:16 -07:00
Peter Steinberger 262baedcf7 test(agents): isolate full-suite shared state (#104682) 2026-07-11 13:28:21 -07:00
Peter Steinberger fe261b0f59 chore(tooling): typecheck root test/** with a dedicated tsgo lane (#104475)
* chore(types): add declaration files for scripts/lib and scripts/e2e modules

* chore(types): add declaration files for top-level script modules (a-m)

* chore(types): add declaration files for top-level script modules (n-z)

* test: use a non-secret-shaped gateway token fixture

* test: type ci workflow guard helpers for the root test lane

* chore(tooling): typecheck root test/** with a dedicated tsgo lane

- test/tsconfig/tsconfig.test.root.json: root-test program (strict unused checks,
  fixtures excluded; two Docker E2E clients that import built dist/** stay out,
  same rationale as the scripts/e2e exclusion in tsconfig.scripts.json)
- tsgo:test:root wired into tsgo:test, check:test-types, scripts/check.mjs, and
  the ci.yml test-types shard, mirroring the tsgo:scripts lane (#104348)
- changed-lane routing: test/**/*.ts (excluding fixtures) and the lane tsconfig
  now trigger 'typecheck test root' in check:changed; previously test/ paths ran
  lint only, so harness type errors surfaced first in CI (#104287 envDir case)
- burn down all 1071 latent type errors in the program: precise param/local
  types across test/scripts, test/vitest, test/e2e, and transitive scripts/e2e
  program members; 205 sibling .d.mts declaration files for imported .mjs
  modules (committed separately); zero any, zero ts-expect-error
- resolve the pre-existing testing star-export ambiguity in
  scripts/e2e/parallels/common.ts with an explicit re-export

Closes #104388

* chore(types): correct declaration fidelity per structured review

- re-derive 51 .d.mts files from implementation data flow instead of
  initializers: fix a wrong never return (runTestProjectsDelegation returns
  the child), add encoding-sensitive exec/spawn overloads (plain-gh), restore
  the full release profile union, make parsed paths string | null, add missing
  parseArgs fields via help/non-help unions, add a missing sibling declaration
  (budget-number-args), drop 15 unused lint directives
- precise install-record/tuple typing removes the type-aware oxlint
  regressions the first declarations caused in scripts/e2e implementations
- route .mts declaration edits under test/ to the testRoot lane and reference
  the test-root project from tsconfig.projects.json so tsgo:all covers it
  (closes both review findings against the lane wiring)

* chore(scripts): keep telegram runner dist typing structural for the boundary guard

* chore(types): declare runtime pack and gateway readiness exports added on main

* test: pin the importTargetPlan form of the plugin-contract plan import

The guard expectation still referenced the raw await import( form that
7ae5996bb3 (#103975) replaced with the importTargetPlan fallback helper;
the assertion fails on current main.
2026-07-11 06:15:41 -07:00
Peter Steinberger ecb6779a16 docs: document root test files 2026-06-04 20:37:28 -04:00
Vincent Koc d10d71cdb6 fix(codex): stabilize app-server cleanup tests 2026-06-01 13:15:05 +02:00
Peter Steinberger aab5410bd5 test: speed up slow test suite (#87611)
* test: speed up slow test suite

* test: preserve fake timer cleanup hooks

* test: avoid timeout readiness race

* test: satisfy reply test types

* test: restore runner and image coverage

* test: restore final media runner path

* test: make cli auth status fixture deterministic

* test: repair runtime alias fixtures
2026-05-28 13:20:19 +01:00
Peter Steinberger bb46b79d3c refactor: internalize OpenClaw agent runtime (#85341)
* refactor: extract agent core package

Introduce packages/agent-core as the OpenClaw-owned home for reusable agent loop, harness, session, prompt, and runtime dependency contracts.

* refactor: extract shared llm runtime

Move provider model registries, stream wrappers, OAuth helpers, and LLM utilities into src/llm with plugin-sdk barrels instead of depending on the old embedded runtime layout.

* refactor: remove pi runtime internals

Rename remaining Pi-shaped agent surfaces to OpenClaw agent runtime names, delete obsolete Pi docs and package graph checks, and add the third-party notice for incorporated code.

* refactor: tighten agent session runtime

Make agent-core/runtime dependencies explicit, consolidate compaction and session transcript helpers, and move model/session helpers behind OpenClaw-owned contracts.

* refactor: remove static model and pi auth paths

Drop static model catalogs and Pi auth bridges, move model/provider facts to manifest-owned runtime contracts, and harden internal embedded-agent utilities.

* refactor: remove legacy provider compat paths

* docs: remove agent parity notes

* fix: skip provider wildcard metadata parsing

* refactor: share session extension sdk loading

* refactor: inline acpx proxy error formatter

* refactor: fold edit recovery into edit tool

* fix: accept extension batch separator

* test: align startup provider plugin expectations

* fix: restore provider-scoped release discovery

* test: align static asset packaging expectations

* fix: run static provider catalogs during scoped discovery

* fix: add provider entry catalogs for scoped live discovery

* fix: load lightweight provider catalog entries

* fix: refresh provider-scoped plugin metadata

* fix: keep provider catalog entries on release live path

* fix: keep static manifest models in release live checks

* fix: harden release model discovery

* fix: reduce OpenAI live cache probe reasoning

* fix: disable OpenAI cache probe reasoning

* ci: extend OpenAI gateway live timeout

* fix: extend live gateway model budget

* fix: stabilize release validation regressions

* fix: honor provider aliases in model rows

* fix: stabilize release validation lanes

* fix: stabilize release memory qa

* ci: stabilize release validation lanes

* ci: prefer ipv4 for live docker node calls

* fix: restore shared tool-call stream wrapper

* ci: remove legacy pi test shard alias

* fix: clean up embedded agent test drift

* fix: stabilize runtime alias status

* fix: clean up embedded agent ci drift

* fix: restore release ci invariants

* fix: clean up post-rebase runtime drift

* fix: restore release ci checks

* fix: restore release ci after rebase

* fix: remove stale pi runtime path

* test: align compaction runtime expectations

* test: update plugin prerelease expectations

* fix: handle claude live tool approvals

* fix: stabilize release validation gates

* fix: finish agent runtime import

* test: finish post-rebase agent runtime mocks

* fix: keep codex compaction native

* fix: stabilize codex app-server hook tests

* test: isolate codex diagnostic active run

* test: remove codex diagnostic completion race

# Conflicts:
#	extensions/codex/src/app-server/run-attempt.test.ts

* ci: fix full release manifest performance run id

* refactor: narrow llm plugin sdk boundary

* chore: drop generated google boundary stamps

* fix: repair rebase fallout

* fix: clean up rebased runtime references

* fix: decode codex jwt payloads as base64url

* fix: preserve shipped pi runtime alias

* fix: add scoped sdk virtual modules

* fix: decode llm codex oauth jwt as base64url

* fix: avoid stale vertex adc negative cache

* fix: harden tool arg decoding and codeql path

* fix: keep vertex adc negative checks live

* refactor: consolidate codex jwt and edit helpers

* fix: await codex oauth node runtime imports

* fix: preserve sdk tool and notice contracts

* fix: preserve shipped compat config boundaries

* fix: align codex oauth callback host

* fix: terminate agent-core loop streams on failure

* fix: keep codex oauth callback alive during fallback

* ci: include session tools in critical codeql scans

* fix: keep Cloudflare Anthropic provider auth header

* docs: redirect legacy pi runtime pages

* fix: honor bundled web provider compat discovery

* fix: protect session output spill files

* fix: keep legacy agent dir env blocked

* fix: contain auto-discovered skill symlinks

* fix: harden agent core sdk proxy surfaces

* fix: restore approval reaction sdk compat

* fix: keep live docker runs bounded

* fix: keep codex oauth redirect host aligned

* fix: resolve post-rebase agent runtime drift

* fix: redact anthropic oauth parse failures

* fix: preserve responses strict tool shaping

* fix: repair agent runtime rebase cleanup

* docs: redirect retired parity pages

* fix: bound auto-discovered resources to roots

* fix: repair post-rebase agent test drift

* fix: preserve bundled provider allowlist migration

* fix: preserve manifest-owned provider aliases

* fix: declare photon image dependency

* fix: keep provider headers out of proxy body

* fix: preserve shipped env aliases

* fix: refresh control ui i18n generated state

* fix: quote read fallback paths

* fix: preview edits through configured backend

* test: satisfy core test typecheck

* fix: preserve ZAI usage auth fallback

* test: repair codex diagnostic test

* fix: repair agent runtime rebase drift

* test: finish embedded runner import rename

* fix: repair agent runtime rebase integrations

* test: align compaction oauth fallback expectations

* fix: allow sdk-auth session models

* fix: update doctor tool schema import

* fix: preserve bedrock plugin region

* fix: stream harmony-like prose immediately

* ci: include session runtime in codeql shards

* fix: repair latest rebase integrations

* fix: honor explicit codex websocket transport

* fix: keep openai-compatible credentials provider-scoped

* fix: refresh sdk api baseline after rebase

* fix: route cli runtime aliases through openclaw harness

* test: rename stale harness mock expectation

* test: rename embedded agent overflow calls

* test: clean embedded auth test wording

* test: use openclaw stream types in deepinfra cache test

* fix: refresh sdk api baseline on latest main

* fix: honor bundled discovery compat allowlists

* fix: refresh sdk api baseline after latest rebase

* fix: remove stale rebase imports

* test: rename stale model catalog mock

* test: mock renamed doctor runtime modules

* fix: map canonical kimi env auth

* fix: use internal model registry in bench script

* fix: migrate deepinfra provider catalog entry

* fix: enforce builtin tool suppression

* fix: route compaction auth and proxy payloads safely

* refactor: prune unused llm registry leftovers

* test: update codex hooks session import

* test: fix model picker ci coverage

* test: align model picker auth mock types
2026-05-27 19:24:04 +01:00
Peter Steinberger 59318d9ff8 Tests: preserve isolated home across non-isolated files 2026-04-07 07:54:39 +01:00
Peter Steinberger 1ceaad18a6 test: harden vitest no-isolate coverage 2026-03-22 10:48:21 -07:00