The standalone debug proxy wrote raw request and response headers into
capture rows while the patched-fetch runtime redacted at the parallel
call sites, so a capture taken through the proxy could persist
Authorization, Cookie, and API-key values to disk.
Move the runtime's redaction policy into a leaf module both writers
import instead of adding a second copy, so the two capture paths cannot
drift. The shared helper also flattens node's array-valued headers,
which the standalone proxy passes in directly, and keeps value-level
registered-secret redaction for header names that are not themselves
sensitive.
Reported by SebTardif in #90009; supersedes #82951, which redacted only
by header name and predates the proxy-server rewrite.
* fix(proxy-capture): guard body-less arrayBuffer reads against oversized responses
* fix(proxy-capture): exercise body-less fallback in bounded read tests
New tests use mock clones with body: null plus arrayBuffer spies to
prove the content-length precheck guards the !body path. A real
Response clone exposes body.getReader in Node 24, so the prior test
only exercised the streaming branch and would stay green even if
the precheck were deleted.
* chore: retrigger CI
* fix(proxy-capture): reject non-safe content-length before arrayBuffer (#101268)
ClawSweeper P2: the body-less fallback used Number(content-length), so a
huge digit-only Content-Length value could overflow to Infinity, bypass the
Number.isFinite guard, and still call arrayBuffer() — leaving an OOM path in
the hardening PR.
Add declaredContentLengthExceedsCap, which accepts only plain digit strings,
treats any value longer than Number.MAX_SAFE_INTEGER as oversized, and
compares safe-integer parsed values against the cap. Non-numeric or malformed
values fall through to the post-read length check.
Adds a regression test for a 100-digit Content-Length that would previously
have bypassed the guard.
* fix(proxy-capture): normalize zero-padded Content-Length before digit-count guard
* fix(proxy-capture): fail closed without response streams
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* refactor(deadcode): trim auto-reply and CLI exports
* refactor(deadcode): trim cron and task exports
* refactor(deadcode): trim fleet and process exports
* test(deadcode): exercise live task and process seams
* test(fleet): cover stream redaction through owner module
* refactor(security): trim dead internal exports
* refactor(secrets): trim dead internal exports
* refactor(deadcode): trim remaining src exports
* refactor(deadcode): remove test-only runtime exports
* refactor(deadcode): trim pairing test exports
* refactor(deadcode): reconcile refreshed baseline
* test(auto-reply): deduplicate queue state imports
* fix(proxy-capture): truncate inline preview on UTF-8 character boundary
persistEventPayload stores a small UTF-8 preview (dataText) inline for fast CLI
listings and query output, capped at previewLimit bytes (default 8192). It built
the preview with buffer.subarray(0, previewLimit).toString("utf8"), which slices
the buffer at an arbitrary byte offset. For any non-ASCII body (emoji/CJK/accented
text, common in model-provider HTTP traffic), the cut lands mid-multibyte-sequence
and Node emits a trailing U+FFFD replacement char into the stored preview column.
Decode the full buffer first, then truncate with the existing truncateUtf8Prefix
helper (src/utils/utf8-truncate.ts), which backtracks off UTF-8 continuation bytes
so the preview ends on a complete character while staying within the byte budget.
ASCII previews are byte-identical; only malformed trailing U+FFFD is removed.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(proxy-capture): align stub store return type with SharedCaptureBlobRecord
The preview-boundary test stub returned a bare { blobId, sha256, byteLength },
which fails tsgo typecheck against persistPayload's declared
CaptureBlobRecord | SharedCaptureBlobRecord return. Return a complete
SharedCaptureBlobRecord instead. Also use template strings for the multibyte
fixtures to satisfy no-useless-concat. No assertion behavior change.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(proxy-capture): byte-bounded UTF-8 preview, unify store and proxy paths
The prior fix decoded the whole payload with buffer.toString("utf8") before
truncating, an availability regression for large captured payloads, and it left
the proxy-server finishBodyPreviewCapture path on the same byte-subarray decode
that can split a trailing multibyte sequence into U+FFFD.
Add truncateUtf8PrefixFromBuffer: decodes only the byte-bounded prefix and walks
back to verify the trailing multibyte sequence is complete (handles both a hard
byte cap and a buffer already sliced mid-sequence upstream, e.g. a dangling lead
byte). Use it from persistEventPayload (no full-payload decode) and
finishBodyPreviewCapture (same boundary safety), so both preview paths share one
canonical helper.
Add completeUtf8PrefixLength coverage plus proxy-server body-preview regression
tests for dangling-lead, byte-cap, emoji, and oversized-body cases.
Co-Authored-By: Claude <noreply@anthropic.com>
* style(proxy-capture): fix no-useless-assignment and unnecessary template expr
- utf8-truncate.ts: expected is reassigned in every branch, so declare without
the unused initial value (no-useless-assignment).
- proxy-server.body-preview.test.ts: drop the redundant template wrapper around
a single expression (no-unnecessary-template-expression). No behavior change.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(utf8-truncate): clarify completeUtf8PrefixLength byte-budget contract
Document that maxBytes is a byte (not character) limit and that the returned
length is the largest offset that does not split a multibyte sequence.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(proxy-capture): preserve UTF-8 preview boundaries
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* feat(secrets): resolve SecretRef model credentials at egress via process-local sentinels
SecretRef-managed model-provider credentials now travel as opaque
oc-sent-v1 sentinels through auth storage, stream options, and SDK
config; the guarded model fetch injects real values into headers and
URLs immediately before the SSRF-guarded send and fails closed on
unknown sentinels. packages/ai adapters converge on the host guarded
fetch where the SDK supports custom fetch and unwrap at construction
where it does not. Resolved values (and their percent-encoded forms)
register for exact-value log redaction. Kill switch:
OPENCLAW_SECRET_SENTINELS=off. Also fixes a pre-existing unhandled
rejection race in capNonOkResponseBodyLazily (pipeThrough writer leak).
* test(plugin-sdk): update public surface budget
* fix(state): close cached SQLite state and agent databases on process exit
* refactor(state): pair SQLite exit-close with cache lifecycle and rebind stale proxy-capture stores
* fix(cli): register debug-proxy finalize before the state DB exit hook
* refactor(state): narrow to agent-db and proxy-capture exit lifecycle after #100691 landed shared-state close
---------
Co-authored-by: Alex Knight <15041791+amknight@users.noreply.github.com>
Resolve explicit relative SQLite DB paths before caching handles and centralize durable SQLite connection pragmas so busy_timeout is applied before WAL/NFS negotiation.
Detects NFS-backed SQLite database paths in the shared WAL helper and uses rollback journaling for those paths while preserving WAL/checkpoint maintenance on local filesystems. The NFS path now verifies SQLite's effective journal mode before disabling WAL maintenance, and core/memory/proxy-capture callers pass database path context into the centralized helper.
Fixes#90491.
Proof: local focused Vitest/format/lint; autoreview clean after fixing the journal-mode verification finding; Crabbox AWS focused test run `run_2ea7014350da`; Crabbox AWS changed gate `run_c828bbfe7d23`; exact-head GitHub CI green on `59674305ecd863d4815eec6098ccd3daab79ca4f`.
Extract shared normalization/coercion helpers into private @openclaw/normalization-core workspace package while preserving existing plugin SDK helper subpaths.\n\nAlso keeps direct normalization-core imports internal, wires UI/build/loader resolution, and replaces the slow PR network CodeQL lane with a fast added-line boundary scan while retaining full CodeQL for scheduled/manual runs.\n\nVerification: local moved tests, plugin SDK boundary tests, extension loader tests, agents-support shard, UI build/test, build artifacts, lint, workflow guards, autoreview, and GitHub CI passed on PR head 963d893715.
Ayaan Zaidi