25337df1c8
fix(proxy-capture): truncate inline preview on UTF-8 character boundary ( #102409 )
...
* fix(proxy-capture): truncate inline preview on UTF-8 character boundary
persistEventPayload stores a small UTF-8 preview (dataText) inline for fast CLI
listings and query output, capped at previewLimit bytes (default 8192). It built
the preview with buffer.subarray(0, previewLimit).toString("utf8"), which slices
the buffer at an arbitrary byte offset. For any non-ASCII body (emoji/CJK/accented
text, common in model-provider HTTP traffic), the cut lands mid-multibyte-sequence
and Node emits a trailing U+FFFD replacement char into the stored preview column.
Decode the full buffer first, then truncate with the existing truncateUtf8Prefix
helper (src/utils/utf8-truncate.ts), which backtracks off UTF-8 continuation bytes
so the preview ends on a complete character while staying within the byte budget.
ASCII previews are byte-identical; only malformed trailing U+FFFD is removed.
Co-Authored-By: Claude <noreply@anthropic.com >
* test(proxy-capture): align stub store return type with SharedCaptureBlobRecord
The preview-boundary test stub returned a bare { blobId, sha256, byteLength },
which fails tsgo typecheck against persistPayload's declared
CaptureBlobRecord | SharedCaptureBlobRecord return. Return a complete
SharedCaptureBlobRecord instead. Also use template strings for the multibyte
fixtures to satisfy no-useless-concat. No assertion behavior change.
Co-Authored-By: Claude <noreply@anthropic.com >
* fix(proxy-capture): byte-bounded UTF-8 preview, unify store and proxy paths
The prior fix decoded the whole payload with buffer.toString("utf8") before
truncating, an availability regression for large captured payloads, and it left
the proxy-server finishBodyPreviewCapture path on the same byte-subarray decode
that can split a trailing multibyte sequence into U+FFFD.
Add truncateUtf8PrefixFromBuffer: decodes only the byte-bounded prefix and walks
back to verify the trailing multibyte sequence is complete (handles both a hard
byte cap and a buffer already sliced mid-sequence upstream, e.g. a dangling lead
byte). Use it from persistEventPayload (no full-payload decode) and
finishBodyPreviewCapture (same boundary safety), so both preview paths share one
canonical helper.
Add completeUtf8PrefixLength coverage plus proxy-server body-preview regression
tests for dangling-lead, byte-cap, emoji, and oversized-body cases.
Co-Authored-By: Claude <noreply@anthropic.com >
* style(proxy-capture): fix no-useless-assignment and unnecessary template expr
- utf8-truncate.ts: expected is reassigned in every branch, so declare without
the unused initial value (no-useless-assignment).
- proxy-server.body-preview.test.ts: drop the redundant template wrapper around
a single expression (no-unnecessary-template-expression). No behavior change.
Co-Authored-By: Claude <noreply@anthropic.com >
* docs(utf8-truncate): clarify completeUtf8PrefixLength byte-budget contract
Document that maxBytes is a byte (not character) limit and that the returned
length is the largest offset that does not split a multibyte sequence.
Co-Authored-By: Claude <noreply@anthropic.com >
* fix(proxy-capture): preserve UTF-8 preview boundaries
---------
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Peter Steinberger <steipete@gmail.com >
2026-07-10 03:18:38 +01:00
bd4d4c0d31
fix(state): close agent-db and proxy-capture SQLite handles on exit; rebind stale proxy stores after shared-state close ( #100827 )
...
* fix(state): close cached SQLite state and agent databases on process exit
* refactor(state): pair SQLite exit-close with cache lifecycle and rebind stale proxy-capture stores
* fix(cli): register debug-proxy finalize before the state DB exit hook
* refactor(state): narrow to agent-db and proxy-capture exit lifecycle after #100691 landed shared-state close
---------
Co-authored-by: Alex Knight <15041791+amknight@users.noreply.github.com >
2026-07-08 21:48:18 +10:00
614e87cce1
chore: update dependencies ( #100027 )
2026-07-04 14:56:50 -04:00
2c7fe6a39c
test(sqlite): use shared temp directory helper
2026-06-19 01:27:49 +08:00
1d6b4ccfef
fix(proxy): preserve mode-specific blob types
2026-06-19 01:20:06 +08:00
d37de2a39a
fix(proxy): preserve capture store compatibility
2026-06-19 01:20:06 +08:00
0876ed59e0
refactor(proxy): store captures in shared state database
2026-06-19 01:20:06 +08:00
c06b7959ec
fix(proxy): preserve in-memory capture databases
2026-06-16 09:22:48 +02:00
e83926747c
fix(proxy): keep capture storage private
2026-06-16 09:22:48 +02:00
ea346f4361
fix(sqlite): close databases after failed initialization
2026-06-16 03:00:23 +02:00
4ca8bf086c
fix(proxy): close cached SQLite stores by path
...
Track debug proxy capture stores per database path so replacing or concurrently leasing capture paths cannot orphan SQLite handles and WAL checkpoint timers.
2026-06-16 01:54:09 +02:00
5b21a0337b
fix(state): avoid sqlite wal on nfs state volumes
...
Detects NFS-backed SQLite database paths in the shared WAL helper and uses rollback journaling for those paths while preserving WAL/checkpoint maintenance on local filesystems. The NFS path now verifies SQLite's effective journal mode before disabling WAL maintenance, and core/memory/proxy-capture callers pass database path context into the centralized helper.
Fixes #90491 .
Proof: local focused Vitest/format/lint; autoreview clean after fixing the journal-mode verification finding; Crabbox AWS focused test run `run_2ea7014350da`; Crabbox AWS changed gate `run_c828bbfe7d23`; exact-head GitHub CI green on `59674305ecd863d4815eec6098ccd3daab79ca4f`.
2026-06-14 03:32:13 +08:00
d8326f13c3
docs: document proxy and mcp helpers
2026-06-04 20:27:46 -04:00
694ca50e97
Revert "refactor: move runtime state to SQLite"
...
This reverts commit f91de52f0d
2026-05-13 13:33:38 +01:00
f91de52f0d
refactor: move runtime state to SQLite
...
* refactor: remove stale file-backed shims
* fix: harden sqlite state ci boundaries
* refactor: store matrix idb snapshots in sqlite
* fix: satisfy rebased CI guardrails
* refactor: store current conversation bindings in sqlite table
* refactor: store tui last sessions in sqlite table
* refactor: reset sqlite schema history
* refactor: drop unshipped sqlite table migration
* refactor: remove plugin index file rollback
* refactor: drop unshipped sqlite sidecar migrations
* refactor: remove runtime commitments kv migration
* refactor: preserve kysely sync result types
* refactor: drop unshipped sqlite schema migration table
* test: keep session usage coverage sqlite-backed
* refactor: keep sqlite migration doctor-only
* refactor: isolate device legacy imports
* refactor: isolate push voicewake legacy imports
* refactor: isolate remaining runtime legacy imports
* refactor: tighten sqlite migration guardrails
* test: cover sqlite persisted enum parsing
* refactor: isolate legacy update and tui imports
* refactor: tighten sqlite state ownership
* refactor: move legacy imports behind doctor
* refactor: remove legacy session row lookup
* refactor: canonicalize memory transcript locators
* refactor: drop transcript path scope fallbacks
* refactor: drop runtime legacy session delivery pruning
* refactor: store tts prefs only in sqlite
* refactor: remove cron store path runtime
* refactor: use cron sqlite store keys
* refactor: rename telegram message cache scope
* refactor: read memory dreaming status from sqlite
* refactor: rename cron status store key
* refactor: stop remembering transcript file paths
* test: use sqlite locators in agent fixtures
* refactor: remove file-shaped commitments and cron store surfaces
* refactor: keep compaction transcript handles out of session rows
* refactor: derive transcript handles from session identity
* refactor: derive runtime transcript handles
* refactor: remove gateway session locator reads
* refactor: remove transcript locator from session rows
* refactor: store raw stream diagnostics in sqlite
* refactor: remove file-shaped transcript rotation
* refactor: hide legacy trajectory paths from runtime
* refactor: remove runtime transcript file bridges
* refactor: repair database-first rebase fallout
* refactor: align tests with database-first state
* refactor: remove transcript file handoffs
* refactor: sync post-compaction memory by transcript scope
* refactor: run codex app-server sessions by id
* refactor: bind codex runtime state by session id
* refactor: pass memory transcripts by sqlite scope
* refactor: remove transcript locator cleanup leftovers
* test: remove stale transcript file fixtures
* refactor: remove transcript locator test helper
* test: make cron sqlite keys explicit
* test: remove cron runtime store paths
* test: remove stale session file fixtures
* test: use sqlite cron keys in diagnostics
* refactor: remove runtime delivery queue backfill
* test: drop fake export session file mocks
* refactor: rename acp session read failure flag
* refactor: rename acp row session key
* refactor: remove session store test seams
* refactor: move legacy session parser tests to doctor
* refactor: reindex managed memory in place
* refactor: drop stale session store wording
* refactor: rename session row helpers
* refactor: rename sqlite session entry modules
* refactor: remove transcript locator leftovers
* refactor: trim file-era audit wording
* refactor: clean managed media through sqlite
* fix: prefer explicit agent for exports
* fix: use prepared agent for session resets
* fix: canonicalize legacy codex binding import
* test: rename state cleanup helper
* docs: align backup docs with sqlite state
* refactor: drop legacy Pi usage auth fallback
* refactor: move legacy auth profile imports to doctor
* refactor: keep Pi model discovery auth in memory
* refactor: remove MSTeams legacy learning key fallback
* refactor: store model catalog config in sqlite
* refactor: use sqlite model catalog at runtime
* refactor: remove model json compatibility aliases
* refactor: store auth profiles in sqlite
* refactor: seed copied auth profiles in sqlite
* refactor: make auth profile runtime sqlite-addressed
* refactor: migrate hermes secrets into sqlite auth store
* refactor: move plugin install config migration to doctor
* refactor: rename plugin index audit checks
* test: drop auth file assumptions
* test: remove legacy transcript file assertions
* refactor: drop legacy cli session aliases
* refactor: store skill uploads in sqlite
* refactor: keep subagent attachments in sqlite vfs
* refactor: drop subagent attachment cleanup state
* refactor: move legacy session aliases to doctor
* refactor: require node 24 for sqlite state runtime
* refactor: move provider caches into sqlite state
* fix: harden virtual agent filesystem
* refactor: enforce database-first runtime state
* refactor: rename compaction transcript rotation setting
* test: clean sqlite refactor test types
* refactor: consolidate sqlite runtime state
* refactor: model session conversations in sqlite
* refactor: stop deriving cron delivery from session keys
* refactor: stop classifying sessions from key shape
* refactor: hydrate announce targets from typed delivery
* refactor: route heartbeat delivery from typed sqlite context
* refactor: tighten typed sqlite session routing
* refactor: remove session origin routing shadow
* refactor: drop session origin shadow fixtures
* perf: query sqlite vfs paths by prefix
* refactor: use typed conversation metadata for sessions
* refactor: prefer typed session routing metadata
* refactor: require typed session routing metadata
* refactor: resolve group tool policy from typed sessions
* refactor: delete dead session thread info bridge
* Show Codex subscription reset times in channel errors (#80456 )
* feat(plugin-sdk): consolidate session workflow APIs
* fix(agents): allow read-only agent mount reads
* [codex] refresh plugin regression fixtures
* fix(agents): restore compaction gateway logs
* test: tighten gateway startup assertions
* Redact persisted secret-shaped payloads [AI] (#79006 )
* test: tighten device pair notify assertions
* test: tighten hermes secret assertions
* test: assert matrix client error shapes
* test: assert config compat warnings
* fix(heartbeat): remap cron-run exec events to session keys (#80214 )
* fix(codex): route btw through native side threads
* fix(auth): accept friendly OpenAI order for Codex profiles
* fix(codex): rotate auth profiles inside harness
* fix: keep browser status page probe within timeout
* test: assert agents add outputs
* test: pin cron read status
* fix(agents): avoid Pi resource discovery stalls
Co-authored-by: dataCenter430 <titan032000@gmail.com >
* fix: retire timed-out codex app-server clients
* test: tighten qa lab runtime assertions
* test: check security fix outputs
* test: verify extension runtime messages
* feat(wake): expose typed sessionKey on wake protocol + system event CLI
* fix(gateway): await session_end during shutdown drain and track channel + compaction lifecycle paths (#57790 )
* test: guard talk consult call helper
* fix(codex): scale context engine projection (#80761 )
* fix(codex): scale context engine projection
* fix: document Codex context projection scaling
* fix: document Codex context projection scaling
* fix: document Codex context projection scaling
* fix: document Codex context projection scaling
* chore: align Codex projection changelog
* chore: realign Codex projection changelog
* fix: isolate Codex projection patch
---------
Co-authored-by: Eva (agent) <eva+agent-78055@100yen.org >
Co-authored-by: Josh Lehman <josh@martian.engineering >
* refactor: move agent runtime state toward piless
* refactor: remove cron session reaper
* refactor: move session management to sqlite
* refactor: finish database-first state migration
* chore: refresh generated sqlite db types
* refactor: remove stale file-backed shims
* test: harden kysely type coverage
# Conflicts:
# .agents/skills/kysely-database-access/SKILL.md
# src/infra/kysely-sync.types.test.ts
# src/proxy-capture/store.sqlite.test.ts
# src/state/openclaw-agent-db.test.ts
# src/state/openclaw-state-db.test.ts
* refactor: remove cron store path runtime
* refactor: keep compaction transcript handles out of session rows
* refactor: derive embedded transcripts from sqlite identity
* refactor: remove embedded transcript locator handoff
* refactor: remove runtime transcript file bridges
* refactor: remove transcript file handoffs
* refactor: remove MSTeams legacy learning key fallback
* refactor: store model catalog config in sqlite
* refactor: use sqlite model catalog at runtime
# Conflicts:
# docs/cli/secrets.md
# docs/gateway/authentication.md
# docs/gateway/secrets.md
* fix: keep oauth sibling sync sqlite-local
# Conflicts:
# src/commands/onboard-auth.test.ts
* refactor: remove task session store maintenance
# Conflicts:
# src/commands/tasks.ts
* refactor: keep diagnostics in state sqlite
* refactor: enforce database-first runtime state
* refactor: consolidate sqlite runtime state
* Show Codex subscription reset times in channel errors (#80456 )
* fix(codex): refresh subscription limit resets
* fix(codex): format reset times for channels
* Update CHANGELOG with latest changes and fixes
Updated CHANGELOG with recent fixes and improvements.
* fix(codex): keep command load failures on codex surface
* fix(codex): format account rate limits as rows
* fix(codex): summarize account limits as usage status
* fix(codex): simplify account limit status
* test: tighten subagent announce queue assertion
* test: tighten session delete lifecycle assertions
* test: tighten cron ops assertions
* fix: track cron execution milestones
* test: tighten hermes secret assertions
* test: assert matrix sync store payloads
* test: assert config compat warnings
* fix(codex): align btw side thread semantics
* fix(codex): honor codex fallback blocking
* fix(agents): avoid Pi resource discovery stalls
* test: tighten codex event assertions
* test: tighten cron assertions
* Fix Codex app-server OAuth harness auth
* refactor: move agent runtime state toward piless
* refactor: move device and push state to sqlite
* refactor: move runtime json state imports to doctor
* refactor: finish database-first state migration
* chore: refresh generated sqlite db types
* refactor: clarify cron sqlite store keys
* refactor: remove stale file-backed shims
* refactor: bind codex runtime state by session id
* test: expect sqlite trajectory branch export
* refactor: rename session row helpers
* fix: keep legacy device identity import in doctor
* refactor: enforce database-first runtime state
* refactor: consolidate sqlite runtime state
* build: align pi contract wrappers
* chore: repair database-first rebase
* refactor: remove session file test contracts
* test: update gateway session expectations
* refactor: stop routing from session compatibility shadows
* refactor: stop persisting session route shadows
* refactor: use typed delivery context in clients
* refactor: stop echoing session route shadows
* refactor: repair embedded runner rebase imports
# Conflicts:
# src/agents/pi-embedded-runner/run/attempt.tool-call-argument-repair.ts
* refactor: align pi contract imports
* refactor: satisfy kysely sync helper guard
* refactor: remove file transcript bridge remnants
* refactor: remove session locator compatibility
* refactor: remove session file test contracts
* refactor: keep rebase database-first clean
* refactor: remove session file assumptions from e2e
* docs: clarify database-first goal state
* test: remove legacy store markers from sqlite runtime tests
* refactor: remove legacy store assumptions from runtime seams
* refactor: align sqlite runtime helper seams
* test: update memory recall sqlite audit mock
* refactor: align database-first runtime type seams
* test: clarify doctor cron legacy store names
* fix: preserve sqlite session route projections
* test: fix copilot token cache test syntax
* docs: update database-first proof status
* test: align database-first test fixtures
* docs: update database-first proof status
* refactor: clean extension database-first drift
* test: align agent session route proof
* test: clarify doctor legacy path fixtures
* chore: clean database-first changed checks
* chore: repair database-first rebase markers
* build: allow baileys git subdependency
* chore: repair exp-vfs rebase drift
* chore: finish exp-vfs rebase cleanup
* chore: satisfy rebase lint drift
* chore: fix qqbot rebase type seam
* chore: fix rebase drift leftovers
* fix: keep auth profile oauth secrets out of sqlite
* fix: repair rebase drift tests
* test: stabilize pairing request ordering
* test: use source manifests in plugin contract checks
* fix: restore gateway session metadata after rebase
* fix: repair database-first rebase drift
* fix: clean up database-first rebase fallout
* test: stabilize line quick reply receipt time
* fix: repair extension rebase drift
* test: keep transcript redaction tests sqlite-backed
* fix: carry injected transcript redaction through sqlite
* chore: clean database branch rebase residue
* fix: repair database branch CI drift
* fix: repair database branch CI guard drift
* fix: stabilize oauth tls preflight test
* test: align database branch fast guards
* test: repair build artifact boundary guards
* chore: clean changelog rebase markers
---------
Co-authored-by: pashpashpash <nik@vault77.ai >
Co-authored-by: Eva <eva@100yen.org >
Co-authored-by: stainlu <stainlu@newtype-ai.org >
Co-authored-by: Jason Zhou <jason.zhou.design@gmail.com >
Co-authored-by: Ruben Cuevas <hi@rubencu.com >
Co-authored-by: Pavan Kumar Gondhi <pavangondhi@gmail.com >
Co-authored-by: Shakker <shakkerdroid@gmail.com >
Co-authored-by: Kaspre <36520309+Kaspre@users.noreply.github.com >
Co-authored-by: dataCenter430 <titan032000@gmail.com >
Co-authored-by: Kaspre <kaspre@gmail.com >
Co-authored-by: pandadev66 <nova.full.stack@outlook.com >
Co-authored-by: Eva <admin@100yen.org >
Co-authored-by: Eva (agent) <eva+agent-78055@100yen.org >
Co-authored-by: Josh Lehman <josh@martian.engineering >
Co-authored-by: jeffjhunter <support@aipersonamethod.com >
2026-05-13 13:15:12 +01:00
dce5f809a1
test: tighten proxy capture store assertions
2026-05-09 12:08:46 +01:00
ea2799389a
test: clarify proxy tui cli accepted paths
2026-05-08 12:47:48 +01:00
1323683d72
fix: stabilize qa lab capture store cleanup
2026-04-26 09:13:30 +01:00
958c34e82c
feat(qa-lab): Add proxy capture stack and QA Lab inspector ( #64895 )
...
* Add proxy capture core and CLI
* Expand transport capture coverage
* Add QA Lab capture backend
* Refine QA Lab capture UI
* Fix proxy capture review feedback
* Fix proxy run cleanup and TTS capture
* Fix proxy capture transport follow-ups
* Fix debug proxy CONNECT target parsing
* Harden QA Lab asset path containment
2026-04-11 12:34:57 -05:00
Miorbnli