* feat(ui): show session-reset dividers and fix boundary markers on DB-backed sessions
/reset now leaves a durable 'Session reset' divider at the transcript
boundary in the Control UI. Root-cause fix underneath: the SQLite
transcript projection only selected message events, so compaction (and
now reset) markers never reached clients for DB-backed sessions; marker
synthesis now has one owner (session-transcript-message.ts) consumed by
both storage backends across full/recent/paged/by-id/anchor reads.
Additive __openclaw marker kind 'reset' documented in clients.md.
* fix(gateway): keep history readers out of the plugin SDK barrel and fix CI gates
Direct imports for the sqlite history readers (the session-accessor barrel
is SDK-reachable via session-transcript-lock-runtime); reset marker added
to the kept-tail chat.history expectation; lint naming fixes; marker tests
split into session-transcript-readers.markers.test.ts.
* fix(sessions): complete tail-read windows despite short positional reads
Replace single-shot handle.read() calls in two session tail-read
paths with readFileWindowFully so multibyte-range positional reads
do not silently return incomplete tail data on network filesystems.
- readRecentTranscriptTailLinesAsync: tail-window read for recent
session messages now loops until the requested window fills
- readLastMessagePreviewFromOpenTranscriptAsync: last-message
preview tail read now loops until the requested window fills
The same readFileWindowFully helper was introduced in #108253 and
expanded with a sync variant in #108127 (both by sunlit-deng).
* fix(sessions): complete sync title preview reads
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* fix(core): make indexed access explicit in auto-reply, infra, and config
Part 1/3 of the src NUIA phase-3b burn-down (#104600): iteration and
destructuring over index reads, boundary guards on parsed input, and
named invariants. Config path walkers bind the path head once; SQLite
migration key handling is hoisted without query-shape changes.
* fix(core): make indexed access explicit in cli, gateway, commands, security, shared
Part 2/3: argv/token selection restructured, gateway event/attachment
invariants named, security parsers stay fail-closed (invariant
violations throw), edit-distance matrices access checked entries.
* fix(core): make indexed access explicit across remaining src surfaces
Part 3/3: channels, plugins, process, cron, plugin-sdk, media, logging,
tui, hooks, daemon, and small directories. Latent bug fixed: a tailnet
resolver could leak undefined through a string|null contract and now
fails with a descriptive local error.
* fix(core): keep optional boundaries optional after per-commit review
Review findings: expectDefined misused where absence is a legitimate
state. CLI --profile/route-args missing next tokens take their existing
miss paths; help normalization compares --help against the last
positional again; first-time plugin install spreads absent cfg.plugins;
denylist scan iterates manifest dependency entries instead of throwing
on omitted sections; tailnet resolver returns a guaranteed string at
the source instead of a caller-side undefined throw.
* refactor(core): closed-key provider labels and honest optional passthroughs
PROVIDER_LABELS becomes a satisfies-typed closed record (static reads
provably defined; dynamic lookups go through providerUsageLabel with
honest string|undefined). Status-scan overview passes its optional
params through unchanged instead of asserting them.
* fix(channels): make getChatChannelMeta honestly optional
The original signature claimed ChatChannelMeta while leaking undefined
on bundled channel id metadata drift; three of four callers already
handled absence. The return type now says so, and the one assuming
caller falls back to the raw channel label.
* fix(core): index-safety for post-rebase main drift
Covers the sqlite-sessions flip and auth-source-plan code that landed
mid-phase, plus the channel-validation test consuming the now honestly
optional getChatChannelMeta.
* refactor(channels): split chat-meta accessors along the SDK contract
getChatChannelMeta keeps its shipped plugin-SDK signature (defined for
bundled ids, fail-loud on impossible misses); new findChatChannelMeta
carries the drift-tolerant optional contract for core auto-enable and
formatting paths.
* fix(qa-channel): own channel metadata instead of a guaranteed-undefined catalog lookup
qa-channel spread getChatChannelMeta over an id that is never in the
bundled catalog, shipping an empty setup meta by accident; the fail-loud
SDK accessor exposed it. The channel now declares its metadata once.
* fix(gateway): heartbeat projection lookahead is optional at the transcript tail
expectDefined wrapped messages[i + 1] whose absence on the final message
is the normal case; the adjacent ternary already handled it. Restores
the plain optional read with an explicit guard in the pair condition.
* fix(plugin-sdk): channel plugin factory tolerates non-bundled channel ids again
createChannelPluginBase spreads bundled catalog meta for ANY channel id,
where absence is the normal case for external plugins; the resolver is
honestly optional again while the exported bundled-id accessor keeps the
fail-loud contract.
* fix(core): spreads of optional config sections stay optional
Fresh-setup and first-install paths (crestodian setup inference, hook
installs, agent config base, target agent models) legitimately lack the
section being rebuilt; spreading undefined is the shipped {} semantics.
Removes the remaining gratuitous assertion wraps found by tree audit.
* fix(sessions): read zstd transcript archives through a materialized cache
Compressed archives were discovered but unreadable: the reset-archive
header probe read raw zstd bytes and every JSONL reader parsed them as
malformed lines, so compressed archives silently contributed zero
history and zero usage. Archives are write-once, so they materialize
once into a plain JSONL cache under the OpenClaw tmp dir and every
downstream reader (index, tail chunks, probes, usage scans) works
unchanged; the usage scanner also decompresses directly for whole-file
iteration.
* fix(sessions): keep usage byte-range semantics for compressed archives
Route zstd archive usage reads through the materialized plain-JSONL
cache so persisted incremental-scan offsets always measure decompressed
bytes; the whole-file decompress branch ignored requested ranges and
could overcount archived usage.
* fix(sessions): normalize compressed archives to decompressed space at usage discovery
Sizes, incremental offsets, and cache signatures for zstd archives now
all measure the materialized plain-JSONL cache, closing the truncation
window when persisted offsets from the compressed path met decompressed
reads.
* fix(sessions): gate the archive read cache on source identity
Every materialized-cache hit now stats the source archive: a deleted or
budget-evicted archive scrubs its cache entries instead of serving
stale plaintext, and the cache name carries the source size so a
same-path rewrite can never alias.
* fix(sessions): include source mtime in the archive cache identity
Same-path same-size rewrites can no longer alias a stale plaintext
cache entry; any source change mints a new cache name and the previous
entries are scrubbed on the next materialization.
* fix(sessions): bound and isolate the materialized archive cache
Cache entries expire on a 24h TTL sweep so budget-evicted archives
cannot leave plaintext copies behind, stale-identity scrubbing skips
the live identity and in-flight temp files, and unique temp names plus
last-rename-wins make concurrent materialization race-free.
* fix(sessions): drop useless probe-path initializer flagged by lint
* refactor(sessions): migrate runtime storage to sqlite
* test(sessions): fix sqlite CI regressions
* test(sessions): align remaining sqlite fixtures
* fix(codex): require sqlite trajectory recorder
* test(sessions): align orphan recovery sqlite fixture
* test(sessions): align sqlite rebase fixtures
* fix(sessions): finish current-main integration of the sqlite flip
Resolve the whole-store SDK removal across its owner boundary: drop the
loadSessionStore re-export and the registry whole-store wrappers, wire
hasTrackedActiveSessionRun into gateway chat, complete the
preserveLockedHarnessIds cleanup contract, flip the codex thread-history
import to storePath targets, and port remaining main-side tests from
file-store helpers to session accessor reads.
* chore: drop committed pebbles log, revert plugin-inspector bump, refresh generated docs
Remove the 1.8k-line .pebbles/events.jsonl work log from the branch, restore
the plugin-inspector advisory lane to main's pinned 0.3.10 so the supply-chain
bump gets its own review, and regenerate docs_map, the plugin SDK API baseline,
and the export-surface ratchet for the merged tree.
* feat(sessions): keep archived transcripts by default with zstd cold storage
Codex-style retention: deleting or resetting a session archives its
transcript as a zstd-compressed JSONL artifact (plain when the runtime
lacks node:zlib zstd) and keeps it until the disk budget evicts oldest
first. resetArchiveRetention now governs both deleted and reset archives
and defaults to keep; maxDiskBytes defaults to 2gb so retention stays
bounded, with archives evicted before live sessions. The cron reaper
follows the same knob instead of deleting archives on its own timer.
* fix(state): converge agent DB migration lineages and bound database growth
Merge coherence: run both structure-gated legacy memory-schema repairs
(flip-lineage drop, main-lineage identity rebuild) before the flip
migration so pre-flip v1/v2 and pre-merge flip v1/v4 databases all
converge, and hoist foreign_keys=OFF outside the schema transaction
where the pragma was silently ignored and the v1 sessions rebuild
cascade-deleted session_entries.
Growth guards: fresh agent DBs enable auto_vacuum=INCREMENTAL, WAL
maintenance releases freed pages in bounded passes (never a blocking
full VACUUM), and doctor reports state/agent DB bloat from freelist
stats.
* fix(codex): resolve the store path for thread-history import via the SDK
The supervision catalog passed the legacy sessionFile locator to the
storePath-targeted transcript mirror; resolve the agent store path with
the session-store SDK helper instead of a runtime-object seam so test
fakes and headless callers need no extra surface. Drop the obsolete
missing-session-id preprocessing case: sessions rows are NOT NULL on
session_id and upsert repairs id-less patches at write time.
* fix(sessions): fail safe on malformed disk-budget config and doctor stat errors
A malformed explicit maxDiskBytes disables the budget instead of
falling back to the destructive 2gb default the user never chose, and
the doctor bloat check skips databases whose paths stat-fail instead of
aborting doctor.
* fix(sessions): complete sqlite conflict translations
* test(sqlite): align hardening checks with maintenance
* test(sessions): inspect compressed transcript archives
* fix(tests): await session seeds and drop unused helpers flagged by CI lint
The five unawaited writeSessionStoreSeed calls raced their SQLite seeds
against the assertions, failing compact shards; the bloat probe drops a
useless initializer and the merged tests drop now-unused helpers.
* test(sessions): type legacy proof events directly
* test(sessions): align hardening contracts
* perf(sessions): read usage transcript sizes from SQL aggregates
Usage/cost scans walked every session and materialized every transcript
event just to re-stringify it for a byte estimate — the #86718 stall
class reborn on the DB. readTranscriptStatsSync sums stored JSON bytes
in SQLite without loading a single row.
* fix(sessions): re-root foreign-root transcript paths onto the current sessions dir
Restored backups, moved OPENCLAW_STATE_DIR, and rehearsal copies carry
absolute sessionFile paths from the old root; the containment fallback
kept those foreign paths, so migration read (and would archive) files in
the original root and reported local copies missing. Re-root the
canonical agents/<id>/sessions suffix onto the current dir when the file
exists there; genuine cross-root layouts still fall through unchanged.
* test(agents): seed harness admission through sqlite
* fix(sqlite): close agent db on pragma setup failure
* fix(doctor): compact and retrofit incremental auto-vacuum after session import
The migration is the sanctioned offline window: post-import compact
reclaims import churn and applies auto_vacuum=INCREMENTAL to databases
created before the fresh-DB pragma existed, so runtime maintenance can
release pages in bounded passes on every install.
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* fix(gateway): keep session title and preview text truncation UTF-16 safe
* test(gateway): cover session UTF-16 boundaries in place
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Preserve concurrent prompt-time transcript updates across stale session managers, side appends, transcript navigation, nested owned writes, and doctor repair.
Fixes#93193.
Thanks @snowzlm for the report and original fix.
Co-authored-by: snowzlm <snowzlm@noreply.codeberg.org>
Fall back to valid reset transcript archives when active async session transcripts are missing, while keeping active transcript priority and choosing the newest valid archive across roots.
Validation:
- node scripts/run-vitest.mjs src/gateway/session-utils.fs.test.ts src/gateway/sessions-history-http.test.ts src/gateway/sessions-history-http.revocation.test.ts src/gateway/session-history-state.test.ts src/gateway/server.chat.gateway-server-chat-b.test.ts src/gateway/managed-image-attachments.test.ts src/agents/tools/embedded-gateway-stub.test.ts src/tui/embedded-backend.test.ts
- node scripts/run-tsgo.mjs -p test/tsconfig/tsconfig.test.src.json --incremental --tsBuildInfoFile .artifacts/tsgo-cache/test-src-pr92879.tsbuildinfo
- git diff --check origin/main...HEAD && git diff --check
- autoreview --mode branch --base origin/main: clean
Direct-landed from #92879 because the source branch has maintainer edits disabled and the landed diff needed maintainer repair before merge.
Co-authored-by: Masato Hoshino <246810661+masatohoshino@users.noreply.github.com>
Co-authored-by: Hu Yitao <39733381+CadanHu@users.noreply.github.com>
Co-authored-by: Vincent Koc <25068+vincentkoc@users.noreply.github.com>
Add a bounded `chat.message.get` gateway method so Control UI can fetch one display-normalized transcript message by id when an assistant history preview was truncated. Keep `chat.history` lightweight, reject oversized/hidden/missing rows with explicit unavailable reasons, and wire the WebChat side reader to request full content only for visible truncated assistant messages.
Also refresh the generated Swift gateway protocol models and document the new assistant-message side-reader behavior.
Closes#84651.
Related #53242.
Co-authored-by: NianJiuZst <3235467914@qq.com>
Extract shared normalization/coercion helpers into private @openclaw/normalization-core workspace package while preserving existing plugin SDK helper subpaths.\n\nAlso keeps direct normalization-core imports internal, wires UI/build/loader resolution, and replaces the slow PR network CodeQL lane with a fast added-line boundary scan while retaining full CodeQL for scheduled/manual runs.\n\nVerification: local moved tests, plugin SDK boundary tests, extension loader tests, agents-support shard, UI build/test, build artifacts, lint, workflow guards, autoreview, and GitHub CI passed on PR head 963d893715.
Fix stale `subagent_announce` history hydration after `/new` by filtering pre-session-start announce/user reply pairs before `chat.history` projection.
Maintainer fixups added:
- require the adjacent assistant reply to carry a pre-session timestamp before dropping it
- preserve record timestamps for oversized transcript placeholders
- run the filter after Claude CLI history import and support imported timestamp/text fallback
- overread one local transcript message only as boundary context so limit-window edges do not leak stale assistant replies
Verification:
- `git diff --check`
- `node scripts/run-vitest.mjs src/gateway/server-methods/server-methods.test.ts src/gateway/session-utils.fs.test.ts src/gateway/session-history-state.test.ts src/gateway/cli-session-history.test.ts src/gateway/server.chat.gateway-server-chat-b.test.ts` -> 11 files, 463 tests passed
- `/Users/steipete/Projects/agent-scripts/skills/autoreview/scripts/autoreview --mode branch --base origin/main` -> clean, no accepted/actionable findings
Thanks @openperf.
extractJsonStringFieldPrefix and extractJsonNullableStringFieldPrefix
interpolate the `field` parameter into `new RegExp(...)` without
escaping. All current callers pass hardcoded strings ("id",
"parentId", "type", "role"), but the function signature accepts
any string. A future caller passing a field containing regex
metacharacters (e.g. "foo.bar") would match unintended patterns.
Wrap the interpolation with escapeRegExp() from src/shared/regexp.ts
so metacharacters are treated literally.
Signed-off-by: Sebastien Tardif <sebtardif@ncf.ca>
Fixes #73990.\n\nAdds a transcript-derived token estimate for local/OpenAI-compatible session transcripts that have real content but no provider usage telemetry, preserving provider-reported usage when available and gating estimation on assistant model identity.\n\nVerification:\n- CI run 25965717279: success\n- Real behavior proof run 25965716561: success\n- Azure Crabbox clean-clone proof: pnpm test src/gateway/session-utils.fs.test.ts src/status/status-message.test.ts; pnpm check:changed; pnpm exec tsx /tmp/openclaw-transcript-proof.mts; git diff --check origin/main...HEAD
Peter Steinberger