mirror of
https://github.com/openclaw/openclaw.git
synced 2026-08-12 21:53:00 -06:00
refactor(sessions): remove file-era transcript runtime (#113233)
* refactor(sessions): keep helper transcripts in memory * refactor(sessions): remove file-era transcript storage * test(sessions): use SQLite identity in attempt persistence * test(codex): isolate legacy transcript fixtures * fix(sessions): preserve SQLite transcript identity * fix(sessions): harden transcript lifecycle invariants * fix(sessions): validate transcript identities * fix(sessions): close identity compatibility gaps * fix(sessions): preserve leaf and plugin identities * fix(sessions): retain dispatch transcript targets * fix(sessions): preserve active transcript context * fix(sessions): isolate artifact accounting * fix(sessions): bound SQLite usage accounting * fix(sessions): retain bounded latest usage * fix(sessions): align rebased transcript targets * test(sessions): align accessor scope fixture * fix(telegram): derive SQLite transcript identity * refactor(sessions): remove file-era compaction residue * chore(sessions): lower max-lines baseline * fix(sessions): preserve structured transcript identity * test(sessions): align doctor identity assertions * fix(sessions): isolate default SDK database * refactor(sessions): remove dead file-era exports * fix(sessions): reconcile SQLite transcript identity * fix(sessions): pass checkpoint identity explicitly * test(sessions): make entry field probe explicit * test(sessions): satisfy transcript cleanup lint * test(sessions): align diagnostics identity proof * fix(sessions): finish transcript runtime teardown * fix(sessions): preserve transcript identity invariants * fix(sessions): harden transcript compatibility edges * fix(sessions): preserve checkpoint transcript anchors * fix(sessions): preserve SQLite lifecycle invariants * fix(sessions): retarget compaction successors * test(sessions): preserve transcript fixture semantics * feat(plugin-sdk): add command transcript targets * fix(sessions): serialize transcript rewrites * fix(sessions): validate legacy successor identity * fix(sessions): normalize compaction ownership * fix(sessions): validate successor identity before adoption * fix(sessions): preserve plugin transcript ownership * fix(sessions): carry transcript identity through commands * fix(sessions): import legacy checkpoint artifacts into SQLite * fix(sessions): preserve successor transcript ownership * fix(sessions): align transcript consumers with target identity * fix(sessions): scope transcript token estimates * fix(sessions): retain agent identity across lifecycle hooks * fix(sessions): resolve scoped SQLite targets * fix(sessions): isolate lifecycle transcript targets * fix(sessions): validate compaction agent ownership * fix(sessions): preserve reset and cleanup lifecycle * fix(sessions): serialize prompt cleanup lifecycle * fix(sessions): remove stale lock import * fix(sessions): preserve reset target context * fix(sessions): fence prompt reload takeover * fix(sessions): unblock abort and default lifecycle reads * fix(sessions): validate legacy successor scope * fix(sessions): reject metadata-only runtime rows * fix(sessions): propagate custom transcript stores * fix(sessions): preserve adopted retry targets * fix(sessions): allow unkeyed usage reads * fix(sessions): harden runtime target boundaries * fix(sessions): serialize retry transcript writes * fix(sessions): bound prompt reload disposal * fix(sessions): complete retry marker identity * fix(sessions): keep legacy marker identity minimal * test(sessions): tighten teardown fixture types * fix(sessions): preserve compatibility target identity * test(sessions): persist post-checkpoint boundary turn * test(sessions): align runtime store mock contracts * style(sessions): simplify persisted identity guard * fix(sessions): prefer complete typed targets * fix(sessions): recover legacy marker targets * test(sessions): align marker lookup fixture scope * fix(sessions): validate partial transcript targets * fix(sessions): reconcile partial transcript identities * fix(sessions): canonicalize compatibility identities * test(sessions): cover compatibility aliases * fix(sessions): adopt legacy successor identity * fix(sessions): preserve usage read identity * fix(sessions): preserve partial marker compatibility * fix(sessions): validate legacy successor mappings * fix(sessions): reconcile marker store mappings * fix(sessions): preserve legacy fallback identity * fix(sessions): harden marker alias resolution * fix(sessions): prefer verified successor aliases * fix(sessions): resolve preferred marker aliases * fix(sessions): serialize cleanup admission * fix(sessions): align marker lookup scopes * fix(codex): type marker alias summaries * style(sessions): satisfy changed lint * test(sessions): align structured target assertions * fix(sessions): reconcile latest identity contracts * fix(sessions): validate transcript identity boundaries * docs(sessions): explain stable registry keys * fix(sessions): harden compatibility target round trips * fix(sessions): port usage identity to split modules * test(sessions): align subagent transcript identity * fix(sessions): finish transcript identity migration * fix(agents): route subagent completion capture through transcript targets * fix(agents): settle SQLite prompt handoff during cleanup * chore: shrink max-lines baseline after teardown * fix(sessions): port teardown across split runtime owners * fix(sessions): carry transcript targets through split owners * test(agents): use SQLite compaction target in abort coverage * chore: retain unrelated max-lines suppressions * chore: shrink max-lines baseline after main splits * style(agents): const compaction checkpoint locals * fix(sessions): harden SQLite teardown boundaries * test(sessions): use typed metadata in predicate isolation fixture * test(agents): cover malformed settlement rejections lint-safely * fix(sessions): close remaining SQLite identity races * fix(agents): fail closed on incomplete successor targets * fix(sessions): preserve transcript identity fallbacks * fix(agents): preserve session-key abort admission * fix(trajectory): validate incomplete export targets * test(sessions): drop retired pricing cache imports * fix(sessions): validate partial transcript identities * fix(sessions): close transcript identity edge cases * fix(plugins): reserve retired transcript locator slot * fix(sessions): scope transcript locks by target * style(sessions): simplify SDK initialization error * fix(sessions): preserve initialized transcript state * fix(codex): verify mirrored history session keys * fix(sessions): reject stale transcript ownership * fix(sessions): anchor asynchronous transcript ownership * fix(sessions): measure active transcript state * fix(sessions): preserve scoped transcript compaction * fix(sessions): harden transcript identity and lifecycle * fix(sessions): resolve scoped command transcript stores * fix(sessions): make transcript appends failure-atomic * fix(sessions): enforce scoped transcript ownership * fix(sessions): reject cross-owner transcript handoffs * fix(sessions): fence cleanup transcript ownership * fix(sessions): retire stale write ownership contexts * fix(sessions): preserve pending session migration state * fix(sessions): validate migrated transcript ownership * fix(sessions): validate usage transcript targets * fix(sessions): clear predecessor transcript metadata * fix(sessions): align durable session event targets * fix(sessions): fence late prompt handoffs * fix(sessions): fence lifecycle transcript fallbacks * fix(sessions): bound zero-length memory capture * fix(sessions): preserve transcript teardown ownership * fix(sessions): reject duplicate cleanup ownership * fix(sessions): serialize runtime writes with sqlite leases * fix(sessions): close sqlite teardown concurrency gaps * fix(sessions): preserve nested lifecycle failures * fix(sessions): canonicalize sqlite transcript ownership * fix(sessions): settle disposed prompt handoffs * fix(sessions): resolve canonical attempt lock targets * test(sessions): align canonical target fixtures * test(sessions): retire redundant jsonl parser coverage * refactor(sessions): split active transcript cursors * test(memory): retire legacy marker fixture * fix(sessions): preserve canonical transcript access after rebase * fix(sessions): fence prompt lease and return transcript targets * fix(sessions): colocate transcript leases with target store * fix(sessions): canonicalize transcript lease and worker targets * fix(sessions): preserve plugin and fork identity markers * fix(sessions): complete sqlite transcript target migration * fix(sessions): integrate canonical followup identity * fix(sessions): preserve bounded transcript topology * fix(sessions): validate transcript identity boundaries * fix(context): separate caller and successor targets * test(sessions): split persistence compatibility coverage * test(sessions): preserve fixture topology efficiently * chore(sdk): refresh plugin api baseline * test(agents): align compaction lock target mocks * test(sessions): seed malformed transcript fixtures directly * fix(agents): canonicalize transcript compatibility inputs * fix(agents): type optional tool result ids * test(ci): stabilize loaded process timing * test(tui): wait for collect queue admission
This commit is contained in:
Peter Steinberger
committed by
GitHub
GitHub
parent
6559e3f3b9
commit
4273ca9dbd
@@ -7,7 +7,7 @@ read_when:
|
||||
title: "Session management deep dive"
|
||||
---
|
||||
|
||||
A single **Gateway process** owns session state end-to-end. UIs (macOS app, web Control UI, TUI) query the Gateway for session lists and token counts. In remote mode, session files live on the remote host, so checking your local Mac's files will not reflect what the Gateway is using.
|
||||
A single **Gateway process** owns session state end-to-end. UIs (macOS app, web Control UI, TUI) query the Gateway for session lists and token counts. In remote mode, the per-agent SQLite database lives on the remote host, so checking your local Mac's state will not reflect what the Gateway is using.
|
||||
|
||||
Overview docs first: [Session management](/concepts/session), [Compaction](/concepts/compaction), [Memory overview](/concepts/memory), [Memory search](/concepts/memory-search), [Session pruning](/concepts/session-pruning), [Transcript hygiene](/reference/transcript-hygiene), full config reference at [Agent config](/gateway/config-agents).
|
||||
|
||||
@@ -217,7 +217,7 @@ Two triggers in the embedded OpenClaw agent:
|
||||
|
||||
Two additional guards run outside these two triggers:
|
||||
|
||||
- **Preflight local compaction**: set `agents.defaults.compaction.maxActiveTranscriptBytes` (bytes or a string like `"20mb"`) to trigger local compaction before opening the next run once the active transcript reaches that size. This is a size guard for local reopen cost, not raw archival - normal semantic compaction still runs, and it requires `truncateAfterCompaction` so the compacted summary becomes a new successor transcript.
|
||||
- **Preflight local compaction**: set `agents.defaults.compaction.maxActiveTranscriptBytes` (bytes or a string like `"20mb"`) to trigger local compaction before opening the next run once the active transcript reaches that size. Normal semantic compaction still runs. Durable size reduction requires `truncateAfterCompaction` plus a context engine that returns an explicit successor identity.
|
||||
- **Mid-turn precheck**: set `agents.defaults.compaction.midTurnPrecheck.enabled: true` (default `false`) to add a tool-loop guard. After a tool result is appended and before the next model call, OpenClaw estimates prompt pressure using the same preflight budget logic used at turn start. If context no longer fits, the guard does not compact inline - it raises a structured mid-turn precheck signal, stops the current prompt submission, and lets the outer run loop use the existing recovery path (truncate oversized tool results when that is enough, or trigger the configured compaction mode and retry). Works with both `default` and `safeguard` compaction modes, including provider-backed safeguard compaction. Independent of `maxActiveTranscriptBytes`: the byte-size guard runs before a turn opens, mid-turn precheck runs later, after new tool results are appended.
|
||||
|
||||
## Compaction settings
|
||||
@@ -239,9 +239,9 @@ OpenClaw enforces a built-in reserve for embedded runs and caps it against the a
|
||||
|
||||
Set `enabled: false` to disable threshold-driven auto-compaction inside the embedded agent runtime. OpenClaw's preflight and overflow-recovery compaction paths remain available, and manual `/compact` continues to work.
|
||||
|
||||
Manual `/compact` honors an explicit `agents.defaults.compaction.keepRecentTokens` and keeps the runtime's recent-tail cut point. Without an explicit keep budget, manual compaction is a hard checkpoint and rebuilt context starts from the new summary.
|
||||
Manual `/compact` uses `agents.defaults.compaction.keepRecentTokens` (default: `20000`) and keeps that recent-tail cut point.
|
||||
|
||||
When `truncateAfterCompaction` is enabled, OpenClaw rotates the active transcript to a compacted successor after compaction. Branch/restore checkpoint actions use that compacted successor; legacy pre-compaction checkpoint files remain readable while referenced.
|
||||
When `truncateAfterCompaction` is enabled, OpenClaw adopts an explicit successor identity returned by a context engine. The built-in SQLite compactor keeps the current session identity. Branch/restore checkpoint actions use a returned successor when present; legacy pre-compaction checkpoint files remain readable while referenced.
|
||||
|
||||
## Pluggable compaction providers
|
||||
|
||||
@@ -278,12 +278,12 @@ Before auto-compaction happens, OpenClaw can run a silent agentic turn that writ
|
||||
|
||||
Config (`agents.defaults.compaction.memoryFlush`), full reference at [/gateway/config-agents](/gateway/config-agents#agentsdefaultscompaction):
|
||||
|
||||
| Key | Default | Notes |
|
||||
| --------------------------- | ---------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `enabled` | `true` | |
|
||||
| `model` | unset | exact provider/model override for the flush turn only, for example `ollama/qwen3:8b` |
|
||||
| `softThresholdTokens` | `4000` | gap below the compaction threshold that triggers a flush |
|
||||
| `forceFlushTranscriptBytes` | unset (disabled) | force a flush once the transcript file reaches this byte size (or string like `"2mb"`), even if token counters are stale; `0` disables |
|
||||
| Key | Default | Notes |
|
||||
| --------------------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `enabled` | `true` | |
|
||||
| `model` | unset | exact provider/model override for the flush turn only, for example `ollama/qwen3:8b` |
|
||||
| `softThresholdTokens` | `4000` | gap below the compaction threshold that triggers a flush |
|
||||
| `forceFlushTranscriptBytes` | unset (disabled) | force a flush once active transcript history reaches this estimated byte size (or string like `"2mb"`), even if token counters are stale; `0` disables |
|
||||
|
||||
Notes:
|
||||
|
||||
|
||||
@@ -159,7 +159,7 @@ Other behavior: the runner preflights Docker by default, cleans stale OpenClaw E
|
||||
| `pnpm test:docker:live-cli-backend:claude`, `:claude:resume`, `:claude:mcp` | Focused CLI backend live probes; Gemini has matching `:resume` and `:mcp` aliases. |
|
||||
| `pnpm test:docker:openwebui` | Dockerized OpenClaw + Open WebUI: sign in, check `/api/models`, run a real proxied chat through `/api/chat/completions`. Requires a usable live model key and pulls an external image; not expected to be CI-stable like the unit/e2e suites. |
|
||||
| `pnpm test:docker:mcp-channels` | Seeded Gateway container plus a client container spawning `openclaw mcp serve`: routed conversation discovery, transcript reads, attachment metadata, live event queue behavior, outbound send routing, and Claude-style channel + permission notifications over the real stdio bridge (assertion reads raw stdio MCP frames directly). |
|
||||
| `pnpm test:docker:upgrade-survivor` | Installs the packed tarball over a dirty old-user fixture, runs package update plus non-interactive doctor without live provider/channel keys, starts a loopback Gateway, checks agents/channel config/plugin allowlists/workspace/session files/stale legacy plugin dependency state/startup/RPC status survive. |
|
||||
| `pnpm test:docker:upgrade-survivor` | Installs the packed tarball over a dirty old-user fixture, runs package update plus non-interactive doctor without live provider/channel keys, starts a loopback Gateway, checks agents/channel config/plugin allowlists/workspace/session state/stale legacy plugin dependency state/startup/RPC status survive. |
|
||||
| `pnpm test:docker:published-upgrade-survivor` | Installs `openclaw@latest` by default, seeds realistic existing-user files, configures via a baked `openclaw config set` recipe, updates to the packed tarball, runs non-interactive doctor, writes `.artifacts/upgrade-survivor/summary.json`, checks `/healthz`, `/readyz`, RPC status. Override with `OPENCLAW_UPGRADE_SURVIVOR_BASELINE_SPEC`, expand a matrix with `OPENCLAW_UPGRADE_SURVIVOR_BASELINE_SPECS`, or add scenario fixtures with `OPENCLAW_UPGRADE_SURVIVOR_SCENARIOS=reported-issues` (includes `configured-plugin-installs` and `stale-source-plugin-shadow`). Package Acceptance exposes these as `published_upgrade_survivor_baseline(s)` / `_scenarios` and resolves meta tokens like `last-stable-4` or `all-since-2026.4.23`. |
|
||||
| `pnpm test:docker:update-migration` | Published-upgrade survivor harness in the `plugin-deps-cleanup` scenario, starting at `openclaw@2026.4.23` by default. The `Update Migration` workflow expands this with `baselines=all-since-2026.4.23` to prove configured-plugin dependency cleanup outside Full Release CI. |
|
||||
| `pnpm test:docker:plugins` | Install/update smoke for local path, `file:`, npm registry packages with hoisted dependencies, git moving refs, ClawHub fixtures, marketplace updates, and Claude-bundle enable/inspect. |
|
||||
@@ -270,7 +270,7 @@ Optional; only needed for containerized onboarding smoke tests. Full cold-start
|
||||
scripts/e2e/onboard-docker.sh
|
||||
```
|
||||
|
||||
Drives the interactive wizard via a pseudo-tty, verifies config/workspace/session files, then starts the gateway and runs `openclaw health`.
|
||||
Drives the interactive wizard via a pseudo-tty, verifies config/workspace/session state, then starts the gateway and runs `openclaw health`.
|
||||
|
||||
## QR import smoke (Docker)
|
||||
|
||||
|
||||
@@ -8,19 +8,12 @@ title: "Transcript hygiene"
|
||||
---
|
||||
|
||||
OpenClaw applies **provider-specific fixes** to transcripts before a run
|
||||
(building model context). Most of these are **in-memory** adjustments used to
|
||||
satisfy strict provider requirements. A separate session-file repair pass may
|
||||
also rewrite stored JSONL before the session is loaded, but only for
|
||||
malformed lines or persisted turns that are invalid durable records.
|
||||
Delivered assistant replies are preserved on disk; provider-specific
|
||||
(building model context). These are **in-memory** adjustments used to satisfy
|
||||
strict provider requirements. Runtime transcript state stays in SQLite;
|
||||
provider-specific
|
||||
assistant-prefill stripping happens only while constructing outbound
|
||||
payloads.
|
||||
|
||||
When a repair occurs, the original file is written to a transient
|
||||
`*.bak-<pid>-<ts>` sibling before the atomic replace, then removed once the
|
||||
replace succeeds. The backup is retained only if cleanup itself fails, in
|
||||
which case the path is reported back.
|
||||
|
||||
Scope includes:
|
||||
|
||||
- Runtime-only prompt context staying out of user-visible transcript turns
|
||||
@@ -64,12 +57,8 @@ All transcript hygiene is centralized in the embedded runner:
|
||||
- Sanitization/repair application: `sanitizeSessionHistory` in
|
||||
`src/agents/embedded-agent-runner/replay-history.ts`
|
||||
|
||||
Separate from transcript hygiene, session files are repaired (if needed)
|
||||
before load:
|
||||
|
||||
- `repairSessionFileIfNeeded` in `src/agents/session-file-repair.ts`
|
||||
- Called from `src/agents/embedded-agent-runner/run/attempt.ts` and
|
||||
`src/agents/embedded-agent-runner/compact.ts`
|
||||
Legacy JSONL validation and import belong to `openclaw doctor --fix`; the
|
||||
embedded runner does not repair or reopen file-backed runtime transcripts.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user