0a8e3604ba
refactor: flip sessions and transcripts to sqlite storage ( #98236 )
...
* refactor(sessions): migrate runtime storage to sqlite
* test(sessions): fix sqlite CI regressions
* test(sessions): align remaining sqlite fixtures
* fix(codex): require sqlite trajectory recorder
* test(sessions): align orphan recovery sqlite fixture
* test(sessions): align sqlite rebase fixtures
* fix(sessions): finish current-main integration of the sqlite flip
Resolve the whole-store SDK removal across its owner boundary: drop the
loadSessionStore re-export and the registry whole-store wrappers, wire
hasTrackedActiveSessionRun into gateway chat, complete the
preserveLockedHarnessIds cleanup contract, flip the codex thread-history
import to storePath targets, and port remaining main-side tests from
file-store helpers to session accessor reads.
* chore: drop committed pebbles log, revert plugin-inspector bump, refresh generated docs
Remove the 1.8k-line .pebbles/events.jsonl work log from the branch, restore
the plugin-inspector advisory lane to main's pinned 0.3.10 so the supply-chain
bump gets its own review, and regenerate docs_map, the plugin SDK API baseline,
and the export-surface ratchet for the merged tree.
* feat(sessions): keep archived transcripts by default with zstd cold storage
Codex-style retention: deleting or resetting a session archives its
transcript as a zstd-compressed JSONL artifact (plain when the runtime
lacks node:zlib zstd) and keeps it until the disk budget evicts oldest
first. resetArchiveRetention now governs both deleted and reset archives
and defaults to keep; maxDiskBytes defaults to 2gb so retention stays
bounded, with archives evicted before live sessions. The cron reaper
follows the same knob instead of deleting archives on its own timer.
* fix(state): converge agent DB migration lineages and bound database growth
Merge coherence: run both structure-gated legacy memory-schema repairs
(flip-lineage drop, main-lineage identity rebuild) before the flip
migration so pre-flip v1/v2 and pre-merge flip v1/v4 databases all
converge, and hoist foreign_keys=OFF outside the schema transaction
where the pragma was silently ignored and the v1 sessions rebuild
cascade-deleted session_entries.
Growth guards: fresh agent DBs enable auto_vacuum=INCREMENTAL, WAL
maintenance releases freed pages in bounded passes (never a blocking
full VACUUM), and doctor reports state/agent DB bloat from freelist
stats.
* fix(codex): resolve the store path for thread-history import via the SDK
The supervision catalog passed the legacy sessionFile locator to the
storePath-targeted transcript mirror; resolve the agent store path with
the session-store SDK helper instead of a runtime-object seam so test
fakes and headless callers need no extra surface. Drop the obsolete
missing-session-id preprocessing case: sessions rows are NOT NULL on
session_id and upsert repairs id-less patches at write time.
* fix(sessions): fail safe on malformed disk-budget config and doctor stat errors
A malformed explicit maxDiskBytes disables the budget instead of
falling back to the destructive 2gb default the user never chose, and
the doctor bloat check skips databases whose paths stat-fail instead of
aborting doctor.
* fix(sessions): complete sqlite conflict translations
* test(sqlite): align hardening checks with maintenance
* test(sessions): inspect compressed transcript archives
* fix(tests): await session seeds and drop unused helpers flagged by CI lint
The five unawaited writeSessionStoreSeed calls raced their SQLite seeds
against the assertions, failing compact shards; the bloat probe drops a
useless initializer and the merged tests drop now-unused helpers.
* test(sessions): type legacy proof events directly
* test(sessions): align hardening contracts
* perf(sessions): read usage transcript sizes from SQL aggregates
Usage/cost scans walked every session and materialized every transcript
event just to re-stringify it for a byte estimate — the #86718 stall
class reborn on the DB. readTranscriptStatsSync sums stored JSON bytes
in SQLite without loading a single row.
* fix(sessions): re-root foreign-root transcript paths onto the current sessions dir
Restored backups, moved OPENCLAW_STATE_DIR, and rehearsal copies carry
absolute sessionFile paths from the old root; the containment fallback
kept those foreign paths, so migration read (and would archive) files in
the original root and reported local copies missing. Re-root the
canonical agents/<id>/sessions suffix onto the current dir when the file
exists there; genuine cross-root layouts still fall through unchanged.
* test(agents): seed harness admission through sqlite
* fix(sqlite): close agent db on pragma setup failure
* fix(doctor): compact and retrofit incremental auto-vacuum after session import
The migration is the sanctioned offline window: post-import compact
reclaims import churn and applies auto_vacuum=INCREMENTAL to databases
created before the fresh-DB pragma existed, so runtime maintenance can
release pages in bounded passes on every install.
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com >
2026-07-11 14:50:37 -07:00
c2fc7aa28a
refactor: consolidate exact keyed async queues ( #99691 )
...
* refactor: consolidate keyed async queues
* chore: refresh merge-ref CI
* refactor: encapsulate keyed queue ownership
2026-07-04 16:45:07 -07:00
bf2a8ecfdb
fix(trajectory): preserve usage in truncated events
2026-06-26 22:34:55 -05:00
4ae94d1d46
refactor(trajectory): trim trace config exports
2026-06-17 16:48:13 +08:00
1bdf210b43
docs: document rescue and trajectory tests
2026-06-04 20:29:59 -04:00
82c0a60777
fix(trajectory): bound runtime source ordering state
2026-05-29 18:55:53 +05:30
6361c46fe2
fix(trajectory): assign file-global runtime source order
2026-05-29 18:55:53 +05:30
d879340ed9
fix(trajectory): preserve runtime window order
2026-05-29 18:55:53 +05:30
39e3daa168
fix(trajectory): preserve safe path checks for window writes
2026-05-29 18:55:53 +05:30
538b405fdd
fix(trajectory): merge concurrent runtime window flushes
2026-05-29 18:55:53 +05:30
ac605a463a
fix(trajectory): keep latest runtime capture within cap
2026-05-29 18:55:53 +05:30
f17c472f26
fix(trajectory): keep latest runtime capture within cap
2026-05-29 18:55:53 +05:30
ddeaebfc68
fix(agents): add trajectory flush timeout diagnostics
...
Adds bounded queued-writer diagnostics to pi-trajectory-flush cleanup timeout warnings so operators can see pending write count, queued bytes, active operation, and append size without exposing paths or payloads.
Closes #82961
2026-05-18 23:29:37 -07:00
4f4d108639
chore(lint): remove underscore-dangle allow list ( #83542 )
...
* chore(lint): reduce underscore-dangle exceptions
* chore(lint): reduce more underscore exceptions
* chore(lint): remove underscore-dangle allow list
* fix(lint): repair underscore cleanup regressions
* test(lint): track version define suppression
2026-05-18 14:56:06 +01:00
694ca50e97
Revert "refactor: move runtime state to SQLite"
...
This reverts commit f91de52f0d
2026-05-13 13:33:38 +01:00
f91de52f0d
refactor: move runtime state to SQLite
...
* refactor: remove stale file-backed shims
* fix: harden sqlite state ci boundaries
* refactor: store matrix idb snapshots in sqlite
* fix: satisfy rebased CI guardrails
* refactor: store current conversation bindings in sqlite table
* refactor: store tui last sessions in sqlite table
* refactor: reset sqlite schema history
* refactor: drop unshipped sqlite table migration
* refactor: remove plugin index file rollback
* refactor: drop unshipped sqlite sidecar migrations
* refactor: remove runtime commitments kv migration
* refactor: preserve kysely sync result types
* refactor: drop unshipped sqlite schema migration table
* test: keep session usage coverage sqlite-backed
* refactor: keep sqlite migration doctor-only
* refactor: isolate device legacy imports
* refactor: isolate push voicewake legacy imports
* refactor: isolate remaining runtime legacy imports
* refactor: tighten sqlite migration guardrails
* test: cover sqlite persisted enum parsing
* refactor: isolate legacy update and tui imports
* refactor: tighten sqlite state ownership
* refactor: move legacy imports behind doctor
* refactor: remove legacy session row lookup
* refactor: canonicalize memory transcript locators
* refactor: drop transcript path scope fallbacks
* refactor: drop runtime legacy session delivery pruning
* refactor: store tts prefs only in sqlite
* refactor: remove cron store path runtime
* refactor: use cron sqlite store keys
* refactor: rename telegram message cache scope
* refactor: read memory dreaming status from sqlite
* refactor: rename cron status store key
* refactor: stop remembering transcript file paths
* test: use sqlite locators in agent fixtures
* refactor: remove file-shaped commitments and cron store surfaces
* refactor: keep compaction transcript handles out of session rows
* refactor: derive transcript handles from session identity
* refactor: derive runtime transcript handles
* refactor: remove gateway session locator reads
* refactor: remove transcript locator from session rows
* refactor: store raw stream diagnostics in sqlite
* refactor: remove file-shaped transcript rotation
* refactor: hide legacy trajectory paths from runtime
* refactor: remove runtime transcript file bridges
* refactor: repair database-first rebase fallout
* refactor: align tests with database-first state
* refactor: remove transcript file handoffs
* refactor: sync post-compaction memory by transcript scope
* refactor: run codex app-server sessions by id
* refactor: bind codex runtime state by session id
* refactor: pass memory transcripts by sqlite scope
* refactor: remove transcript locator cleanup leftovers
* test: remove stale transcript file fixtures
* refactor: remove transcript locator test helper
* test: make cron sqlite keys explicit
* test: remove cron runtime store paths
* test: remove stale session file fixtures
* test: use sqlite cron keys in diagnostics
* refactor: remove runtime delivery queue backfill
* test: drop fake export session file mocks
* refactor: rename acp session read failure flag
* refactor: rename acp row session key
* refactor: remove session store test seams
* refactor: move legacy session parser tests to doctor
* refactor: reindex managed memory in place
* refactor: drop stale session store wording
* refactor: rename session row helpers
* refactor: rename sqlite session entry modules
* refactor: remove transcript locator leftovers
* refactor: trim file-era audit wording
* refactor: clean managed media through sqlite
* fix: prefer explicit agent for exports
* fix: use prepared agent for session resets
* fix: canonicalize legacy codex binding import
* test: rename state cleanup helper
* docs: align backup docs with sqlite state
* refactor: drop legacy Pi usage auth fallback
* refactor: move legacy auth profile imports to doctor
* refactor: keep Pi model discovery auth in memory
* refactor: remove MSTeams legacy learning key fallback
* refactor: store model catalog config in sqlite
* refactor: use sqlite model catalog at runtime
* refactor: remove model json compatibility aliases
* refactor: store auth profiles in sqlite
* refactor: seed copied auth profiles in sqlite
* refactor: make auth profile runtime sqlite-addressed
* refactor: migrate hermes secrets into sqlite auth store
* refactor: move plugin install config migration to doctor
* refactor: rename plugin index audit checks
* test: drop auth file assumptions
* test: remove legacy transcript file assertions
* refactor: drop legacy cli session aliases
* refactor: store skill uploads in sqlite
* refactor: keep subagent attachments in sqlite vfs
* refactor: drop subagent attachment cleanup state
* refactor: move legacy session aliases to doctor
* refactor: require node 24 for sqlite state runtime
* refactor: move provider caches into sqlite state
* fix: harden virtual agent filesystem
* refactor: enforce database-first runtime state
* refactor: rename compaction transcript rotation setting
* test: clean sqlite refactor test types
* refactor: consolidate sqlite runtime state
* refactor: model session conversations in sqlite
* refactor: stop deriving cron delivery from session keys
* refactor: stop classifying sessions from key shape
* refactor: hydrate announce targets from typed delivery
* refactor: route heartbeat delivery from typed sqlite context
* refactor: tighten typed sqlite session routing
* refactor: remove session origin routing shadow
* refactor: drop session origin shadow fixtures
* perf: query sqlite vfs paths by prefix
* refactor: use typed conversation metadata for sessions
* refactor: prefer typed session routing metadata
* refactor: require typed session routing metadata
* refactor: resolve group tool policy from typed sessions
* refactor: delete dead session thread info bridge
* Show Codex subscription reset times in channel errors (#80456 )
* feat(plugin-sdk): consolidate session workflow APIs
* fix(agents): allow read-only agent mount reads
* [codex] refresh plugin regression fixtures
* fix(agents): restore compaction gateway logs
* test: tighten gateway startup assertions
* Redact persisted secret-shaped payloads [AI] (#79006 )
* test: tighten device pair notify assertions
* test: tighten hermes secret assertions
* test: assert matrix client error shapes
* test: assert config compat warnings
* fix(heartbeat): remap cron-run exec events to session keys (#80214 )
* fix(codex): route btw through native side threads
* fix(auth): accept friendly OpenAI order for Codex profiles
* fix(codex): rotate auth profiles inside harness
* fix: keep browser status page probe within timeout
* test: assert agents add outputs
* test: pin cron read status
* fix(agents): avoid Pi resource discovery stalls
Co-authored-by: dataCenter430 <titan032000@gmail.com >
* fix: retire timed-out codex app-server clients
* test: tighten qa lab runtime assertions
* test: check security fix outputs
* test: verify extension runtime messages
* feat(wake): expose typed sessionKey on wake protocol + system event CLI
* fix(gateway): await session_end during shutdown drain and track channel + compaction lifecycle paths (#57790 )
* test: guard talk consult call helper
* fix(codex): scale context engine projection (#80761 )
* fix(codex): scale context engine projection
* fix: document Codex context projection scaling
* fix: document Codex context projection scaling
* fix: document Codex context projection scaling
* fix: document Codex context projection scaling
* chore: align Codex projection changelog
* chore: realign Codex projection changelog
* fix: isolate Codex projection patch
---------
Co-authored-by: Eva (agent) <eva+agent-78055@100yen.org >
Co-authored-by: Josh Lehman <josh@martian.engineering >
* refactor: move agent runtime state toward piless
* refactor: remove cron session reaper
* refactor: move session management to sqlite
* refactor: finish database-first state migration
* chore: refresh generated sqlite db types
* refactor: remove stale file-backed shims
* test: harden kysely type coverage
# Conflicts:
# .agents/skills/kysely-database-access/SKILL.md
# src/infra/kysely-sync.types.test.ts
# src/proxy-capture/store.sqlite.test.ts
# src/state/openclaw-agent-db.test.ts
# src/state/openclaw-state-db.test.ts
* refactor: remove cron store path runtime
* refactor: keep compaction transcript handles out of session rows
* refactor: derive embedded transcripts from sqlite identity
* refactor: remove embedded transcript locator handoff
* refactor: remove runtime transcript file bridges
* refactor: remove transcript file handoffs
* refactor: remove MSTeams legacy learning key fallback
* refactor: store model catalog config in sqlite
* refactor: use sqlite model catalog at runtime
# Conflicts:
# docs/cli/secrets.md
# docs/gateway/authentication.md
# docs/gateway/secrets.md
* fix: keep oauth sibling sync sqlite-local
# Conflicts:
# src/commands/onboard-auth.test.ts
* refactor: remove task session store maintenance
# Conflicts:
# src/commands/tasks.ts
* refactor: keep diagnostics in state sqlite
* refactor: enforce database-first runtime state
* refactor: consolidate sqlite runtime state
* Show Codex subscription reset times in channel errors (#80456 )
* fix(codex): refresh subscription limit resets
* fix(codex): format reset times for channels
* Update CHANGELOG with latest changes and fixes
Updated CHANGELOG with recent fixes and improvements.
* fix(codex): keep command load failures on codex surface
* fix(codex): format account rate limits as rows
* fix(codex): summarize account limits as usage status
* fix(codex): simplify account limit status
* test: tighten subagent announce queue assertion
* test: tighten session delete lifecycle assertions
* test: tighten cron ops assertions
* fix: track cron execution milestones
* test: tighten hermes secret assertions
* test: assert matrix sync store payloads
* test: assert config compat warnings
* fix(codex): align btw side thread semantics
* fix(codex): honor codex fallback blocking
* fix(agents): avoid Pi resource discovery stalls
* test: tighten codex event assertions
* test: tighten cron assertions
* Fix Codex app-server OAuth harness auth
* refactor: move agent runtime state toward piless
* refactor: move device and push state to sqlite
* refactor: move runtime json state imports to doctor
* refactor: finish database-first state migration
* chore: refresh generated sqlite db types
* refactor: clarify cron sqlite store keys
* refactor: remove stale file-backed shims
* refactor: bind codex runtime state by session id
* test: expect sqlite trajectory branch export
* refactor: rename session row helpers
* fix: keep legacy device identity import in doctor
* refactor: enforce database-first runtime state
* refactor: consolidate sqlite runtime state
* build: align pi contract wrappers
* chore: repair database-first rebase
* refactor: remove session file test contracts
* test: update gateway session expectations
* refactor: stop routing from session compatibility shadows
* refactor: stop persisting session route shadows
* refactor: use typed delivery context in clients
* refactor: stop echoing session route shadows
* refactor: repair embedded runner rebase imports
# Conflicts:
# src/agents/pi-embedded-runner/run/attempt.tool-call-argument-repair.ts
* refactor: align pi contract imports
* refactor: satisfy kysely sync helper guard
* refactor: remove file transcript bridge remnants
* refactor: remove session locator compatibility
* refactor: remove session file test contracts
* refactor: keep rebase database-first clean
* refactor: remove session file assumptions from e2e
* docs: clarify database-first goal state
* test: remove legacy store markers from sqlite runtime tests
* refactor: remove legacy store assumptions from runtime seams
* refactor: align sqlite runtime helper seams
* test: update memory recall sqlite audit mock
* refactor: align database-first runtime type seams
* test: clarify doctor cron legacy store names
* fix: preserve sqlite session route projections
* test: fix copilot token cache test syntax
* docs: update database-first proof status
* test: align database-first test fixtures
* docs: update database-first proof status
* refactor: clean extension database-first drift
* test: align agent session route proof
* test: clarify doctor legacy path fixtures
* chore: clean database-first changed checks
* chore: repair database-first rebase markers
* build: allow baileys git subdependency
* chore: repair exp-vfs rebase drift
* chore: finish exp-vfs rebase cleanup
* chore: satisfy rebase lint drift
* chore: fix qqbot rebase type seam
* chore: fix rebase drift leftovers
* fix: keep auth profile oauth secrets out of sqlite
* fix: repair rebase drift tests
* test: stabilize pairing request ordering
* test: use source manifests in plugin contract checks
* fix: restore gateway session metadata after rebase
* fix: repair database-first rebase drift
* fix: clean up database-first rebase fallout
* test: stabilize line quick reply receipt time
* fix: repair extension rebase drift
* test: keep transcript redaction tests sqlite-backed
* fix: carry injected transcript redaction through sqlite
* chore: clean database branch rebase residue
* fix: repair database branch CI drift
* fix: repair database branch CI guard drift
* fix: stabilize oauth tls preflight test
* test: align database branch fast guards
* test: repair build artifact boundary guards
* chore: clean changelog rebase markers
---------
Co-authored-by: pashpashpash <nik@vault77.ai >
Co-authored-by: Eva <eva@100yen.org >
Co-authored-by: stainlu <stainlu@newtype-ai.org >
Co-authored-by: Jason Zhou <jason.zhou.design@gmail.com >
Co-authored-by: Ruben Cuevas <hi@rubencu.com >
Co-authored-by: Pavan Kumar Gondhi <pavangondhi@gmail.com >
Co-authored-by: Shakker <shakkerdroid@gmail.com >
Co-authored-by: Kaspre <36520309+Kaspre@users.noreply.github.com >
Co-authored-by: dataCenter430 <titan032000@gmail.com >
Co-authored-by: Kaspre <kaspre@gmail.com >
Co-authored-by: pandadev66 <nova.full.stack@outlook.com >
Co-authored-by: Eva <admin@100yen.org >
Co-authored-by: Eva (agent) <eva+agent-78055@100yen.org >
Co-authored-by: Josh Lehman <josh@martian.engineering >
Co-authored-by: jeffjhunter <support@aipersonamethod.com >
2026-05-13 13:15:12 +01:00
17ceca86d6
Redact persisted secret-shaped payloads [AI] ( #79006 )
...
* fix: redact persisted secret-shaped payloads
* docs: add changelog entry for PR merge
2026-05-11 12:52:32 +05:30
01741f81f8
test: remove stale unused imports
2026-05-09 11:26:43 +01:00
474bea162b
fix: bound trajectory runtime flush ( #77154 )
...
* fix: bound trajectory runtime flush
* fix: keep trajectory export cap compatible
* test: keep followup delivery test pure
2026-05-04 09:48:03 +01:00
1888242bd3
perf: split trajectory export paths
2026-04-25 12:52:27 +01:00
a3d9c53db2
feat: add trajectory bundle export and default-on runtime capture ( #70291 )
...
* Trajectory: export session bundles by default
* Harden trajectory export diagnostics integration
* Address trajectory export review feedback
* Share diagnostics and trajectory bundle plumbing
* Harden trajectory recording and export
* Confine trajectory export outputs
* Document trajectory export command
* Harden trajectory export bundle privacy
* Redact trajectory sidecar paths
* Fix plugin install checks after rebase
* Keep queued writers working without O_NOFOLLOW
* Keep Codex trajectory writes without O_NOFOLLOW
* Harden trajectory export path handling
* Redact mixed trajectory export paths
2026-04-22 23:29:01 -07:00
Josh Lehman