5 Commits

Author SHA1 Message Date
Patrick Buckley 7a06f5e8bc refactor(session): make ModelLane the provider boundary (#979) (#989)
* refactor(session): make ModelLane the provider boundary (#979)

## Summary

This closes the model-lane ownership gap left by #832: `ChatSession` no longer stores raw provider/client handles. `ResolvedModelBinding` now carries the provider, client, model, capabilities, registry generation, and backend-auth configuration as one coherent snapshot.

- Atomically rebind existing sessions after model-registry changes while pinning each in-flight send, fallback, judge, output guard, task agent, title, compaction, perception, and voice operation to its initiating principal and binding.
- Fence UI publication, canonical trajectory folds, durable writes, streams, retries, child scopes, and judge work by generation. Stop can hand off to a successor without accepting late state; cancelled tools retain typed effect receipts, and concurrent approval batches resolve by exact cycle or call.
- Make create, fork, open, close, and delete race-safe with hidden `creating` reservations, incarnation-aware state tails, and an ACL-rechecked transaction that clones checkpoint-bounded history, configuration, project/persona state, and attachment references.
- Extend REST/OpenAPI and Python/TypeScript SDK contracts for create/fork inputs, routed-create metadata, live-workstream probes, targeted approvals, and structured cancellation results.
- Update architecture, storage, authentication, judge, channel, console, API, and SDK documentation, including regenerated architecture diagrams and OpenAPI artifacts.

## Validation

- SQLite suite: 11,188 passed, 9 skipped, 10 deselected
- PostgreSQL suite: 11,195 passed, 2 skipped, 10 deselected
- Live backend: 3 passed
- SSE recovery: 6 passed; browser recovery harness passed all scenarios
- Ruff: clean; 595 files correctly formatted
- mypy: 243 source files clean
- TypeScript: typecheck/build and 35 tests passed
- OpenAPI artifacts fresh; all 14 changed diagrams reproduce byte-for-byte
- `git diff --check` and Git LFS integrity clean

Closes #979.

* fix(deps): update nanoid for GHSA-2v37-7h3g-55p8

Refresh the transitive lock entry admitted by PostCSS so the TypeScript security gate no longer resolves the vulnerable custom-generator implementation.

Validation:
- npm ci
- npm audit --audit-level=moderate: 0 vulnerabilities
- TypeScript typecheck and build
- TypeScript tests: 35 passed

* fix(test): assert canonical model registry URLs

Replace prefix checks with exact canonical base URL assertions so the tests do not model incomplete URL validation.

Validation: tests/test_model_registry.py (185 passed); Ruff check/format; mypy.
2026-08-08 16:13:35 -07:00
Patrick Buckley 0f1d03755a refactor(storage): drop dead ws_id/user_id from workstream_attachments
The blob store is global content-addressed — identical bytes dedupe across
workstreams and users, so the per-tenant ws_id/user_id scope columns are dead:
nothing reads them, and a committed blob is authorised via the
conversations.attachments ref-list (attachment_referenced_in_ws), not a row
scope. Drop both columns and idx_ws_attachments_ws_id from the schema and from
the save_attachment signature (protocol + both backends + memory wrapper + the
caller); fold the column/index drops and their downgrade into the unshipped
migration 060.

tool_name stays: it is a live denormalised search label (search_history →
recall + /history), not trajectory data — "never rehydrated" held only for the
wire path, which already ignores it.
2026-06-04 11:03:13 -07:00
Patrick Buckley 63e9205e83 refactor(storage): drop the load-time orphan synth; repair only at send
reconstruct_messages(repair=True) did two things: strip a trailing incomplete
tool-call turn AND synthesize cancellation results for mid-conversation
orphans.  The mid-orphan synth was a near-duplicate of
lowering.repair_wire_messages — same detector, same contiguous insert past
interspersed system turns, same cancellation string — running at the wrong
layer (storage, on every load).

Drop it: load is now trailing-strip only (boot-crash recovery), and the
mid-orphan synth happens once, at send, in lowering.repair_wire_messages — the
single place the wire path fills orphans.  The session send path gets it via
_prepare_wire_messages; export, which bypasses that path, now runs
repair_wire_messages itself (otherwise a mid-conversation orphan would
serialize as an unanswered tool_call).  The duplicated cancellation string
goes with the synth — CANCELLED_TOOL_RESULT lives only in lowering now.

Safe: a bare mid-orphan is harmless between load and send (token count is
additive, /history reads repair=False, compaction summarizes to text), and
every wire path repairs it.  Reconstruct tests updated to the new load
contract; an export mid-orphan test added.
2026-06-04 11:03:13 -07:00
Patrick Buckley 98cee4d20c feat(storage): content-addressed refcounted attachments + in-memory upload buffer
Replace the persisted pending/reserved/consumed upload lifecycle (and its orphan-sweep
and per-user cap) with a content-addressed, refcounted blob store fronted by the per-node
in-memory pending buffer:

- Upload stages bytes in the buffer (keyed by sha256); send-commit drains the referenced
  handles, writes each blob content-addressed (INSERT-OR-IGNORE then refcount += 1, so a
  stored blob is born referenced and dedupes across messages/workstreams), and records the
  ordered conversations.attachments ref-list — the sole message->blob link.
- reconstruct rebuilds inline image_url/document multipart content from the ref-list,
  role-agnostically (so tool-produced images via _exec_read_image now persist + rehydrate
  instead of being flattened to text and lost). Output shape unchanged.
- GC is reference counting: delete_messages_after / delete_workstream decrement once per
  reference and prune a blob at 0; a deduped blob shared with a kept turn (or another ws)
  survives.
- get_content for a committed blob is gated by reference-ownership (the requester owns a
  turn in the ws whose ref-list names the id), replacing the dropped ws_id/user_id scope.
- Migration 060 re-keys legacy consumed attachments to their content hash, dedups, sets
  refcounts, writes the ref-lists, and drops message_id/reserved_*; pending legacy rows are
  dropped (pending now lives only in the buffer).

Both backends symmetric; the reservation methods, cap, and orphan-sweep are removed across
storage/facade/protocol/endpoints/coordinator. Wire harness byte-identical; full suite green.
2026-06-04 11:03:13 -07:00
Patrick Buckley a2834349b0 feat(export): export workstream conversations as OpenAI messages JSON
Add a workstream conversation export on three surfaces, all sharing one
serializer (turnstone/core/export.py):

- `turnstone-admin export <ws_id> [--children] [-o FILE|-]` — offline,
  direct-DB. `--children` bundles a coordinator's parent conversation
  plus one JSON per child into a zip (parent.json + children/<id>.json,
  no manifest).
- `GET /v1/api/workstreams/{ws_id}/export` — conversation-only file
  download, mounted on both the node (interactive) and console
  (coordinator) lifespans via `make_export_handler(cfg)`, reusing the
  /history gate ladder (permission_gate, tenant_check, list_kind
  cross-kind isolation) so ownership and isolation come for free.
- Web UI — an "Export conversation" item in the interactive per-tab
  dropdown (scoped to that tab's workstream) and an Export button on the
  coordinator appbar.

Format is OpenAI Chat Completions messages JSON (`{"messages": [...]}`),
built from `sanitize_messages(load_messages(repair=True))`. Persisted
reasoning is surfaced on assistant messages as a flat `reasoning_content`
field (the convention OpenAI-compatible inference servers use) via a
dedicated helper that runs before sanitize strips the internal
_provider_content lane. Attachments ride along as the standard image_url
/ inlined-document content parts.

Lets users get conversations out in a portable interchange format
(backup, fine-tuning datasets, sharing, interop) without lock-in.
Closes #613.

Non-obvious decisions:
- Single format (openai-json); children/zip is CLI-only. The HTTP
  endpoint and web UI are conversation-only, keeping the served surface
  — and its security surface (no child rows read through the coordinator
  handler) — small.
- `reasoning_content`, not the `reasoning` field /history and the
  reasoning-replay path use: export targets the chat-completions
  convention. Documented in export.py to prevent a "consistency fix".
- list_workstreams exposes no cursor, so the child walk passes an
  explicit high limit rather than inheriting the default 100, which
  would silently drop a coordinator's children past 100.
- Interactive export lives in the per-tab menu (interactive is
  per-tab/pane — avoids focused-workstream ambiguity); the coordinator
  is one conversation, so it keeps an appbar button.

Tested: 25 new tests through real storage + handlers (TestClient), incl.
cross-kind isolation 404, misconfig 500, the reasoning + attachment
pipeline, and the coordinator children zip. The shared frontend helper
is verified by a node sandbox harness (re-entrancy guard, button
disable/aria-busy, no-button tab-menu path). Full non-live suite green
(6714 passed); ruff + format + mypy clean; OpenAPI spec updated.
2026-05-29 21:11:16 -07:00