* fix(session): fail-closed history-commit handoff (#981)
The deleted-workstream discovery is now a terminal, ws_id-keyed latch:
keyed conversation commits refuse admission once the durable parent is
gone (convergence finalizers and force-abandon are exempt), history
handoff refuses to mint a proof token so /history fails closed with a
503 instead of silently wiping the pane, and the SSE stream carries a
workstream_gone resync reason. Discarded commits leave a forensic log
of commit keys and roles, never content.
Conversation rows gain a commit_key (migration 071): keyed saves are
idempotent under retry, validated against the full commit identity, and
refused when they would cross a workstream deletion. The prune orphan
category now requires a NULL alias plus a two-hour updated grace, with
cutoffs computed at discovery time and carried into both dialects'
rechecks.
The mid-turn interjection queue is owner-partitioned with no per-site
mode flags: pops take the acting principal's and unowned rows, other
participants' rows are structurally retained, and enforcement lives at
queue admission plus the shared before_spawn gates. The retraction
ledger is bounded by open pop windows: pops open a window atomically
with the queue delete, restores close their ids atomically with the
ledger consume, every other exit closes through one helper, and misses
for unheld ids record nothing. The workstream-gone latch refuses
unattended wakes at all three gates (watcher spawn, claim, delivery
pre-pop), and the retry dispatcher regained its pre-envelope
cancel/error convergence net.
Persistence-state reporting derives through the session bound to each
UI instead of a registry lookup by id that failed open to healthy
during tombstone retention. The dashboard roster no longer re-inserts
ghost entries from trailing activity events, the history tool-outcome
scan tolerates interleaved non-turn rows, and the shared
handoff-deadline handle owns its own retirement.
Single-sourced across call sites: keyed-commit row values, attachment
save wrappers, tail-truncation and conflict-resolution bodies for both
storage dialects; worker-slot lifecycle field sets; the direct-commit
admission frame; queued-row layout accessors; the string-aware comment
stripper shared by every JS harness suite.
Refs #981#964
* fix(session): sweep handoff fixes to their sibling surfaces
The interactive replay loop treated a system row as a tool-batch
boundary, so every tool result after an interleaved row vanished from
that pane while the coordinator rendered the same history correctly.
Only a conversational turn ends the batch window now, matching the
shared outcome index.
Accepted user turns clear the composer's attachment chips on the same
viewer policy that settles optimistic bubbles rather than on having
matched a local bubble, so a workstream created with an upload no
longer keeps a chip for an attachment the create dispatch already
consumed. The coordinator's raced-Stop arm emits the stream-end hook it
inherits alongside the idle state, leaving no unfinalized bubble or
unflushed tool output. Ending a session surfaces a failure toast when
the request never lands or answers with a non-JSON body.
The per-second persistence reconcile now probes each session without
blocking: a workstream whose generation and handoff locks are held is
skipped until the next pass instead of contending the locks every
commit needs. The one-shot repair that gates workstream creation at
capacity keeps a definite probe — it has no next pass, and the sessions
likeliest to be contended are the ones whose unresolved journals
emptied its candidate list.
Single-sourced: the attachment lane builds its conversation row through
the shared commit-identity builder; the ordinary worker exit releases
its slot through the lifecycle owner; both operator surfaces snapshot
their counters through one non-consuming helper; the replay preamble
loses its per-kind wrappers and its config hook; the browser harness
suites share one brace walker; and each in-flight history attempt is
one record carrying both its abort controller and its deadline.
Refs #981#964
* feat(models): add per-alias concurrency admission
Add registry-backed FIFO admission limits with queue-aware deadlines and full-stream leases. Expose max_concurrency through storage, admin configuration, OpenAPI, documentation, and diagrams, with role and live backend count coverage.
* fix(api): omit null concurrency schema default
Keep max_concurrency optional for presence-keyed updates without advertising a null default for its non-null integer OpenAPI shape.
* refactor(session): make ModelLane the provider boundary (#979)
## Summary
This closes the model-lane ownership gap left by #832: `ChatSession` no longer stores raw provider/client handles. `ResolvedModelBinding` now carries the provider, client, model, capabilities, registry generation, and backend-auth configuration as one coherent snapshot.
- Atomically rebind existing sessions after model-registry changes while pinning each in-flight send, fallback, judge, output guard, task agent, title, compaction, perception, and voice operation to its initiating principal and binding.
- Fence UI publication, canonical trajectory folds, durable writes, streams, retries, child scopes, and judge work by generation. Stop can hand off to a successor without accepting late state; cancelled tools retain typed effect receipts, and concurrent approval batches resolve by exact cycle or call.
- Make create, fork, open, close, and delete race-safe with hidden `creating` reservations, incarnation-aware state tails, and an ACL-rechecked transaction that clones checkpoint-bounded history, configuration, project/persona state, and attachment references.
- Extend REST/OpenAPI and Python/TypeScript SDK contracts for create/fork inputs, routed-create metadata, live-workstream probes, targeted approvals, and structured cancellation results.
- Update architecture, storage, authentication, judge, channel, console, API, and SDK documentation, including regenerated architecture diagrams and OpenAPI artifacts.
## Validation
- SQLite suite: 11,188 passed, 9 skipped, 10 deselected
- PostgreSQL suite: 11,195 passed, 2 skipped, 10 deselected
- Live backend: 3 passed
- SSE recovery: 6 passed; browser recovery harness passed all scenarios
- Ruff: clean; 595 files correctly formatted
- mypy: 243 source files clean
- TypeScript: typecheck/build and 35 tests passed
- OpenAPI artifacts fresh; all 14 changed diagrams reproduce byte-for-byte
- `git diff --check` and Git LFS integrity clean
Closes#979.
* fix(deps): update nanoid for GHSA-2v37-7h3g-55p8
Refresh the transitive lock entry admitted by PostCSS so the TypeScript security gate no longer resolves the vulnerable custom-generator implementation.
Validation:
- npm ci
- npm audit --audit-level=moderate: 0 vulnerabilities
- TypeScript typecheck and build
- TypeScript tests: 35 passed
* fix(test): assert canonical model registry URLs
Replace prefix checks with exact canonical base URL assertions so the tests do not model incomplete URL validation.
Validation: tests/test_model_registry.py (185 passed); Ruff check/format; mypy.
Adds the dedicated `rfc8693_obo` model auth mode (#955): model
definitions gain an `obo_scopes` column (migration 069), the mint
threads the scopes to the token-exchange leg (RFC 8693), and every
dynamic mode pins its grant leg — a mode is a dialect commitment, not a
hint the deployment profile resolves. Exchange-capable IdPs refuse an
audience whose scope was not requested; this closes the structurally
unmintable model-OBO path on token-exchange deployments.
The model mint-cache is identity-keyed on the owning definition's
alias (`__model_obo__:<alias>` per user, `__model_app__:<alias>` under
the shared app principal), matching the MCP discipline where rows key
on the unique server name. The bearer's shape lives in the row's
audience/scopes columns and the freshness gate compares it on every
read, so a re-aimed alias refuses its old row and overwrites the same
key in place. Admin lifecycle (rename, re-aim, scope change, delete)
purges a definition's own rows through one shared helper — sound
because one definition owns each key; a sibling's rows are untouchable
by construction. Cooldown and backoff additionally key on the dispatch
shape, so an operator's config repair is an instant clean slate. Cause
records, cooldowns, locks and memoization are per-alias end to end,
and the session heartbeat reads refusal causes under the same keys.
Console: default-deny write gating for dynamic rows (value-diff over
the full column ladder, admin.mcp escalation, a never-blockable
pure-disable carve-out), a two-tier validator (audience allow-list on
every write; deployment-posture checks when the pair is chosen), one
shared scopes parser whose omit-unchanged arm keeps over-cap DB-direct
residue rows disarmable without ungating real changes, and served
constraints (dynamic/scopes/app-identity mode lists, mode-to-profile
pairing) so the shelf tracks the registry by data. The admin shelf
gains the mode option, a scopes input with residue affordances,
pairing-aware option greying, and a derived auth badge.
Registry load refuses control characters in alias, audience, and
scopes — including the C0 separator block that str.split() would
silently collapse — and the C0/DEL class has one exported spelling
shared by every surface. Profile-mismatch visibility warns at reload
and boot with the mode-correct cause, gated on OIDC being enabled.
Breaking: a stored `entra_obo` alias on a deployment whose
`[oidc] obo_grant_profile` is `rfc8693` (or the inverse pairing) no
longer mints via the profile-driven overload — the mint refuses before
any IdP traffic with cause `grant_profile_mismatch`, and the
`model.auth_fail_closed` policy governs static fallback. Such rows
never minted usefully on scope-gating IdPs; the shelf now surfaces the
pairing and the per-turn heartbeat names the refusal cause.
Live-verified end to end: scoped token exchange mints, the warm cache
serves with zero IdP calls, and the mode/profile mismatch refuses with
zero IdP traffic (scripts/obo-e2e/keycloak_e2e.sh); the
refresh-redemption profile's E1-E7 hold via scripts/obo-e2e/entra_e2e.py.
Closes#955.
Follow-up to the per-alias Entra OBO/app-identity backend auth: the
console write path now applies default-deny field classification, the
admin shelf gains full backend-auth support, and the session/registry
rebind machinery is hardened for config changes landing under live
sessions.
Console write gate:
- Default-deny classification: any non-neutral change to a row that is
or becomes dynamic requires admin.mcp plus validation; the provably
auth-neutral columns are enumerated (MODEL_AUTH_NEUTRAL_FIELDS) and a
live-schema classification test forces every future column to be
classified. The derivation is a pure function (_derive_auth_gate)
with unit-pinned exclusivity invariants.
- Two-tier validation mirroring the MCP oauth_obo validator: the row
tier (audience allow-list) runs on every gated write; the posture
tier (OIDC configured, token store present) runs on pair changes and
on enable-arming.
- Pure-disable carve-out: disabling a dynamic row is de-escalation and
is never blocked — admin.models suffices and validation is skipped,
including for rows with corrupt or skewed stored values.
- Capabilities are compared canonically (key order, integral floats),
the audience compare normalizes both sides, and staging an audience
on a static row is refused on both write twins.
- Calibrate writes the capabilities column under an enforced
confinement invariant with a compare-and-swap persist.
Admin shelf:
- Backend-auth section with a per-open constraints fetch
(GET /model-definitions/auth-constraints: audience allow-list, grant
profile, dynamic modes), datalist audience suggestions,
server-defined modes preserved on round-trip, and permission-aware
visibility built on cache-skew-safe helpers shared through auth.js.
- Refused live-registry swaps surface as an amber registry_warning on
the write, delete, reload, and calibrate responses; audit rows carry
auth_gated / auth_disarmed markers visible in the audit view.
Registry and sessions:
- The encryption-key requirement for dynamic auth is enforced inside
ModelRegistry.reload() itself — nodes refuse with 503 and the
console records coord_registry_error — and reload bumps the
generation before the map swap so a racing reader can never pair a
stale generation with new maps.
- resolve()/resolve_binding() return the generation from inside the
registry lock; sessions rebind per send on generation change with
atomic client/provider/config commits, fallback-first handling of
removed or unconstructable aliases, and judge/limiter resets only
when the binding actually changed.
- Mint refusals record per-user causes surfaced in the per-turn
heartbeat logs; misconfiguration warnings are deduplicated with
bounded state.
Verification: 10417 tests (99 added on this branch), a 71-scenario
browser harness over the real admin shelf, and a live rfc8693
token-exchange e2e run (MCP legs verified end to end; the model-leg
scope gap is tracked as #955 under a narrow known-gap signature).
Closes#950.
Adds a per-alias `auth_mode` on model definitions so a model backend can
authenticate to an Entra-fronted gateway with a per-request minted token instead
of one shared static API key, letting the gateway attribute calls to the actual
user or to the app as a machine identity.
- `static` (default, unchanged) sends the stored `api_key`.
- `entra_obo` mints a per-user On-Behalf-Of token for `obo_audience` from the
caller's captured refresh credential.
- `entra_app` mints an app-identity token via the client-credentials grant, and
covers userless turns that OBO cannot.
Reuses the existing OBO grant legs, refresh-token rotation CAS, cluster advisory
lock and the `mcp_user_tokens` mint-cache, keyed under synthetic
`__model_obo__:<audience>` / `__model_app__:<audience>` rows. The token binds at
the call site through `client.with_options(api_key=...)` so each SDK emits it on
its own auth path rather than through header injection.
Migration 068 adds `auth_mode` and `obo_audience`. Both are additive and existing
rows default to `static`, so behaviour is unchanged unless an alias opts in.
Operator controls: `model.auth_audience_allowlist` is an exact-match allow-list
that gates which audiences may be configured and denies all by default, and
changing a mode or audience requires `admin.mcp`. `model.auth_fail_closed`
decides whether a failed mint may fall back to an explicitly configured static
key. A delegated call with no user, or a dynamic alias with no real static key,
always refuses.
Two changes here apply regardless of whether any alias opts in:
- Storage and app state are now wired into the console MCP client manager. This
fixes per-user `oauth_user` / `oauth_obo` dispatch for coordinator-hosted
sessions, which previously raised `RuntimeError` on first call because
`set_app_state` was only ever called on the node.
- Unattended watch restores and `--resume` resolve the persisted workstream
owner instead of constructing the session under an empty principal. A
workstream with no owner is now a permanent refusal rather than an anonymous,
auto-approved run.
Picks up the needs_user status description and the new note property
from console_schemas.py. Spec-only, no API behavior change.
The server spec is knowingly left stale here: the generator writes
both files unconditionally, and its drift belongs to the change that
introduced it rather than to this one.
The console OpenAPI spec had drifted from build_console_spec(): the committed
file was last generated at 1.7.0rc1 and was missing the persona and project_id
workstream-creation fields (Personas and Projects, both 1.7) plus the version
bump to 1.8.0a2. Regenerate via sdk/typescript/scripts/generate-types.py to
resync. Spec-only; no console API behavior change (openapi-server.json was
already current).
Eight rounds of findings against the defer-and-drain seam shared one
generator: N sites each hand-copying M obligations (spawn discipline,
the order-barrier pair, backpressure, best-effort emission, the client
settle matrix), with every review finding an empty (site x obligation)
cell. This round makes each obligation a single primitive:
- The order barrier is Workstream.send_barrier_active() — one
definition of the two-term pair (pending entries OR drain alive),
consulted by the /send route, the coordinator adapter, and the
queued-nudge wake gate, which previously carried only the list term
and let a synthetic wake jump an acknowledged send during the
claimed-entry window. _PendingSend moved to workstream.py beside the
invariant that justifies the drain-alive term; the pending fields got
precise types and worker_kind became a Literal, so a typo'd
"command" comparison is now a type error instead of a silently
never-firing defer guard.
- _defer_send probes the barrier before constructing anything, bounds
acceptance at 10 pending (the interjection queue's own backpressure
contract — unbounded acceptance pinned message + attachment bytes
per entry for a whole command window and then ran one unattended
turn each), and spawns the drain with rollback: a Thread.start
failure pops the just-accepted entry and answers the retryable
queue_full instead of 500ing after registration (a phantom the
client could neither see nor retract, dispatched later as duplicate
turns). start() deliberately stays inside the lock, unlike
session_worker's outside-lock discipline: this slot is
is_alive()-gated, false for a constructed-but-unstarted thread, so
an outside-lock start would open a double-drain window.
- A /command whose worker never spawned answers 503
{"status": "error"} (spec + docs + a pane error arm) instead of the
generic 200 ok that told SDK callers their /clear ran.
- The compaction lifecycle emitter is raise-proof at its single
dispatch tail: a raising duck-typed hook degrades to a lost render,
never a lost end event — previously a raising on_error or a raising
failed-end emit left every pane a frozen progress bar, and a raising
SUCCESS end after the committed swap fabricated a failed end.
- The client settle matrix lives once: composer_queue's
settleSendResponse owns every /send response arm for both panes
(the near-verbatim twins were already drifting), parsePriority is
shared, and the busy stamp is centralized in setBusy(b, source) with
"server" as the fail-safe default. Deferred sends release the
composer (no worker exists for them; retracting the chip no longer
strands the pane in Stop mode), queue_full on an idle-looking pane
removes the optimistic bubble and restores busy (the refusal can now
fire with no worker and no drain to ever emit a state event), and
the pre-bind settle buffer is TTL-based — a burst of deferred
dispatches parked this tab's own raced settle first, where the old
size cap evicted exactly it.
- The command backstop / console proxy timeout inequality is enforced
by a test importing both named constants (both proxy_client
constructions, startup and the mTLS re-create); the compaction card
wears blue (magenta is reserved for the MCP surface); the redundant
TerminalUI.on_compaction override is gone (the inherited protocol
default is the policy site).
Completes the defer-and-drain seam against the matrix rows round 6 never
enumerated (the defer contract itself took no hits):
- Crash row: a claimed entry survives a dispatch crash — the
per-iteration handler re-inserts it at head (claim-flagged so a
claim-section failure can't duplicate it), backs off ~1s, retries.
The last-resort handler spawns no successor (Thread.start fails under
the exact exhaustion that reaches it): the route's ensure-drain stays
the single spawn site, so single-flight is structural and a dead
drain revives on the next defer.
- Order row: the pending list is the order authority. The /send route
pre-checks pending/drain-alive under the same lock acquisition that
appends (one _defer_send helper serves the barrier and command-window
triggers); the coordinator adapter refuses via its return value; the
queued-nudge wake gate yields to pending sends and is re-armed by the
drain's clean exit — which covers lists emptied by pure retraction —
as well as every deferred turn's exit; retry-after-rewind is a
documented accepted overtake; init/create is fresh-ws-by-construction.
- Client settle row: queued responses carry "deferred": true
(SendResponse + regenerated openapi-server.json, status enumeration
completed); bind(el, msgId, {deferred, attachedCount}) replaces the
_deferredAttachments expando; the idle sweep skips deferred and
unbound chips; the shared dispatch attempt emits pane-tier
message_dispatched (folded: true for interjection fold-ins — the chip
clears only its deferred flag and keeps a live x while DELETE still
genuinely retracts); settles that beat bind() park in a bounded
buffer; an idle-thinking pane retro-converts its optimistic bubble
into a real queued chip instead of presenting a parked message as
sent. Rejection polling waits on the slot flags — one dispatch
attempt per slot-state change, not 4 Hz.
- SessionUI.on_compaction's protocol stub became a real default body
(the classic on_info rendering): explicit subclasses inherit protocol
members as real methods, which defeated _compaction_event's getattr
fallback for exactly the pre-1.8 embedders it serves.
- _coerce_event_id() rejects bools (isinstance(True, int) is True) at
all three duck-typed event-id coercions: the compaction marker stamp,
on_system_turn's persisted return, and _ui_event_id.
- Quick-command backstop 60s -> 25s, under the console proxy's 30s so
the degraded "running" answer can traverse a proxied pane (which now
surfaces it); /resume docs drop the fictional history SSE event
(clear_ui + REST re-fetch is the contract); /send response docs match
the wire.
Replace park-and-abandon /send semantics with defer-and-drain: a send
landing in a command window is answered {status: queued, msg_id}
immediately and dispatched full-fidelity by a per-workstream drain
thread when the window closes. Parking encoded client disconnect as
message retraction — true only for the composer's ✕-abort; every
bounded caller (coordinator client and console proxy at timeout=30,
SDKs, stock proxies) timed out and lost its message for the whole
window, and the compensating client machinery was racy (one-shot
sendAbortMs sample) and over-broad (_sendAbort fired on the
interjection path, dispatching dismissed messages while showing a
connection error). Dismissal is now uniformly bind() → DELETE, with a
fall-through that retracts pending entries; retracting an
attachment-bearing deferred send surfaces the discarded-attachments
consequence. The drain claims entries under ws._lock immediately
before dispatch (DELETE can never remove an in-flight message),
refuses the truncating interjection fallback for oversized or
attachment entries atomically inside the enqueue callback, and never
gives up while the workstream lives; durability is documented as
node-local at-most-once. sendAbortMs, _sendAbort, the 600s bound and
the park loop are deleted; route and drain share one dispatch
implementation (spawn metrics included).
Also: the initial-send completion notify is un-gated from slot
ownership (_fire_notify_targets has exactly one call site — successor
turns never notify, so the round-5 guard prevented a duplicate that
cannot exist while converting force-cancel into permanent notification
loss for scheduled workstreams); /compact on an ERROR workstream
restores the badge instead of stamping idle over it; duck-typed
SessionUIs without on_compaction get the classic on_info lines back
via a shared renderer (superseded OK ends included — a committed swap
must never be silent; pre-1.8 SSE clients are deliberately not
dual-emitted, documented as a 1.8 breaking change); failed-end notice
suppression is computed once by the emitter as a notice bool on the
end event (SDK py+ts), replacing the hand-synced cli/JS policy while
the panes keep their pane-local card-ownership clause.
Compaction becomes visible: a first-class 'compaction' SSE lifecycle
(start/progress/end, compaction_id-correlated, superseded-flagged ends)
replaces the loose info lines; both web panes render a progress-bar card
that settles into a persistent result card, re-rendered after reload via
the /history projection of the compaction marker row. Slash commands echo
as command chips instead of fake user turns.
The enabling rework: /command dispatches onto the workstream worker slot
(the old inline path blocked the node's event loop for whole compactions
and let /clear interleave with live turns). Busy refusals answer 409;
quick commands are awaited loop-natively with a 60s backstop; /compact is
fire-and-forget. Sends during a command window park in the /send route
and dispatch full-fidelity afterwards — the interjection queue (length
cap, cross-user guard, identity-swap hazards) is unreachable there — with
a compaction-aware client abort bound shared by both panes. compact_now()
carries send()'s full generation discipline; Stop aborts the in-flight
summary HTTP stream via a generation-scoped cancel ref; force-abandoned
compactions retire at their next checkpoint and their stragglers are
fenced off every surface (panes, pill latch, CLI). Every session retry
backoff is cancel-aware via one shared helper. Docs, OpenAPI spec, and
both SDKs updated.
Verified: 9457-test non-live suite, JS pin suites, headless-Chrome
reducer harness; five unprimed multi-agent review rounds (correctness
trend 15/6/6/4/4) with plan-level design passes on every fix round.
str | None under-specified the field: the implementation and the TS SDK
union both constrain it to queue_full / refused_closed, and the Literal
projects a proper enum into the generated OpenAPI spec so clients
reject unexpected values. Specs regenerated.
Wake path:
- Denial metacog nudge moves to the tool channel so it drains with the
denied tool batch instead of the next user-message seam.
- wake_workstream_if_pending: shared wake gate for watch fires on
already-idle workstreams (no IDLE transition for the watcher to
observe), wired as wake_fn at every set_watch_runner site via the
shared _watch_fire_wake_fn helper (closes over the Workstream OBJECT
— after eviction+restore an id-keyed manager lookup would miss).
- session_worker exit backstop re-runs the wake gate the moment worker
ownership clears: IDLE fans out on the worker thread, so
transition-time wakes always landed on the reuse path and no-op'd
(the coordinator idle_children strand).
- deliver_wake_nudge_from_queue contains GenerationCancelled — it is
the wake worker's run() closure and only Exception is caught
downstream.
Watch delivery:
- Terminal fires that cannot reach their workstream are HELD and
redelivered on min(interval, 60s) without re-running the command,
bounded by MAX_DELIVERY_ATTEMPTS per cycle and the watch's own
max_polls across cycles; the poll charge commits durably at hold
time so restarts stay budget-bounded.
- Restore admission control: per-ws dedup + MAX_CONCURRENT_RESTORES
cap, presence-only re-check under the lock, detection-only stall
alerts (reclaiming a wedged admission would trade capped degradation
for total poll-pool collapse).
- Permanent-vs-transient restore taxonomy: corrupt persona stamp and
genuinely-missing history (confirmed by a raising storage probe —
the resume loader swallows read blips into []) deactivate the watch
immediately; everything else holds and retries.
- Cancel-race defense: delivery paths re-check is_watch_active before
stashing/dispatching, cancel paths write the row BEFORE
forget_terminal_dispatched, the HTTP cancel endpoint clears runner
state, and a per-tick sweep bounds the residual stash-after-clear
interleaving to one check_interval.
- Abandon/exhaustion commits are write-then-clear so storage that can
read but not write retries the row write instead of re-running the
command every cycle; the fresh-fire unrestorable path stashes before
its deactivation write for the same reason.
Registry follows identity:
- The dispatch registry is keyed by _ws_id at registration time; every
rebind now moves it: non-fork resume() and /new go through
_follow_watch_registration (new key live before the old is removed,
never stealing a registration another live session holds), removals
are owner-checked so tearing down a watch-restore shell or a
resumed-away session cannot unregister a live pane, the restore
shell yields to a registration that appears mid-restore, CLI
--resume registers after the successful resume, and both the open
path and the detail-GET lazy rehydrate wire the registration.
Teardown gating and backpressure honesty:
- cleanup_session_ui marks ws._closed FIRST under ws._lock — every
teardown path (close, close_idle, evict, delete, discard) funnels
through it — and session_worker.send re-checks under the same lock,
so a wake can never spawn a worker on a torn-down workstream.
- Create responses carry initial_message_status when the initial
message could not be delivered (queue_full / refused_closed) instead
of reading as success; staged attachments survive for the retry;
/send surfaces a closed workstream as 404 rather than queue_full.
Docs/spec: OpenAPI artifacts regenerated; api-reference documents the
new create-response field; TS SDK type extended.
Tests: ~30 new pins (cancel races, budget durability across restarts,
owner-checked registry moves, teardown gating, stall alerts,
backpressure surfaces, wait_until final re-check); wide subsystem
sweep green (2353 passed).
Three server.test.ts / server-attachments.test.ts cases still asserted
the pre-verb-lift send shape (POST /v1/api/send with ws_id in the body).
The server route has been POST /v1/api/workstreams/{ws_id}/send (ws_id
in the path, {message} body) since that lift, and the SDK send() was
updated with it — only these expectations rotted. They fail on main
too; unrelated to approvals, folded in here to leave the TS SDK suite
green. Test-only, no SDK source change.
ApproveRequestEvent.cycle_id and ApprovalResolvedEvent.cycle_id /
call_ids were typed required, but they are 1.7 additions: a pre-1.7
server omits them on the wire, so a current SDK talking to an older
node sees undefined. The UI and channel adapters already keep the
legacy no-selector fallback for exactly that case. Mark them optional
so consumer code can't assume a string that may be absent — matching
the Python SDK, whose dataclasses default all three.
The checked-in openapi-server.json / openapi-console.json still
described the removed singular pending_approval_detail field; re-run
generate-types.py so the reference specs carry the
pending_approval_details list + cycle_id. Retire two docstring
references to the deleted singular serializer and one stale comment.
POST /approve accepts cycle_id / call_id selectors and 409s on stale
selectors with the current cycle's ids so clients re-render instead of
silently resolving an unrelated batch. Selector-less bodies pin the
resolve to the cycle the lookup returned (not "whichever is oldest by
the time the resolve runs"), and Approve+Always names apply only after
the pinned cycle actually resolved — the auto-approve whitelist can no
longer describe a different batch than the one that resolved.
approve_request carries cycle_id; approval_resolved carries cycle_id +
call_ids; SSE reconnect replays every live cycle's card. The console
collector, coordinator UI fan-out, and both SDKs (Python + TS) thread
the cycle correlation through.
BREAKING (1.7): the singular pending_approval_detail field is removed
from dashboard rows, workstream detail, and node snapshots — replaced
by the pending_approval_details list (one entry per live cycle, each
carrying its cycle_id). stable/1.6 keeps the old shape.
Spec models now describe what the endpoints do: ListPersonasResponse
declares the tool_inventory the shelf depends on, both console create
models declare persona, CreatePersonaRequest declares org_id, and
UpdatePersonaRequest documents the null-vs-absent split (null clears
base_prompt/tool_allowlist, null on flags/kinds is ignored). Console
OpenAPI regenerated.
Protocol contracts match the implementations: update_persona's return
covers the no-op case, create_persona's raises-list is complete, and
both extended row-shape docstrings gain their tail columns plus the
append-only rule. The workstreams.persona comments say slug, not
display name.
Page corrections from the docs review: personas.md documents the
creative_mode-to-writer migration conversion, the mid-session /resume
MCP-lever behavior, visibility-based nudge gating, the soft-set
prompt-cache cost, and the executive tool list — and drops internal
jargon. The changelog entry moves under [Unreleased] with the house
breaking-marker style and the auto-conversion note. coordinator-skills
and the API tour stop using persona to mean framing; governance,
api-reference, sdk, console, tools, and memory pick up the new
permission family, endpoints, kwargs, picker, and lever caveats.
The persona resolved at creation is snapshotted into workstream_config
(five keys, all-or-none) and applied ONLY from the stamp — the personas
table is never read post-create, so edits/archives never touch existing
workstreams, and a corrupt stamp fails construction loudly instead of
silently reverting to a default envelope. Legacy pre-063 workstreams
carry no keys and keep today's behavior byte-for-byte.
The four levers (turnstone/core/personas.py holds the codec):
1. Base override — compose_system_message(base_override=...) replaces
exactly the BASE module; ENV/CONTEXT/TOOLS/POLICIES keep composing so
mandatory prompt policies ride on top of every persona. This also
closes the old /creative hole where the fork bypassed composition
(no CONTEXT, no DB policies).
2. Tool visibility — the allowlist intersects both the composition name
set (TOOLS block self-suppresses, tool-gated policies drop, the
memory advisory drops) and the END of _get_active_tools so the wire
never advertises hidden tools. tool_search in the set = soft
(discovered tools union with the allowlist via the session's
expanded-names set); absent = hard (the whole pathway is disabled,
covering provider-native defer_loading, which has no synthetic name
to filter). Persona sets force client-side tool search.
3. MCP gate (session-wide) — an MCP-off persona drops the client
reference at construction: no merge into _tools OR _task_tools, no
listeners, refresh callbacks inert, resource/prompt catalogs gone.
4. Memory (own hands only) — no recall injection, memory-directed
nudges suppressed (MEMORY_NUDGE_TYPES; behavioural nudges keep
firing), memory tool hidden. _task_tools is NOT filtered; compaction
spill/markers are never persona-gated, and the post-compaction recall
pointer is emitted only when the recall tool is actually visible.
Threading: the create handler resolves once (explicit name -> 400 on
unknown/disabled/kind-mismatch; empty -> the kind's default; pre-seed DB
-> unstamped legacy) and stamps via constructor kwargs + config keys +
the workstreams.persona column; SessionManager.open threads the stamp
pre-construction exactly like the saved model alias. Non-fork resume
adopts the target's stamp so _save_config can't clobber it. spawn /
spawn_batch gain a persona arg with prep-time validation (children are
interactive-kind; omitted = kind default, never the parent's). Python +
TS SDKs, OpenAPI specs, the picker feed GET /v1/api/personas (authed, no
perm), and console admin CRUD /api/admin/personas (persona.* perms,
archive-only — no DELETE) round out the surface.
BREAKING: /creative is removed (the REPL command now points at the
writer persona); turnstone --persona <name> is the replacement. Also
fixes the CLI session factory, which TypeErrored on the project_id
kwarg the shared InteractiveAdapter passes unconditionally.
* feat(projects): governed project containers — memory scope, grouping, manage UI
A workstream can attach to a project: a first-class, shareable resource
container that owns a `project` memory scope, groups conversations, and is
managed from the console.
Storage / migration 062: projects + project_members tables, workstreams.
project_id, and the memory type default project→general; grants
project.{create,read,write,delete} (admin-default).
Recall + writes: project memory is recalled iff the workstream is attached AND
the user has access (owner ∨ member ∨ public-for-read), resolved once at session
construction; coordinators recall it too. New saves default to the project when
attached + writable; the save and delete paths are write-gated; deleting a
project purges its scoped memory; archived projects aren't recalled.
Access = RBAC capability ∧ per-project ACL (auth.resolve_project_access, a
single-fetch resolver); visibility changes, member management, and delete are
owner-only.
API: project CRUD routes on both the server and console; project_id threaded
through workstream creation, spawn inheritance, the cluster-create proxy, the
dashboard / snapshot / coordinator row builders, and the collector deltas.
UI: a project picker with an inline "+ New project" creator in every creation
box (console launcher + standalone dialog + dashboard); group-by-project in the
rail; a project badge in the composer and on dashboard rows; a console manage
tab (list + create/edit + members shelves). The admin Memories view gains
coordinator/project scope filters and human scope labels (name, not hex). The
memory tool schema documents the project scope and the attach-aware default.
* fix(projects): client refresh hardening, creator race guard, SDK project_id
Addresses PR #724 review feedback plus two bugs found while validating it.
- projects.js refreshProjects: a non-OK status (e.g. 403 when the caller
lacks project.read) or a network/parse error no longer blanks the cache
or masquerades as "no projects" -- the prior cache is preserved, the
failure is recorded (new projectsError()) and warned. Honors the
long-standing "a transient error can't blank the rail" docstring.
- projects.js _fp: the fingerprint separators were raw control bytes,
which made git treat the whole file as binary (no reviewable diff).
Rewritten as escape sequences instead of raw bytes -- behavior is
byte-identical at runtime.
- project_creator.js: createProject() could reject unhandled (authFetch
throws on network/401; r.json() throws on a non-JSON body), leaving the
widget stuck busy/disabled. Added a .catch, plus a generation guard so a
create whose widget was cancelled/reopened mid-flight drops its result
instead of selecting a project the user backed out of.
- types.ts: add project_id to CreateWorkstreamRequest / WorkstreamInfo /
DashboardWorkstream to match the server schemas (was SDK-invisible).
- test_project_api.py: move side-effecting HTTP calls out of asserts so
the requests run even under python -O.
* fix(projects): JSON.stringify the cache fingerprint, drop control-byte separators
_fp joined fields/rows on raw NUL/SOH bytes, which made projects.js read as binary to git. Replace with a collision-proof, escape-free JSON.stringify encoding -- same change-detection semantics, zero embedded control characters.
- Remove the unused PerceptionUnavailableError (never raised/caught/imported).
- Reword the now-shipped 'Phase 3' placeholder comments on the Anthropic + OpenAI-Responses audio paths to describe the live upstream STT/perception fallback (these placeholders are defensive, not pending work).
- Clarify the no-vision image fall-through comment (fires when perception is unconfigured OR can't see, not only the former).
- Type AttachmentInfo.kind as the image|text|pdf|audio union in the TS SDK.
- extract_pdf_text: append the truncation marker only when there's actual text, so a scanned PDF over the page cap returns '' (-> placeholder) instead of a content-free document part.
- composer: accept pdf/audio in the upload picker; client-side kind
inference for the optimistic chip (server classify_upload stays
authoritative)
- admin Models tab: supports_pdf + supports_audio_input toggles (flow
through the field-aware capabilities merge into ModelCapabilities, so
flipping supports_audio_input on an omni alias enables native input_audio)
- docs: AttachmentInfo.kind, AttachmentUpload, and the TS SDK note pdf/audio
* chore: relicense BUSL-1.1 -> Apache 2.0 for 1.6.0
Flips every license artifact in the tree; 1.5.x and earlier remain
BUSL-1.1 per their release-time LICENSE files. Contributor consent
record: #548 (rationale: #546).
- LICENSE: canonical Apache 2.0 text
- NOTICE: new; copyright line + pointer to THIRD-PARTY-NOTICES
- pyproject.toml: SPDX expression + explicit license-files trio
- Dockerfile: COPY the license trio (hatchling needs them at build)
- THIRD-PARTY-NOTICES: BUSL line reworded; bundled-version drift
fixed (KaTeX 0.17.0, Mermaid 11.15.0, hls.js 1.6.16)
- README badge + License section, CONTRIBUTING inbound-license line,
TS SDK package(+lock), example pyproject
- docs/pgbouncer.md: drop stray ':' introduced in #353
* docs: add CONTRIBUTORS.md
* chore: drop LICENSE leading blank line
The apache.org LICENSE-2.0.txt begins with a newline; the SPDX
canonical text and GitHub license templates do not. Use the
conventional form — detection is whitespace-normalized either way.
- A1: add a shared `operator-context` marker to every operator-context row in
both UIs; the retry-skip walk keys on it, so a trailing watch-result /
guard-finding / idle-children card no longer makes retry regenerate the
wrong turn. Pinned by source-grep tests + a headless-DOM self-test.
- A2: wrap get_content's two sync DB gates (get_attachment + the unbounded
ws-scoped attachment_referenced_in_ws LIKE scan) in asyncio.to_thread so a
long scan can't stall the event loop — matching the module's convention.
- A3: add the HistoryEvent attachments docstring bullet; drop the dead
`interjection` class; document the interactive-only system-context label;
correct the SDK attachments-meta docs to {kind, filename, mime_type}
(size_bytes is not carried through the history projection).
Operator-context system turns (watch results, output-guard findings, idle
children, user interjections) carried their kind (_source) and a flattened
text content, but the structured per-kind fields were dropped at every
persist/deliver boundary — so the UI rendered every kind as one generic
operator bubble and the structured watch-result card was lost.
Wire the structured meta through as the single source of truth:
- Storage: new conversations.meta JSON column (migration 060); threaded
through save_message/save_messages_bulk (facade + protocol + both backends)
and rehydrated in reconstruct_turns onto Turn.meta.extra["source_meta"].
- Canonical: make_system_turn carries meta as one _source_meta dict;
turn_from_dict/turn_to_dict bridge it to/from Turn.meta.extra.
- Live + history: widen on_system_turn(content, source, meta) across all
impls + the SSE payload; surface _source_meta -> meta in the /history
projection. SDK HistoryEvent docs note the field.
- Producers derive both the model-facing content text AND the card from one
meta dict, so they cannot drift: render_output_guard_text, build_watch_
reminder carrying output, idle_children and user_interjection metadata.
- Frontend: addSystemContext / renderSystemTurn dispatch by source to the
watch-result, guard-finding, idle-children, and queued-message cards in
both the interactive and coordinator panes; every untrusted field renders
via textContent.
The meta is a leading-underscore key, stripped before the wire (sanitize_
messages and the native mid-conversation path copy only role+content), so the
per-provider wire payloads stay byte-identical. Additive column, no backfill:
operator turns predating it reload as plain text bubbles.
A deep-dive review of the branch surfaced a budget regression, SDK doc
drift, dead code, and stale docstrings. Each was boundary-spiked before
fixing.
- R1 (regression): by-reference document attachments were invisible to the
token budget — _msg_text_chars returned 0 doc_chars for a
{type:document,attachment_id} placeholder, and the comment's claim that
the budget "lands at calibration" was false (calibration discards
doc_chars). Thread the doc size through _attachments_meta (size_bytes, at
both the live-append and reconstruct build sites) and count it in
_msg_text_chars, guarded against double-counting the inline form.
Regression test added.
- F1: the history-DTO schema description and the TS HistoryEvent docstring
still advertised the removed reminders/advisories keys and omitted the
system role; corrected server_schemas.py + sdk/typescript/src/events.ts to
match the shipped shape. The committed OpenAPI JSON snapshots were already
~679 lines stale on main; their regen is left to its own chore branch.
- D1: removed AttachmentBuffer.take() — dead (no production caller; the
commit path uses discard()) and scope-weak (ws_id only, unlike its
siblings) — with its test and the now-orphaned Iterable import.
- O1: 4 docstrings referenced the moved ChatSession._fold_system_turns →
lowering.fold_system_turns.
`man` and `math` duplicated capabilities already reachable through
`bash`; `plan_agent` is better expressed as a `task_agent` running a
planning skill, and carried a large amount of special-case machinery
(plan-review gate, refinement loop, per-kind model routing). Removing
all three shrinks the tool surface and cuts per-call token cost.
Also removed, as dead-once-the-tools-are-gone:
- the `math` sandbox executor (`turnstone.core.sandbox`) and its
`[sandbox]` extra; the eval analyst now runs bash-only
- the read-only `AGENT_TOOLS` sub-agent tool set and the `agent`
tool-metadata key (`task_agent`/`TASK_AGENT_TOOLS` retained)
- the plan-review protocol end to end: the `on_plan_review` UI hook,
`resolve_plan`, `POST /v1/api/plan` + `POST /v1/api/route/plan`,
the `plan_review`/`plan_resolved` SSE events, and their Python SDK /
TypeScript SDK / OpenAPI / frontend / Discord+Slack bindings
- the `model.plan_alias` / `model.plan_effort` settings and the
registry `plan_model` / `plan_effort` routing fields
TOOLS 31->28, TASK_AGENT_TOOLS 13->11; COORDINATOR_TOOLS unchanged.
BREAKING CHANGE: removes the `man`, `math`, `plan_agent` tools, the
plan-review SSE/HTTP/SDK surface, and the plan_* model-routing settings
from the experimental 1.6 line.
GET /v1/api/workstreams/saved returned only ws_id/alias/title/created/
updated/message_count — too little to drive the planned saved-list table
redesign. Add seven fields, all sourced from already-persisted data (no
migration):
- state, kind, node_id: columns on the workstreams table
- model_alias, launch_skill: from workstream_config via LEFT JOIN
- child_count: COUNT of child workstreams via parent_ws_id
- context_tokens: most recent usage_events prompt size for the workstream
- context_ratio: context-window occupancy (context_tokens / model context
window), computed in the handler so the NULL / zero-window cases stay
explicit and identical across both storage backends
context_window comes from a model_definitions join; aliases defined only in
config.toml are absent there, so context_ratio degrades to 0.0 rather than
reporting bogus occupancy. The Python SDK reuses the Pydantic model; the
TypeScript SDK OpenAPI snapshot and hand-maintained interface are updated.
Tests cover the new storage columns (including NULL-when-absent), the
handler ratio math + zero-window degradation, and the SDK enriched
round-trip.
openapi-server.json / openapi-console.json had drifted well behind
build_server_spec() / build_console_spec() — the committed snapshots are
regenerated periodically (via sdk/typescript/scripts/generate-types.py)
rather than on every schema-changing PR, so accumulated additions (skill
parsing, pending-approval items, model-definition CRUD, etc.) had not been
captured. This resyncs both with no code changes.
Lift the conversation-modifying /rewind and /retry verbs out of the body-keyed POST /v1/api/command into path-keyed POST /v1/api/workstreams/{ws_id}/rewind ({turns:N}) and /retry, as make_rewind_handler/make_retry_handler in SharedSessionVerbHandlers (template: make_close_handler/make_cancel_handler), wired on both interactive and coordinator kinds. Closes the last unlifted conversation-modifying surface — coordinator workstreams gain rewind/retry where they had none — and removes the surviving exception to the post-#422 path-keyed URL convention.
Handler shape: auth gate (coord -> admin.coordinator via permission_gate; interactive -> conversation.modify via accepted_permissions) -> busy-gate -> session.rewind(n)/retry() -> always emit clear_ui (incl. rewind-to-zero, carries #503) -> audit (conversation.rewind/retry on both kinds). Retry re-dispatch reuses the shared session_worker.send via a per-kind dispatch_retry closure (hard-reject on busy), not a third hand-rolled thread.
The web /command handler now rejects /rewind+/retry with a pointer to the path-keyed endpoint (BREAKING; 1.6.0aN-tolerant); session.handle_command's branches stay for the terminal CLI. auth.py adds the verbs to both write suffix-sets; Python + TS SDKs, OpenAPI (RewindRequest + server/console specs), the /route/ proxy mounts + audit actions, and coordinator_client all gain them.
Interactive frontend (app.js): the 3 /command POST sites + the hand-typed-slash reroute now hit the path-keyed endpoints; the bare .msg.user rewind selector is kept (matches the server's _find_turn_boundaries, which counts system-nudge user turns). The coordinator frontend rewind UX lands in a follow-up commit (browser-verified).
Tests: route-walk mount/order, /route/ audit rows, required_scope, OpenAPI catalog, SDK body-inspection, and HTTP-level handler behavior (busy-gate, turns validation, clear_ui emit, retry dispatch, audit invocation + swallow).
Patrick Buckley