Files
turnstone/docs/diagrams/04-conversation-turn.puml
T
Patrick Buckley 480a1426b3 Fail-closed history-commit handoff (#1005)
* fix(session): fail-closed history-commit handoff (#981)

The deleted-workstream discovery is now a terminal, ws_id-keyed latch:
keyed conversation commits refuse admission once the durable parent is
gone (convergence finalizers and force-abandon are exempt), history
handoff refuses to mint a proof token so /history fails closed with a
503 instead of silently wiping the pane, and the SSE stream carries a
workstream_gone resync reason. Discarded commits leave a forensic log
of commit keys and roles, never content.

Conversation rows gain a commit_key (migration 071): keyed saves are
idempotent under retry, validated against the full commit identity, and
refused when they would cross a workstream deletion. The prune orphan
category now requires a NULL alias plus a two-hour updated grace, with
cutoffs computed at discovery time and carried into both dialects'
rechecks.

The mid-turn interjection queue is owner-partitioned with no per-site
mode flags: pops take the acting principal's and unowned rows, other
participants' rows are structurally retained, and enforcement lives at
queue admission plus the shared before_spawn gates. The retraction
ledger is bounded by open pop windows: pops open a window atomically
with the queue delete, restores close their ids atomically with the
ledger consume, every other exit closes through one helper, and misses
for unheld ids record nothing. The workstream-gone latch refuses
unattended wakes at all three gates (watcher spawn, claim, delivery
pre-pop), and the retry dispatcher regained its pre-envelope
cancel/error convergence net.

Persistence-state reporting derives through the session bound to each
UI instead of a registry lookup by id that failed open to healthy
during tombstone retention. The dashboard roster no longer re-inserts
ghost entries from trailing activity events, the history tool-outcome
scan tolerates interleaved non-turn rows, and the shared
handoff-deadline handle owns its own retirement.

Single-sourced across call sites: keyed-commit row values, attachment
save wrappers, tail-truncation and conflict-resolution bodies for both
storage dialects; worker-slot lifecycle field sets; the direct-commit
admission frame; queued-row layout accessors; the string-aware comment
stripper shared by every JS harness suite.

Refs #981 #964

* fix(session): sweep handoff fixes to their sibling surfaces

The interactive replay loop treated a system row as a tool-batch
boundary, so every tool result after an interleaved row vanished from
that pane while the coordinator rendered the same history correctly.
Only a conversational turn ends the batch window now, matching the
shared outcome index.

Accepted user turns clear the composer's attachment chips on the same
viewer policy that settles optimistic bubbles rather than on having
matched a local bubble, so a workstream created with an upload no
longer keeps a chip for an attachment the create dispatch already
consumed. The coordinator's raced-Stop arm emits the stream-end hook it
inherits alongside the idle state, leaving no unfinalized bubble or
unflushed tool output. Ending a session surfaces a failure toast when
the request never lands or answers with a non-JSON body.

The per-second persistence reconcile now probes each session without
blocking: a workstream whose generation and handoff locks are held is
skipped until the next pass instead of contending the locks every
commit needs. The one-shot repair that gates workstream creation at
capacity keeps a definite probe — it has no next pass, and the sessions
likeliest to be contended are the ones whose unresolved journals
emptied its candidate list.

Single-sourced: the attachment lane builds its conversation row through
the shared commit-identity builder; the ordinary worker exit releases
its slot through the lifecycle owner; both operator surfaces snapshot
their counters through one non-consuming helper; the replay preamble
loses its per-kind wrappers and its config hook; the browser harness
suites share one brace walker; and each in-flight history attempt is
one record carrying both its abort controller and its deadline.

Refs #981 #964
2026-08-11 04:18:36 -07:00

172 lines
6.9 KiB
Plaintext

@startuml
!theme plain
title Turnstone — Generation-Fenced Conversation Turn
skinparam sequenceArrowThickness 1.5
skinparam sequenceLifeLineBackgroundColor #F5F5F5
participant "HTTP / CLI\ncaller" as User
participant "SessionManager" as Manager
participant "ChatSession" as Session
participant "SessionUIBase" as UI
participant "Accepted-row handoff\n(total live prefix)" as Handoff
participant "model_turn()\n+ lowering" as Plant
participant "ModelAdmission\n(per alias)" as Admission
participant "LLM provider" as Provider
participant "Tool workers" as Tools
database "StorageBackend\n(SQLite / PostgreSQL)" as Storage
== Admission and generation claim ==
User -> Manager : dispatch send on one Workstream
Manager -> Session : bind_acting_user(principal)\nsend(text, attachments, send_id)
activate Session
Session -> Session : refresh immutable ResolvedModelBinding
opt token budget exhausted
Session -> UI : approve_tools(__budget_override__)
note right of UI
This gate precedes a generation claim but carries
a monotonic cancellation witness. Stop cannot be
mistaken for a budget-policy denial.
end note
end
Session -> Session : _claim_generation() → generation N\ninstall fresh cancel event
Session -> Session : plan memory / participant context
Session -> Handoff : admit USER row\ncommit_key + prefix revision
Session -> Storage : ordered durable batch:\nappend canonical user Turn + metadata
note over Session, Handoff
Every accepted conversation row enters this lane before durability:
USER, ASSISTANT, TOOL, SYSTEM, compaction checkpoints, and cancellation
markers. Admission shares the handoff lock with its live UI transition
or history_resync repair event.
end note
note over Handoff, Storage
_commit_for_generation(N) admits bounded live mutations under the
generation lock, then executes immutable persistence closures in FIFO
ticket order. A force successor either follows the whole commit or
prevents it. /history projects durable prefix + pending journal suffix;
durable ACK removes the pending copy without changing the prefix revision.
end note
opt already over the hard context ceiling
Session -> Session : compact before first model call\n(preserve the new user turn)
end
== Model / tool loop ==
loop until final answer and no queued input
Session -> UI : on_turn_start()\nreset per-stream replay buffers
Session -> UI : state = thinking\non_thinking_start()
Session -> Session : _stream_response(N)\nretry + fallback policy
Session -> Plant : model_turn(active ModelLane, Turns,\n tools, cancel_ref, on_chunk)
activate Plant
Plant -> Plant : canonical Turns → provider wire\nrestore ids + repair + lane-specific fold
Plant -> Plant : materialize attachment refs\n(nested perception before outer slot)
Plant -> Admission : acquire(cancel_ref)
activate Admission
Plant -> Plant : resolve per-call backend credential\nfrom lane's pinned ModelConfig
Plant -> Provider : create_streaming(...)
activate Provider
loop normalized stream chunks
Provider --> Plant : StreamChunk
Plant --> Session : on_chunk(StreamChunk)
Session -> Session : check cancel event + generation N
Session -> UI : reasoning / content / info token
end
Provider --> Plant : finish + usage + native blocks
deactivate Provider
Plant -> Plant : drain + re-ingest assistant Turn\nwith serving-lane provenance
Plant -> Admission : release before retry backoff
deactivate Admission
Plant --> Session : ModelTurnResult
deactivate Plant
Session -> UI : on_stream_end()
Session -> Session : generation-fenced result commit:\nappend assistant Turn + token accounting
Session -> UI : on_turn_committed()
Session -> Handoff : admit ASSISTANT row\ncommit_key + prefix revision
Session -> Storage : ordered durable assistant row\n(content + tool mirror + native lane)
alt no tool calls
opt over soft threshold
Session -> Session : cooperative / end-of-turn compaction
Session -> Handoff : admit SYSTEM/source=compaction\ncheckpoint projection
Session -> Storage : append checkpoint summary marker\nwith source watermark
note right of Storage
Full history remains durable. Resume loads
[summary] + rows after the checkpoint.
end note
opt model stopped for compaction
Session -> Handoff : admit USER/source=compaction_resume row
Session -> Storage : append synthetic compaction_resume Turn
end
end
alt queued messages drained
Session -> Handoff : admit combined queued USER row
Session -> Storage : append combined queued user Turn
else truly complete
Session -> UI : state = idle
end
else tool calls present
Session -> UI : state = running
Session -> Session : prepare items + previews\nattach cancellation witnesses
opt one or more items require a human
Session -> UI : approve_tools(items)\nregister independent ApprovalCycle
note right of UI
Parallel agents may own concurrent cycles.
cycle_id / call_id routes exactly one decision;
Smart Approvals may clear qualifying items.
end note
User -> UI : approve / deny selected cycle
UI --> Session : decision + optional feedback
end
Session -> Tools : execute admitted items in parallel
activate Tools
Tools --> UI : chunks + result card\nwith effect disposition
Tools --> Session : outputs / errors / effect statuses
deactivate Tools
Session -> Session : output-guard evaluation\nthen generation N re-check
opt compaction owed before result sizing
Session -> Session : compact, preserving assistant tool-call Turn
Session -> Handoff : admit SYSTEM/source=compaction checkpoint
Session -> Storage : append checkpoint marker
end
Session -> Session : one generation-fenced batch:\nappend all Tool Turns, advisories, feedback
Session -> Handoff : admit FIFO TOOL rows\ncommit keys + prefix revisions
Session -> Storage : FIFO durable tool rows + metadata
end
end
== Stop / force-successor boundary ==
User -> Session : cancel()
Session -> Session : atomically set generation event; snapshot\nmain stream, child scopes, judges, subprocesses
Session -> Provider : close live stream handle
Session -> Tools : abort child scopes + kill subprocess groups
Session -> UI : resolve only cancelled operation's\napproval cycles
opt cancellation produced accepted conversation rows
Session -> Handoff : admit partial ASSISTANT and/or\nsynthesized TOOL cancellation markers
Session -> Storage : idempotent keyed cancellation rows
end
note over Session, Storage
Every later publish/commit checks generation ownership. An abandoned
worker may unwind, but cannot append Turns, overwrite state, resolve a
successor approval, or repaint the successor UI. Observed tool effects
are preserved as controller-authored cancellation receipts; unreviewed
tool bytes are not laundered into model context.
end note
deactivate Session
@enduml